|
coldclimb
Nov 4, 2004, 11:18 PM
Post #1 of 28
(4549 views)
Shortcut
Registered: Jan 14, 2002
Posts: 6909
|
All day today the site has been taking over 30 seconds to load any page, usually over a minute, IF it manages to load them, and it never actually completes loading, just sits there with the status bar showing that it's still going. It's also not remembering which topics I have read, as of yesterday morning. Any idea what the problem is here?
|
|
|
|
|
tim
Nov 4, 2004, 11:34 PM
Post #2 of 28
(4549 views)
Shortcut
Registered: Apr 4, 2002
Posts: 4861
|
NFS hiccup. I've been complaining for months about the garbage we run for hardware. Now it's your turn.
I **think** I may have tuned away the problem, but it was pretty gruesome.
A load of 74.6 is not a good thing (where a load of 1.0 means a machine is performing at 100% of its processors' capacity).
Next time someone bitches about ads and such, ask them if they prefer this simulation of the no-funds version of rc.com. We serve almost a million pages a day, and we need to upgrade our hardware. Soon.
|
|
|
|
|
abalch
Nov 4, 2004, 11:35 PM
Post #3 of 28
(4549 views)
Shortcut
Registered: Jul 11, 2003
Posts: 179
|
Probably your own ISP. I have not had any problems today, but I have been surfing on my company's very fast connection. :wink:
|
|
|
|
|
tim
Nov 4, 2004, 11:41 PM
Post #4 of 28
(4549 views)
Shortcut
Registered: Apr 4, 2002
Posts: 4861
|
The problem was between the two machines, which are connected by a crossover cable (no switch). They were dropping packets like mad, and the machine that handles most of the web traffic wedged itself solid.
This appears to have happened starting at 12:30PST and ending a few minutes ago, after I kicked the wedged box several times, and changed various settings that seemed to be causing the system instability.
|
|
|
|
|
tchamber
Nov 4, 2004, 11:47 PM
Post #5 of 28
(4549 views)
Shortcut
Registered: Jun 23, 2004
Posts: 320
|
In reply to: Probably your own ISP. I have not had any problems today, but I have been surfing on my company's very fast connection. :wink:
so have I and the site is still slower than G.W.Bush doing 3rd grade math
|
|
|
|
|
cliffhanger9
Moderator
Nov 5, 2004, 12:13 AM
Post #6 of 28
(4549 views)
Shortcut
Registered: Nov 26, 2002
Posts: 2275
|
In reply to: Tim, What sort of overall cost are we looking at for the necessary upgrades? I'm not wealthy, but perhaps with an idea of the monies necessary we can get some folks together to pool resources and make some magic happen. SKibs
1 million dollars...?
http://us.ent4.yimg.com/.../mike_myers/evil.jpg
|
|
|
|
|
coldclimb
Nov 5, 2004, 12:48 AM
Post #7 of 28
(4549 views)
Shortcut
Registered: Jan 14, 2002
Posts: 6909
|
Do you know why the forums have not been marking topics as read until I click the "Mark all topics read" link?
|
|
|
|
|
johnson6102002
Nov 5, 2004, 1:33 AM
Post #8 of 28
(4549 views)
Shortcut
Registered: Aug 23, 2004
Posts: 843
|
In reply to: In reply to: In reply to: Probably your own ISP. I have not had any problems today, but I have been surfing on my company's very fast connection. :wink: so have I and the site is still slower than G.W.Bush doing 3rd grade math :lol: :lol: :lol: :lol: I concur!
ive been having the same problem today so its not just him
|
|
|
|
|
climballnight
Nov 5, 2004, 1:44 AM
Post #10 of 28
(4549 views)
Shortcut
Registered: Jul 12, 2004
Posts: 102
|
I'd like to take a second just to say thanks to Tim and the team for keeping things up and running, dispite what they have to work with.
I'm glad they know what they're doing because when I talk about "packet loss" it usually involves yelling "Rock!" and going hungry for lunch. :wink:
- bret
|
|
|
|
|
tim
Nov 5, 2004, 1:58 AM
Post #11 of 28
(4549 views)
Shortcut
Registered: Apr 4, 2002
Posts: 4861
|
No worries, it's good practice for my Real Job ;-).
As far as expense, I'd like to think that we can get away with a few thousand spread over the next 14 months, that's the budget I sent Mike. In actual fact, we can probably cover all planned upgrades for the next year, for under $2K, if we finance the server upgrade and pay for it monthly instead of all at once. That is what I have been lobbying for.
We can survive OK for a while longer without spending anything, but if the pace of growth continues to pick up like it has, the best thing we can do is replace the frail, pathetic webserver with a Dell 1U (or, in an ideal world, a dual Opteron with cheapo SATA RAID and a ton of RAM), and beef up the existing database server (which also works part time as a web and mail server). These upgrades are not cheap and I don't want to run around with a collection plate if it turns out that ad revenues and/or promotional fees will cover it.
But feel free to buy partnerships if you haven't, folks. :-)
|
|
|
|
|
climbtothebeet
Nov 5, 2004, 3:05 AM
Post #12 of 28
(4549 views)
Shortcut
Registered: Jan 31, 2004
Posts: 232
|
it seems like the internet is slow all over 2 day.... its all the democrats fault, they invented teh internet right... o wait that was just al gore....
|
|
|
|
|
tyify
Nov 5, 2004, 3:46 AM
Post #13 of 28
(4549 views)
Shortcut
Registered: Apr 22, 2003
Posts: 905
|
It is definently moving to a slower beat than I'd like it to be...thats for sure. However it's only taking 2-4 seconds to load pages for me. What ISP you on Coldclimb? I know Michaelmigure is on the same one I am as he has posted from my dynamic IP before.
|
|
|
|
|
tgreene
Nov 5, 2004, 7:41 PM
Post #14 of 28
(4549 views)
Shortcut
Registered: Oct 22, 2003
Posts: 7267
|
If you're not a paid partner, then you have no right to bitch!
|
|
|
|
|
tim
Nov 5, 2004, 8:18 PM
Post #15 of 28
(4549 views)
Shortcut
Registered: Apr 4, 2002
Posts: 4861
|
Here's an idea I had, after looking around and comparing the various system specs for server-grade machines these days. For around $1700, it appears that we could upgrade from the pathetic $400 junker that I assembled about a year ago (which now handles 60% of web traffic for the site...) and upgrade to a sickeningly fast dual Opteron box. Compared to the poor I/O performance of a Pentium- or Xeon-based machine at a similar price point, it seems like a no-brainer.
So here is a question -- does anyone out there have experience with a 1U (or inexpensive 2U) rack-mountable Opteron server running MySQL? The machine that does not handle read/write database duties -- currently, the $400 piece of trash -- does handle queries under the new search engine. It was the only way to bolt together a serviceable forums search that did not slow down the rest of the site. If we can afford an Opteron server with a quality main board and near-limitless RAM for expansion, we could make this site SCREAM on read requests...
Anyone have any experience with Opterons? Any downsides to them at all, or is it really as simple as buying one and having everything go faster?
The more I look at this chip and chipset, the more I think we can upgrade painlessly to handle 4-6x the traffic we currently have. That would give me an awful lot of headroom... and with a 64-bit processor, we could install enough RAM to run a copy of the site directly out of main memory. That would improve performance to Google levels (because that's exactly what Google does for most index queries, as it happens :-)).
|
|
|
|
|
tgreene
Nov 5, 2004, 8:26 PM
Post #16 of 28
(4549 views)
Shortcut
Registered: Oct 22, 2003
Posts: 7267
|
I can't help ya there Tim, but I would suggest you give the folks at Micron a shout...
Micron is an awesome company to work with, and since they don't spend a fortune on advertising like the big 3, they are one of the cheapest solutions out there! I have been using and suggesting Micron to my clients for years, and so far, everyone has been super happy with the products and level of service.
* Who knows, you may even consider hitting them up for some form of advertising trade-out, since they are a really progressive Idaho based company. :wink:
http://www.mpccorp.com/
Oh, another thing to keep in mind is, the prices shown on the website are VERY negotiable. All you have to do is mention Dell, and they'll start chopping those prices down to bare minimums! :idea:
|
|
|
|
|
tarzan420
Nov 5, 2004, 8:35 PM
Post #17 of 28
(4549 views)
Shortcut
Registered: Jun 19, 2002
Posts: 678
|
In reply to: Anyone have any experience with Opterons? Any downsides to them at all, or is it really as simple as buying one and having everything go faster?
Sure. (I'm writing this from my dual opteron desktop... i know it's a waste...)
for the most part, it is that simple. Now, i'm running as a desktop box, so there are quirks with that, but i don't think there are any issues with apache/sql stuff. While running stuff in 64-bits is nice (gentoo on this puppy...) 32 bit stuff still seems pretty snappy.
In reply to: If we can afford an Opteron server with a quality main board and near-limitless RAM for expansion, |
|
|
|
|
tyify
Nov 6, 2004, 8:41 AM
Post #18 of 28
(4549 views)
Shortcut
Registered: Apr 22, 2003
Posts: 905
|
And to think I was excited today when I upgraded to a 3200+ AMD chip...16 gigs of ram JeBuS!
And Tgreen does that mean I get to bitch? :lol: :lol:
|
|
|
|
|
tim
Nov 7, 2004, 4:59 AM
Post #20 of 28
(4549 views)
Shortcut
Registered: Apr 4, 2002
Posts: 4861
|
In reply to: And to think I was excited today when I upgraded to a 3200+ AMD chip...16 gigs of ram JeBuS!
I'd take a Commodore 64 with SCSI drives and multiple gigs of RAM over a fast processor in a shit box desktop. As Ken Batcher always put it,
In reply to: A supercomputer is a device for transforming a CPU-bound task into an I/O-bound task.
And all of these multi-gigahertz CPU cores are effectively supercomputing processors. Which begs the question -- with all of these supercomputer CPUs in cheap servers, why haven't we got a decent I/O interconnect design? The answer, as best as I can tell, is in the recent AMD designs:
an article on the Opteron's interconnect
My first real job was in a supercomputing center, where I learned (among other things) that if you don't have a lot of fast memory to feed the processors, they just sit there. I don't like things that just sit there. I think we should probably get an Opteron system if it's affordable for us; I checked with the MySQL people and they love the Opteron to death. The computational biology people at USC just got a 64-node 8-way Opteron cluster and they say it's great. I can't seem to find a downside.
|
|
|
|
|
g-funk
Deleted
Nov 7, 2004, 7:32 AM
Post #21 of 28
(4549 views)
Shortcut
Registered:
Posts:
|
In reply to: So here is a question -- blah blah blah, blah blah blah. . .blah blah blah blah $400 piece of trash -- does blah blah blah, blah blah blah blah blah blah blah we could make this site SCREAM on read requests... Anyone have any experience with blah blah blah? Any downsides to them at all, or is it really as simple as buying one and having everything go faster? The more I look at this blah blah blah blah blah blah blah Google blah blah blah blah blah :-)).
You guys are speaking jibberish, but I'm glad that somebody out there can :lol: Keep up the good work.
|
|
|
|
|
mistertyler
Nov 7, 2004, 7:59 AM
Post #22 of 28
(4549 views)
Shortcut
Registered: Dec 9, 2003
Posts: 197
|
Tim -
Just wondering what flavor of Linux this site is running? It used to be on FreeBSD, right? Did Linux end up being faster?
Just curious --
|
|
|
|
|
tyify
Nov 7, 2004, 9:25 AM
Post #23 of 28
(4535 views)
Shortcut
Registered: Apr 22, 2003
Posts: 905
|
I've got a gig of ram in mine so it runs well!
|
|
|
|
|
tgreene
Nov 7, 2004, 2:20 PM
Post #24 of 28
(4535 views)
Shortcut
Registered: Oct 22, 2003
Posts: 7267
|
In reply to: In reply to: I can't help ya there Tim, but I would suggest you give the folks at Micron a shout... I'd do that, but they don't sell Opterons. If we go for a 1U x86 I'll look into them, but they'd better have redundant power supplies in that case :-)
That bitch wasn't goin down for nuthin! :twisted:
|
|
|
|
|
tgreene
Nov 7, 2004, 2:23 PM
Post #25 of 28
(4535 views)
Shortcut
Registered: Oct 22, 2003
Posts: 7267
|
In reply to: I've got a gig of ram in mine so it runs well!
When running under Windows, the biggest thing to keep in mind, is to turn off all of the bullshit features that Micro$oft has force fed to us. If you operate XP as if it were Win95, you'll automatically see a phenominal performance increase, and will rarely if ever receive Out Of Memory errors.
|
|
|
|
|
|
1
