Bug Report for New Server, 10/16/2008

[tgr pro]

YOu know given that the site has been glitchy as shit for the last few months, i think that the dns errors are likely less invasive than just a dysfuncational server. and this way we can watch the resources as we migrate with an awake head. Plus Rackspace has their best techs on during the day watching the db.

Timeout errors can be mostly attributed to a bad configuration on the switch between the database server and live server. moving to windows 2008 helped us identify that as the cpu is running at 1/3rd the level as it was on 2003. Additionally, once we replaced the switch, we were able to run queries on the mysql database and optimize some indeces on a few tables.

[rideit]

I think I need to ride Jimmy's Mom tonight.

[tgrweb]

email was broken. now its not!
We're alive! ignore the ip link.

[jSki]

FUUUUUUUUCK, logged me out while I was writing this and I forgot to copy my response before posting!!!!

The new server is much faster but about 10% of requests are still taking > 15 seconds. While I fully appreciate and understand how much effort goes into maintaining this site and fielding stupid questions about things like back button issues, there performance issues are very difficult to deal with. I'm very surprised that this site is running on a single server. Setting up apache/php and mysql clusters are trivial as long as you have the rack space. Then when performance slows you just add another server. Clusters were made to solve this exact scalability problem. Also, if a server goes down the others are there to take up the slack.

Is the reason the site goes down every night at 11 to do backups? There is no reason to take a site down to do backups. There are ways to do a hot backup of a mysql database, and at worst you could set the database to read only while copying the database to disk for a backup.

If a business had a mission critical site that ran this poorly they would fail. Half the time when the requests take >5 seconds I go to a different site. There are plenty of free and simple tools that will allow simple management of database and apache clusters that will solve the problems this site is experiencing. Feel free to PM me and I can recommend some documentation on this if interested.

[tgr pro]

thanks jSki. It's not apache, or lack thereof, that's causing these hangups. cpu on the webserver was at about 20% when this post came through. We're committed to rooting out these issues, though.

cheers,

[tgr pro]

Thanks for your feedback though- we're going to run some DB optimizations again tonight, as well as install a new RAID array on the database server. As a side note- we do have a separate DB server- not a single server.

[EarlyWood]

I just tried to post a pic as an edit in gear swap and got this

"500 - Internal server error.
There is a problem with the resource you are looking for, and it cannot be displayed."

I'll try again...just thought this might mean something specially since someone just pmd me and said they couldn't pull up my attachment from another post.


edit: yeah, it's not allowing me to upload a resized pic. I don't really want to start a new post to try the function out. What's up with that?
okay maybe I'll do a new post and then delete so's not to interfere with the last poster wanting to find shit..


EDIT#2
NOPE it's not allowing me to download in a new post either. FUUUUUCK.. do you know how long it took me to get the pic post thing to begin with???!!

Nice work on the increased speed though

[beaterdit]

Same here, can't attatch pics.

"500 - Internal server error.
There is a problem with the resource you are looking for, and it cannot be displayed."

Hope you guys can get this resolved, it's not not to have to host photos elsewhere, esp if it's just one or two.

Speed seeems good now though, thanks! We do appreciate you!

[FrankZappa]

Is it just me or did you guys loose every pic I put up here?

-or maybe they're all in limbo for a while.

[nick > jesus]

picture attachments have not been working for me. It shows the box where the photo should be, but it's just blank

[tgrweb]

good times at night!
Fixed!

[nick > jesus]

looking good!

thanks for all the upgrades

[soulman978]

page loads are still rediculously long for me. running xp on one machine and vista on another. I love this place but man, it's been teh suck lately.


edit: it took 30 seconds for this post to load after i hit submit.

[tgr pro]

just tweaked the db server. let me knowif it still sucky.

[soulman978]

much better today.

[summit]

When I hit my bookmark,
http://www.tetongravity.com/forums/search.php?do=getnew

It shows the search for a moment but then I get sent to:
http://www.tetongravity.com/forums/profile.php?do=dst

And get this message:

vBulletin Message

Your submission could not be processed because a security token was missing.

If this occurred unexpectedly, please inform the administrator and describe the action you performed before you received this error.

I have to then hit it again to have the search happen and it is displaying 1hr ahead on the posting times.

I've tried logging out/in and clearing the cookies.

I cannot get in to the User CP period. It shows for a moment but then I get the above error every time.

This occurs with FF3 (latest) and also IE6

[advres]

I have been getting this as well all morning. It happens with FF3 and Safari here.

[nesta]

page loads on this forum didn't used to be so slow for me. Everytime I click on something it takes at least 15 seconds to go to what I clicked on

[nesta]

BTW, that "quick reply" took over a minute from when I clicked on "Post Quick Reply" until I was redirected back to the thread. What gives?

[The Gnarwhale]

Veeerrry slow page loads today. Posting a reply times out and/or takes repeat resends.

Thanks for workin on it guys!

[tgr pro]

ooops- that was a bad query in the new Proshop,.

[mrryde]

When I hit my bookmark,
http://www.tetongravity.com/forums/search.php?do=getnew

It shows the search for a moment but then I get sent to:
http://www.tetongravity.com/forums/profile.php?do=dst

And get this message:


I have to then hit it again to have the search happen and it is displaying 1hr ahead on the posting times.

I've tried logging out/in and clearing the cookies.

I cannot get in to the User CP period. It shows for a moment but then I get the above error every time.

This occurs with FF3 (latest) and also IE6

getting this error all morning too

[Tunco]

This happens to me as well, in FF, Chrome and Opera.

When I hit my bookmark,
http://www.tetongravity.com/forums/search.php?do=getnew

It shows the search for a moment but then I get sent to:
http://www.tetongravity.com/forums/profile.php?do=dst

And get this message:


I have to then hit it again to have the search happen and it is displaying 1hr ahead on the posting times.

I've tried logging out/in and clearing the cookies.

I cannot get in to the User CP period. It shows for a moment but then I get the above error every time.

This occurs with FF3 (latest) and also IE6

[bklyn]

This:

http://www.tetongravity.com/forums/profile.php?do=dst
vBulletin Message

Your submission could not be processed because a security token was missing.

If this occurred unexpectedly, please inform the administrator and describe the action you performed before you received this error.

Is the bane of maggoty existence. Make it stop please.

[skuff]

Well...I've fed and watered the squirrels, turned off the Pong game and I'm still getting thirty second load times using xp. Hmmm...maybe I'll try substituting their water with Gatorade.