Is it just me, or has the site slowed down tremendously for the last few days. It now takes about 5 minutes to load up a page.

 

I havnt noticed that at all

 

The initial response from the server is quick, and the loading of about 20% of the data is about normal from where I am, then there is about a 50-80 second...



















...delay until the remainder of the page content loads.

That second part of the loading



takes place in





several bursts

as the data is




assembled.

Off hand it looks like the problem/delay is either the SQL server or the link between the SQL server and the rest of the chain.

Been like that much of the day.

 

Someone is consuming resources... can't nail it down. I reset the SQL server and restarted Apache -- it isn't the server's fault. I looked into mySQL status and there are a ton of locked requests... someone is performing some kind of denial of service attack on the forums. The fact that they are still running at all is a miracle.

I have noticed this happening about once a week for several hours. Not sure where the traffic is coming from, but it always seems to stop after a while.

You can view the SQL server load graph here:
http://db1.scionlife.com/isu/graph.p...param=CPU_Load

 

just got a message saying something about "malware" from MSN. ???

 

Would not let me post the first time I tried responding to this thread.............

 

I was resetting the servers again.. that was my fault

 

okay cool, it's loading faster now that you reset the mySQL

 

Do the "attacks" coincide with visits from any of the search engine web crawler 'bots? Just a thought. I know that mine take a little bit of a hit from Google, and also from MSN - which my "robots.txt" file specifically forbids access to... (Guess the MSN 'bot can't read.)

 

Hmmm... I don't track them, but that would make sense! Wow... now how would I know if those are the bots or not?

 

Well, for example, these are the TOP two resource users at one of my servers today:
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
msnbot/1.0 (+http://search.msn.com/msnbot.htm)

(That's from the log of which "browsers" were used to access my site.)

This one was the fifth largest resource user:
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

MUCH farther down the list are several more:
SBIder/0.8-dev (SBIder; http://www.sitesell.com/sbider.html; http://support.sitesell.com/contact-support.html)
Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html)
Gigabot/2.0/gigablast.com/spider.html
Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)
SurveyBot/2.3 (Whois Source)

There are probably some others on today's list I don't recognize.

(Here's the chart from my server on the OS used for access today



(The links the 'better' spiders have on their ID plates give a lot of good info on how to talk to them. They also tell how to get them to give you a break if they are digging too hard, but you still want to be indexed. Going by the info from the good ones, you can usually figure out how to smack the others on the nose to get 'em to back off, too.)

Sites with lots of changes/additions can really get brutalized by bad 'bots.

 

I'm asking my server tech if there are any reports that could help me pinpoint that on my server.

 

OK, I was able to locate stats on which spidrs had visited the site, and Yahoo's spider had been doing some pretty intensive crawling. I edited my robots.txt to excludemany of the stpuid spiders and to slow down Google and Yahoo a little.

Also, I tweaked my SQL settings and realized a MAJOR speed increase! Now the entire database is in memory and I think SQL may have sped up about 10x.

 

Nice. Thx. Slow site was annoying!