Aggressive MSN Search Bots

Ickie

Ickie

SOH Administrator
For months now I have been fighting high server loads, and you get "the server is busy messages", well I finally figured it out, it was caused by (Aggressive MSN Search Bots)
I finally got the loads way down to below where I have ever seen them. I killed the beast!

Google on the other hand doesn't seem to bother our loads so I left them alone.
Below is a graph of our server loads for the past month and you can see the results. You will also see spikes and these are when I killed another IP owned by M$.

View attachment 89794
 
For website owners:
for a 2 second delay in bot searches add to robots.txt: (Crawl-delay: 2) or Crawl-delay: 2 , I did both

ban IP range:
157.54.0.0/15
157.56.0.0/14
157.60.0.0/16


​
</STRONG>
 
seems to be a waste of time asking MS, heck it took only seconds to block them altogether once I figured it all out.

I only hope I helped someone else who is wonering why their server runs slow or their bandwidth is being ate up by these out of control bots.
 
Microsoft is supposed to work to prevent DDOS attacks from exploiting vulnerabilities in their software. They are not supposed to be the ones initiating the DDOS attacks!

Sounds like MS needs to get control of some of their software engineers else they might be exposing their company to serious legal jeapardy.

Ken
 
what really burns me, is that our forum software vbulletin has ip ban and I put the ip range in there and msn blew right through there. it didn't even slow them down.

next our forum software has a setting for search engine friendly and I had searches turned off, again they blew right in.

but what really pissed me off is .... our library, we have it set up so you have to be a free member to use it, ... it is tied into our member database, lol, again msn just let anybody download from us.:icon_eek:

between this and the massive searches, brought us to our knees daily, it was a real job managing our server loads, not to mention our bandwidth which the charter membership paid for, not them.

I would feel better about this if only they shared their $$$$ they get from advertising on us with us. :stop:

View attachment 89801
they are now blocked at the servers front and back doors!!
 
Ickie said:
seems to be a waste of time asking MS, heck it took only seconds to block them altogether once I figured it all out.

I only hope I helped someone else who is wonering why their server runs slow or their bandwidth is being ate up by these out of control bots.
Click to expand...

Do they respect ROBOTS.TXT files?

- Ivan.
 
good question, I don't really know, haha


but they cannot get through my security scripts at the server front and back door, and my graphs prove it.:icon_lol:
 
You must log in or register to reply here.
Back
Top