Positive Fusion

There was a period of heavy load about twenty minutes ago due to an experimental bot that ’should’ follow robots.txt files, but of course didn’t, because few actually do. This ‘experimental bot’ called Twiceler has now been banned from accessing the server by IP and useragent.

Abuse, regardless of where it comes from, will not be tolerated.

This morning’s downtime was brought to you by the overzealous bots of Yahoo/Inktomi. This isn’t the first time that Yahoo/Inktomi search engine bots have caused server downtime. In fact, nearly all of the recent downtime has been caused by the bots of Yahoo/Inktomi pummelling the server incessantly. Yahoo/Inktomi needs to learn that sending 200 bots at once to attack a webserver is very bad form. They won’t ever get to be Google-like by annoying server administrators. Microsoft needs to learn that same lesson it seems.

The Inktomi 74.6.0.0/16 Class B has been blocked. Just because it’s a corporation and not some script-kiddie doesn’t make their abuse right.

The server is being bombarded with requests, it looks like there’s a search engine bot that has gone completely haywire. We’re working on getting the situation under control.

11:40 EDT
We’re rebooting the server. It has been impossible to login due to the bombardment of the attack. We have; however, identified the culprit, but given the nature of the attack, blocking will do little unfortunately. Service will be restored momentarily.

Some might be curious as to what the level of spam email versus legitimate email is, well here’s that statistic for the last two days.

Rejected as Spam: 42978
Delivered as Legitimate: 10963

The server has been being bombarded with spam for the past hour. This is resulting in high load averages and sluggishness. Unfortunately the attack is distributed and thus difficult to block fully.