Positive Fusion

This is just a reminder, mod_security cannot currently be disabled via .htaccess files, doing so will result in an error code 500. This is a change in the server software, not a policy change at Positive Fusion.

If you’re seeing a 500 error where you’d not normally have on the old server, that would be the first thing to check.

If you need mod_security to be disabled for CMS posting pages and similar please let support know and we’ll disable it for you. Systems like Wordpress should already be covered within the main server configuration exclude directives.

It has been a very long time that we’ve used Mod_Security filtering. Within the past few weeks we have modified and updated some rules. Due to these changes some users have experienced 406 errors for the first time.

If there’s a 406 error, it simply means that our rules we are filtering with have caught something listed as objectionable. It’s rare that 406 errors will effect the typical website visitor, unless of course they’re trying to do something bad. Most often a 406 error will result when a user is posting something using a content management system, or installing software for the first time.

We do encourage you to let us know when you experience problems, because if a rule is causing a lot of false-positives we’d be better off not using that particular rule.

You, the user, can actually turn off mod_security filtering yourself by using standard .htaccess syntax. This method was first described here in relation to Wordpress, but the method applies to any file on your website.

Create or edit an .htaccess file within the particular file’s directory, enter the following (this is an example for WordPress):

If the file is index.php, for example, place that filename in the directive instead.

We can also tag specific rules with an ID code, in that manner we can give you a code to turn off that particular rule, while allowing all other mod_security filtering to continue.

There was yet another severe comment spam attack this afternoon. We were already closely monitoring the webserver and as such were able to mitigate the attack fast enough as to avoid even tripping the uptime monitoring.

This graph shows the bandwidth effect during such an attack, as is visible the average inbound flow averages at approximately 300kbps, during the attack inbound flow spiked to 3.8mbps. It is important to note that for the purposes of this graph, inbound and outbound is reversed (inbound to the switch means outbound from the webserver itself).

This graph helps to illustrate how intense the load spike can be when traffic increases in such a massive way, particularly when considering this traffic is hitting dynamic and cpu-intensive Perl/cgi scripting.

Essentially we’re working diligently to stop the spam in any way possible, but without impacting negatively legitimate users. If we stop the spam before it gets to comment and trackback processing, we keep the server’s load where it should be, versus having the load averages skyrocket because anti-spam scripts cannot cope with such a heavy influx. Movable Type authors have admitted to serious bugs with their commenting/trackbacking systems that have brought webservers to the ground as a result of comment/trackback spam.

It may be difficult to imagine that spam can cause such havoc, but it really does. When there is a spam attack focused on multiple hosted websites (as is almost always the case), there are hundreds of comments coming in at that very second, every second, for an extended period of time. The server will do its absolute best to process every request, even if that request ends up hitting the rejection queue of a content management system. This in turn drives the server’s load up to fantastic levels.

Under normal load, that is when people are actually looking at websites and making comments versus automated scripts, the system runs well under capacity. It is only when these attacks happen, which brings about a denial of service, that problems begin to occur. In order to have the processing power to keep the system active during an attack, nearly every website would need its own dedicated server as powerful as this one, capable of running hundreds of sites at once.

At this time we are not banning usage of Movable Type versions prior to 3.x. The commenting/trackback systems in 2.x versions are known to be problematic, but we also know that upgrade/license fees may be prohibitive to some. It is our firm recommendation that users of Movable Type either purchase their upgrade licenses to version 3.x or consider migrating to another system such as Wordpress. At this time there have been no recorded incidents of comment/trackback attacks on a Wordpress site severe enough to cause Apache to fail; the same cannot be said of Movable Type.

We noticed an increase in overall system load after implementing the enhanced blacklist. We have since isolated mod_security actions for that blacklist to dynamic content only, be that .cgi, .php, or .shtml.

If you have used mod_rewrite to force files without the .php extension to process as PHP you will also need to make sure an AddHandler for “application/x-httpd-php” is also included in your rewrite rules, otherwise mod_security will not filter for you. There are few instances where this will occur. This does not relate to “clean URLs” in content management systems, for example, even with “clean URLs” enabled within WordPress, comments are sent to wp-comments-post.php; our global handler will already interpret this as dynamic for the purposes of mod_security.

As mentioned previously, there is a way to opt out of the mod_security filtering. We have automatically turned it off for b2 and Wordpress entry posting, for the time being Movable Type users can turn it off manually using the .htaccess method if the rules prevent entry posting for them.