What's all this Spam?

[Yash]

http://www.networkworld.com/news/2006/110806-image-spam.html

Good read. Spam levels are soaring across the board. We are working to improve our spam filters (and this is taking time), but I'm posting this article so customers know its not that we are selling customer emails to spammers (as a couple of you have hinted).

We have been working extensively with the devolopers of DSPAM to implement a global, clustered spam catching system that learns from customers. In our tests, DSPAM is catching 95% of spam with just a few days of training.

We hope to have this system in place within 3 weeks. We understand the current SpamAssasin spam filtering system we use isn't as effective, as it cannot evolve fast enough to the strategies of spammers

Most of you think our current spam filtering system is not working at all. But just to give you an example, Mail3 is filtering 170,000 spam emails a day. So yes, while its not perfect, SpamAssasin is still keeping alot of spam emails out of your mail boxes. We know there is scope for improvement, thats why we are investing in DSPAM

 

[Yash]

this email will be going out to customers shortly:

Dear Customers,

Many of you have been wondering about the large increases in spam hitting our servers over the last 6 to 8 weeks. We’ve had a number of customers complain of a huge increase in the amount of spam they are receiving. I’d like to emphasize on the fact that this increase in spam is not due to any flaw in our systems, but because of an industry trend that saw a massive increase of spam coming in for the month of October, and now into the month of November. There are a number of internet news reports about this massive inflow, here is one customers can read to catch up on this new trend:
http://www.networkworld.com/news/2006/110806-image-spam.html

This huge increase in spam resulted in one of our mail servers – mail1 being overwhelmed back in early October, and forced us to undertake a series of hardware upgrades and disk array changes to correct the issue. Since then, Mail1 has been performing well. However, the amount of spam coming in hasn’t died down.

Our current spam system is based on open-source SpamAssasin. SpamAssasin is a rule-based spam filter that needs constant tweaking and can generate enormous amounts of load fighting spam. It has been ineffective in identifying a lot of the new spam our mail systems have been receiving, because it isn’t able to dynamically adapt and learn new spam. Hence, we took the decision 2 weeks ago to install and setup a new Spam-Fighting solution based on adaptive Bayesian learning. We have contracted a major development team that has made a popular spam-fighting system, to create a solution custom to our needs. They haven’t been cheap, and we have also been investing in the hardware needed for a redundant spam gateway cluster to filter incoming email before it hits our mail servers.

We should have production versions of this new system up by Monday, but don’t expect a fully functional system to be in place before the end of this month, even possibly the first week of December. The new system in our tests has been cutting spam by around 95%, with a low false positive rate. We do plan to give our customers control of their incoming email, so they can sort and identify false positives on this new system.

We understand some customers are questioning our ability to be proactive on such problems. But let me say many IT managers didn’t predict this new wave of SPAM. Nevertheless, we are preparing our systems for the future, to deal with even larger amounts of spam in the months and years ahead with. I’d like to also add that finding a solution that actually can work with our setup has been extremely difficult. The current system we are deploying makes customizations to both HSphere and the way our network handles traffic.

The increasing amounts of spam have caused another issue some customers have been facing. This issue is with misdirected bounces. When spam hits autoresponders or auto-forwarders, it gets bounced back to the sender. However, what spammers are now doing is forging the headers of the spam so the bounce actually goes back to an innocent email address. Hence, that person receives a copy of the spam bounced off our mail system, and reporting us as the source of the spam. Certain ISPs/blacklists are blaming hosts for this new problem and have put the onus on us to stop misdirected bounces. If we don’t, they blacklist us. This issue has been largely confined to Comcast which has been blacklisting one of our mail servers from time to time, over the last 2 weeks. We are working with them, and are promptly requesting unlists for our IP whenever it is reported as blacklisted. But a permanent solution to this will come out once our new spam system is in place, reducing the amount of spam hitting our servers and your mail boxes.

I once again apologize for this issue. We have come under a lot of criticism from customers, but I’d like to say we aren’t the only host fallen victim to spam. I have interacted with other hosts, many using HSphere that are experiencing similar issues. It is however a company goal here at JodoHost to deliver clean email to our clients, and we will be doing everything possible to achieve it

Regards
Yash – CTO/Co-Founder, JodoHost

 

[Yash]

Our new anti-spam system is now very close to operational stage (system-wide). This means we will begin transfering each mail server, in a phased manner to the new anti-spam system (change will be transparent).

Our new anti-spam system has costed us close to $20,000 so far to develop, and we have worked with a leading anti-spam developer to implement a suitable solution for our system. The results we have got so far are way better than the standard of filtering most hosts offer, and it rivals commercial systems.

No Anti-spam system is perfect, and the reports we've got so far are that certain newsletters and bulk commercial mailing can be at times considered spam and flagged. We do have a white-listing and training mechism in place to help reduce such cases in the future.

We recently changed MX records for all customers using our mail systems, effectively separating inbound and outbound SMTP traffic. This was crucial to moving forward with this project and was the main cause for the delay in implementation. However, the MX change went through smoothly without any reported issues. More information about the new system will be posted and announced via email in the coming days