Anti Spam Blog

New tech briefing on outbound filtering from MailChannels

“Outbound spam filtering is all about ensuring reliable email delivery. If your organization counts on email delivery, then you should invest in outbound spam filtering.”

We’ve written a new white paper that discusses the need for a multi-layered approach in dealing with outbound email abuse (i.e. outbound antispam). The layers are thus:

1. Accurate content filtering – using a great spam filter to tag and potentially reject spam messages before they leave the network;
2. Local reputation management and policy enforcement – keeping track of the reputation of “senders” in your network, and preventing “bad” users from getting too abusive; and,
3. IP address management – moving traffic out through different blocks of IP addresses depending on the reputation of the sender.

The idea of rolling out these techniques is to do on the sending side what receivers are doing on the receiving side – except with the benefit of knowing more about your senders, such as their “account id” or “phone number” depending on what kind of network or service you’re operating. We’d love to hear your feedback on this new white paper, so go ahead and download A Multi-Layered Approach to Effective Outbound Spam Protection and let us know what you think.

Tags: literature, outbound spam, whitepaper

Botnet Spam Traffic Spike on May 25

On May 25, 2011 there was a sudden spike in spam volume that lasted approximately an hour.

Global spam volume – particularly spam originating from “botnets” has been falling steadily for at least the past eighteen months. The current average daily spam volume is anywhere from 1/3 to 1/10 the volume attained when spam volume was at its peak many months ago. The explanation given by experts for the decline in spam volume is typically that a spate of recent botnet takedowns have severely damaged or completely destroyed these networks, forcing botnet operators to start from scratch.

As the botnet operators regroup, we can expect to see periods of volatile spam traffic as operators flex the muscles on their newly minted botnets – testing short lived spam campaigns for their deliverability and profitability. Does this spike foretell a coming wave in botnet spam? Only time will tell.

Why is UCEProtect so influential?

I have a question for the spamosphere: Why is the tiny UCEProtect blacklist so influential with large telecommunications providers? Nearly every provider I speak to is enormously concerned about their reputation on the UCEProtect network. This concern seems strange to me, considering the small penetration of UCEProtect versus other blacklists like Spamhaus, SORBS, and Trend Micro MAPS+.

Reviewing millions of delivery attempts at a large service provider, I found literally zero references to UCEProtect in SMPT error messages. Here is a summary of the most popular blacklist service URLs, found by searching the outbound SMTP filtering logs of an ISP and a major cloud hosting provider:

As you can see – there’s no UCEProtect to be found. In fact, I grepped the logs of millions of SMTP connection attempts from many different networks, and could not find examples of receivers indicating that UCEProtect.net was the reason that a connection or message had been blocked.

So, then, my question to you all is: Why is UCEProtect taken so seriously? Because, they definitely are.

Spam from Asia, Chapter 1 – Spam from Hong Kong

In March, I had the pleasure of visiting the great cities of Hong Kong, Manila and Singapore. Since I am on a mission to end the global spam problem by helping ISPs plug up botnet spam through transparent SMTP filtering, and seeing as the CBL ranks Asia as one of the worst emitters of spam on the planet, I figured I had better pay this region a visit. My trip was thoroughly interesting and enjoyable. Whether it was battling Manila’s rush hour traffic and smog, cheering and boozing with a group of bankers at the Rugby 7′s in Hong Kong, or admiring Singapore’s breathtaking new architecture, I learned a few key things about Asia: it’s the future, it’s wireless, and there are lots of poor people. And this is perhaps why so much spam comes from Asia.

Hong Kong: Mobile World

My Asian home base during March was in Hong Kong, where I met with the major mobile carriers to discuss outbound spam. Non-work highlights included dancing at the various night clubs of Lan Kwai Fong, shopping in Mong Kok’s vibrant street market, and swimming with the masters club at Wan Chai Training Pool. Not to mention thoroughly enjoying the Rugby 7′s (Google it if you are so inclined).

Hong Kong has what I would consider to be the world’s most competitive mobile telephone market. In Hong Kong, seven major carriers compete for customers in this the world’s most “vertically oriented” city. Mobile broadband is ubiquitous, and wireless service is incredibly cheap. In Hong Kong, you can buy pre-paid (i.e. no commitment) wireless service for about USD $3/day, which includes unlimited data, unlimited voice, and unlimited texting. As a visitor, this is just mind-blowing. And did I mention the pre-paid SIM card, which costs about USD $15, includes $15 worth of credit? Hong Kong wireless service is so competitive that most people have more than one mobile phone – penetration of wireless is at 170% of the population (source).

Trouble is, Hong Kong’s networks originate a great deal of spam (data courtesy of SenderBase). The pie chart at right summarizes the IP address reputation of all of the email-sending IP addresses owned by a major Hong Kong-based mobile operator. The pie chart represents over 5,000 IP addresses, which is a small slice of the IP address space owned by this operator about which SenderBase is aware.

Fortunately, Hong Kong operators are aware of these issues and are taking steps to contain outbound spam through a variety of techniques, including transparent spam filtering and outbound spam filtering at mail relays. This is much more, sadly, than one can say for other operators in the region, which originates a very large proportion of the world’s spam.

I hope to return to Hong Kong in the very near future and am looking forward to exploring Lan Kwai Fong once more, as well as making greater use of the fantastic and cheap Public Light Bus service (people from Hong Kong will laugh at this).

Botnet Spam Rankings: Holding Steady

The world's 43 worst botnet-hosting networks, as identified by the Composite Blocking List (CBL), over the past 160 days. As the chart shows, there have been some wiggles in the top-43, but few substantial changes in which networks are hosting most of the world's spam bots.

I read with interest Terry Zink’s latest blog post concerning the distribution of botnets since the Rustock botnet was taken down some time ago. According to Terry, specific countries play host to specific botnets. So when the Rustock botnet was taken down, some countries experienced a dramatic reduction in their spam emanation, while others saw almost no change.

Since late last year, I have been keeping track of (on a daily basis) the number of IP addresses listed on the Composite Blocking List (CBL – this is the main component of Spamhaus’s widely used XBL list) for the absolute worst offenders on this list. I scrape my data from a helpful page the CBL maintains and then use Microsoft Excel to compile it into the pretty picture shown above.

If we chart the same data using a cumulative area graph (rather than a percentage-based graph like the one shown above), we can see how the total number of CBL blacklistings from these networks has fluctuated over the same time period:

Some interesting findings:

• India and Vietnam still have a huge share of the CBL, and this share seems to be holding very steady.
• asianet.co.th (True Internet) has done something to clean up its act. While it is still one of the largest sources of botnet spam, its share has plunged from 3% to nearly nothing. On the CBL, it is now ranked #97 – down from #10 several weeks ago.
• The overall stability of the relative CBL blacklist counts indicates that even though Rustock may have reduced global spam volume, the distribution of hosts infected with some kind of spamming bot remained fairly steady.
• The cumulative graph shows some variation, but clearly there has not been a fall in spam host infections – at least, not in these “top” networks.

Here’s something worth considering: If the networks listed here installed outbound spam filtering technology, the number of blacklist entries in the CBL would drop by more than 50%. To make that scale of an impact on the global spam problem by cleaning up botnet infections on all the other networks in the world, one would need to deploy outbound spam filtering on more than 12,000 different networks!

Clearly we should all focus on these “dirty 43″.

Quarantining Infected Users to Secure the Internet Ecosystem, Microsoft

At the RSA Conference 2011, Microsoft’s Scott Charney gave a talk proposing that ISPs should quarantine compromised customer accounts that are spewing spam.

Charney argues that this can be done with existing technology using a system that checks a computer’s “health” before granting unfettered access to the Internet. In general, we think this approach is a good idea, but as with many “big picture” security ideas it has a few flaws. The main flaw is, how do you know that a system is clean? Perfectly clean systems can be infected with zero-day malware and begin spewing out spam and malware on a moment’s notice. What value would the health certificate have in this case? Another flaw is that the security certificate system would have to be “signed” by someone. Who would create certificates, and how would the Internet community know that they can be trusted?

Fortunately, because compromised computers tend to be used for spamming and other obvious network-borne attacks, there is an easier and immediately implementable solution that doesn’t require certificates to provide great protection to the Internet community. The solution involves monitoring the external behaviour of machines through network sensors and filters, and then clamping down on a machine’s access to Internet resources (i.e. bandwidth and ports).

MailChannels specializes in outbound spam filtering, so we can comment on detecting spam in the network. But other companies offer solutions for detecting other kinds of nefarious activity – particularly the accessing of botnet command and control systems.

If you’re not familiar with what we do, consider this a quick refresher. Or, if you’re interested in understanding more about how to protect the reputation of networks from compromised customer accounts sending spam, please read on.

Outbound Filtering

According to reputation security networks, most ISPs in the world have a chunk of IP addresses that are bad – some even as high as 99%. When you take a closer look at “Poor” addresses, we find many of them are listed on blacklists. Anyone sending email from within these IP ranges will be blocked by most of the Internet.

Outbound spam filtering allows ISPs to take immediate action within seconds, and completely automate the process of improving your reputation before botnet infections get you blacklisted.

Transparency

Our outbound spam filtering operates transparently so you can deploy without major configuration changes. It transparently intercepts all port-25 traffic coming out of the network, and passes the traffic through one or multiple content filters from leading vendors. SMTP AUTH and SSL encryption is fully supported so the privacy of conversations are respected.

–

How are you identifying fraudulent customer accounts?

Webcast – Outbound Spam Filtering for VPS & Dedicated Hosts [6 mins]

On January 26, 2011, CEO Ken Simpson gave a talk on outbound spam filtering for the BeanSprout Web Host Showcase.

There are two main drivers for outbound spam filtering at VPS hosts. First, improving IP address space reputation so that VPS customers will be able to reliably deliver email to the rest of the Internet. And second – and more importantly – reducing the number of fraudulent accounts that are set up in their system.

Here is the entire transcript:

SLIDE 2:

Earlier this month, I logged on to the Cisco SenderBase web site, which keeps track of all the spam that is sent to Cisco IronPort customers worldwide. IronPort processes well over a billion email messages a day for tens of thousands of companies worldwide.

SenderBase keeps a list of the top 100 worst spam sending IP addresses. For each IP address, it shows the network owner of that IP address. I downloaded the data as an Excel file, popped it into a Pivot Table, and grouped by network, summing up the millions of spam messages per day reported by SenderBase for each IP address.

The result: Most of the world’s large spam sources are hosted by hosting companies.

SLIDE 3:

Virtualization makes spamming as easy as 1 2 3. The first step is to acquire a VPS account. Spammers use stolen credit card credentials that they obtain from online card number trading sites to buy VPS server capacity.

Armed with a login, they install their spam sending software and begin blasting away. We sometimes see a single VPS instance sending up to 8,000 new SMTP connections per second. VPS operators often don’t find out about a spamming account until the credit card firm issues a chargeback notice, or until their IP space has been blacklisted – but we’ll get to that in detail later.

Finally, the IP address of the VPS appears on multiple RBLs or blacklists. Email delivery from the particular VPS is hampered enough that the spammer abandons the account.

And the process repeats itself. It’s really hard to identify spammers because of their use of proxies and stolen credit cards.

So preventing them from signing up in the first place is next to impossible. You really have to deal with the aftermath of fraudulent signups, which implies being able to monitor the spam coming out of their accounts.

SLIDE 4:

The cost of spamming to a VPS operation increases as the number of spammers increases. When a new VPS service opens, the level of account fraud and spam may initially be relatively low. This low level of fraud generates a small amount of blacklisting, and email delivery for all users is quite reliable.

As more spammers take up residence in the VPS service, the number of blacklist entries increases. Email receivers like Yahoo, Google, AOL, and Hotmail begin to place limits on the amount of email they will tolerate from your IP space. Meanwhile, the costs associated with credit card chargebacks increase.

If you don’t get the spam problem under control, basically the whole Internet blocks your network from sending email. This is what happened to Amazon web services, who in 2009 were listed on two very prominent blacklists: Spamhaus and Trend Micro’s MAPS list. You basically can’t deliver email from Amazon anymore.

SLIDE 5:

Our solution for VPS and dedicated hosting providers is to offer a transparent spam filtering system that integrates with your hosting network. You configure your routers to redirect email traffic through a small number of machines running our software.

We scan the email, block the spam, and send you trouble tickets whenever one of your customers appears to be spamming.

This enables you to break the spamming cycle by quickly shutting down spammer accounts, and limiting the amount of damage they can do to your IP reputation.

SLIDE 6:

This slide shows what we were able to do for an ISP in Asia into whose network we installed our software. Within 72 hours of installing, the number of blacklist entries on the UCEProtect blacklist was reduced by 40%. This is an amazing result.

SLIDE 7:

[live demo]

SLIDE 8:

Coming soon to OnApp

Does your VPS offering filter outbound spam?

Bill Gates Idea Implemented by an Online Book Store – Amazon Simple Email Service (SES)

In 2004, Bill Gates announced that the global spam problem would be completely eliminated in two years as a result of new technology from Microsoft. The new technology he was talking about was basically a way to make email senders pay a small fee for each email message they send. Spammers would not want to pay exorbitant amounts to send email, and would therefore stop spamming the world. Many of us in the anti-spam community laughed at Bill’s suggestion, because it seemed ridiculous that everyone in the world would ever “universally” participate in such a scheme.

But, in a way, I think Bill Gates’ vision has actually come to fruition. If you’re trying to send marketing email or transactional email, you will very quickly discover that it costs money to do so effectively. And now, if you use Amazon’s cloud services for your hosting needs, you’ll find it costs real money to send email from there as well – at least, if you want to send it reliably.

Amazon’s Problem

Amazon Web Services, LLC, the subsidiary of Amazon.com that operates the formidable Electric Elastic Compute Cloud (EC2), announced yesterday the availability of a new email sending service aimed at users of EC2 and Amazon’s other cloud services. In my opinion, the new service, which is called (predictably) “Amazon Simple Email Service,” aims to resolve a longstanding problem with EC2 – namely, that the spam problem in EC2 had ruined EC2′s IP reputation and made it next to impossible to send email from Amazon’s cloud.

The problem started in late 2009, when Amazon EC2 users started complaining in the support forums that their outbound email was being blocked worldwide by the Trend Micro MAPS blacklist service. Despite many appeals to Trend Micro, the blocking continued for a considerable time, causing Amazon EC2 users a great deal of “pain”. Other blacklists joined in, including SORBS; and Spamhaus added Amazon’s entire EC2 IP space to the Policy Block List (PBL). The blacklists had good reason to list EC2′s IP address space: Spammers had gone literally nuts on the Amazon service, signing up accounts and sending spam indiscriminately and in very, very high volumes. And Amazon, despite tasking a team to deal with the problem, was basically unable to convince the world that its dynamic cloud could ever be trusted.

In response, Amazon created Simple Email Service. And now, if you want to send email reliably from the Amazon cloud, all you need to do is sign up for an SES account and learn to use their email sending API. Amazon has built a reputation system to track the content (i.e. spam) and complaint histories for each SES customer, with sending volume gradually increasing as an SES customer demonstrates its ability to send good email that the world wants. Messages cost $0.10 per 2,000 messages delivered.

Our Analysis of Amazon Simple Email Service

Amazon Simple Email Service will do very well, financially. Almost all EC2 users need to send email reliably, and I think a good proportion of users will opt to send via the SES service. Competitors like SendGrid and authsmtp.com will feel some pain, since many of their customers are frustrated Amazon cloud users; however, the more innovative providers (like SendGrid) will prosper as a result of their much richer offering, servicing customers who need much more than just email delivery.

Spammers will abuse the service, of course. I fully predict that spammers will sign up for SES accounts, using credible “front” companies to obtain accounts, and patiently build up their sending volume until it is “ripe” for a bulk email blast. This will repeat and repeat endlessly, with offshore “mechanical turks” (another Amazon innovation) cranking out new SES accounts daily to feed the beast. At the end of the day, whether Amazon SES succeeds in providing truly reliable email delivery will hinge on how good their outbound spam filtering technology is, and how quickly they act when an SES account starts to go bad.

I’m guessing that Amazon will invest enough to do a good job, and that SES will emerge as one of the largest email senders in the world by the end of 2011. After all, they’re being paid to do a good job, which provides just the kind of incentive Bill Gates was talking about.

Outbound Spam Protection for the Hosting Industry

On January 26th at 2pm Eastern, we will be participating in the “Web Host Showcase,” a webinar being hosted by partnering startup Beansprout. If you want to see a live demo of of our outbound filtering software running inside a virtual private server hosting network, visit the following URL to register: https://www3.gotomeeting.com/register/820616766

Tags: beansprout, webinar

The World’s Top Spam Sources

I popped open Excel and generated some stats porn for everyone today.

One of the interesting things we track here at MailChannels is the positioning of the world’s worst spam sources on the world’s best blacklists. The chart above shows the number of blacklist entries on the Composite Blocking List (CBL – link) for each of the top-15 spam sending networks on the Internet. The CBL tracks botnet infections (excellent statistics are available on the CBL web site) by analyzing spam traffic aimed at its extensive honeypot network, and then lists the IP addresses from which this spam traffic originates. The listings are automated, and listings can be easily removed by ISPs through a web page once the bot problem has been resolved. Listings that are not manually removed in this manner do eventually time out on their own.

I suppose one of the interesting things about this chart is that despite the fact that spam almost disappeared over the holidays (see our previous post), the number of CBL listings produced by each of these networks stayed relatively constant during that time period (our chart starts roughly in late November 2010). I’m impressed by the apparent efforts of the folks at vnnic.net.vn (Vietnam Post and Telegraph Company) to clean up their act, resulting in a substantial drop in listings during the time period under analysis. But for most of these providers, it seems that business as usual continues to prevail when it comes to removing bot infections from their networks.

USA vs. Russia vs. Thailand vs. China

The largest spam sources don’t always come from the largest countries. For a variety of reasons, the United States (population 308,745,538) has far fewer bot infections listed in the CBL’s top-100 spamming networks list than the much smaller country of Thailand (population 65,998,436). Russia tops this comparison, however, with nearly 10-times the number of CBL listings in the top-100 spamming networks list during the time period under analysis.

The Worst Spamming Countries

In economic news, we often hear of the “BRIC”, which refers to Brazil, Russia, India, and China. The BRIC nations are fast-growing, with large, young populations, and apparently are also a great source of spam. If we look at the number of spamming networks from each country that are listed in the CBL’s top-100 spamming networks list, we find Russia on top, with India in second place, Brazil in third trailing not far behind, and .. actually, China doesn’t even make the list. China would be on the list weren’t it for the fact that Internet access in that country is highly concentrated among a small group of massive ISPs.

Again, I find it strange that Thailand makes this list, considering its very small population. Armenia is also a surprise – with a population just over 3M, you have to wonder how they manage to get so many networks into the top-100 list of spam sources.

Conclusions

It’s not news (at least, not to me) that the world’s largest spam sources are developing nations. Developing countries are often many years behind developed countries in their acquisition of technology because vendors tend to visit these countries last after developing what is perceived to be more profitable first-world markets initially. We humbly assert that MailChannels is doing its part in the developing world to reduce the spam problem (read our recent case study on outbound spam control at Cambodia’s Ezecom for reference). As we succeed in landing more outbound spam control customers in these markets, my great hope is that the CBL list of 2011 looks a lot better in all respects than it did at the tail end of 2010.

Tags: cbl, ezecom, outbound spam, spam, statistics

ARCHIVES

• July 2011 (1)
• May 2011 (2)
• April 2011 (2)
• February 2011 (2)
• January 2011 (4)
• November 2010 (2)
• October 2010 (6)
• September 2010 (2)
• July 2010 (1)
• June 2010 (1)
• May 2010 (2)
• April 2010 (2)
• March 2010 (2)
• January 2010 (1)
• July 2009 (4)
• June 2009 (1)
• May 2009 (2)
• March 2009 (1)
• January 2009 (1)
• December 2008 (3)
• November 2008 (1)
• October 2008 (8)
• September 2008 (5)
• August 2008 (4)
• July 2008 (9)
• June 2008 (10)
• May 2008 (9)
• April 2008 (13)
• March 2008 (3)
• February 2008 (6)
• January 2008 (6)
• December 2007 (10)
• SEARCH: