Remove me from your mailing list.

If you are sending out newsletters, here's some advice:

Make it very easy to unsubscribe.

Here's why:

1. It's just common decency.
2. Users will not appreciate your lock in tactics, and it will tarnish your brand.
3. Since it's hard to unsubscribe, they'll just mark you as spam.

If you require me to log in to unsubscribe, I'm marking you as spam.
If there is more than 1-2 steps to unsubscribe, I'm marking you as spam.

Ideally, put a link at the bottom of the email that says "unsubscribe". When clicked, it optionally asks, "are you sure?", and then unsubscribes the user. No fuss, no muss.

Image from Sarah G Read more...

Thinking about reputation services

One of the most popular methods of blocking spam these days is to use a device which implements a reputation service. Unfortunately, there is a significant issue with these types of services. With a little research, a spammer could effectively bypass this protection or pollute the reputations of so-called "good" hosts.

A short description of how reputation services work, and then a breakdown of the failure spot after the break.

Reputation services work on a simple principal: they keep a list of hosts and their respective reputations, and they perform actions based on the reputation of the connecting host. This system "trains" like a bayesian filter does--hosts can improve or decline in status over time. Typically this is augmented with a service which updates this host list.

The two important points to keep in mind here are that a) they track IP addresses, and b) they use training.

A brief sidebar to talk about the Internet Protocol (IP). IP packets can become fragmented for various reasons, including intentionally. The IP specification has a method to re-assemble these fragmented packets:

To assemble the fragments of an internet datagram, an internet
protocol module (for example at a destination host) combines
internet datagrams that all have the same value for the four fields:
identification, source, destination, and protocol. The combination
is done by placing the data portion of each fragment in the relative
position indicated by the fragment offset in that fragment's
internet header. The first fragment will have the fragment offset
zero, and the last fragment will have the more-fragments flag reset
to zero.

The important section there is "placing the data portion of each fragment in the relative position indicated by the fragment offset". In other words, each fragment gets to determine where it starts. By specially crafting packets, it is possible to forge the source and destination ip addresses, but this is only evident if the packets are re-assembled. If you are running at high volume, packet re-assembly is expensive.

Here's the potential problem: if you have a reputation service (which, by definition, needs to handle high volume), it could easily fall prey to clever spammers using IP fragmentation to bypass or pollute the reputation data. If they were truly evil, they would deliberately ruin the reputation of valid, non-spamming organizations.

Worse still, most of these reputation devices communicate back to a central reputation store, so it would be possible to create a denial of service against certain organizations: the ruined reputation now propagates out to other subscribers of the service.

Defense in depth is the only strategy that truly works long term.
Read more...

Bayesian Filtering: Why Not?

The earlier pieces I did on Ron Paul spam (here and here) point to some problems with Bayesian filtering. Read on for some further analysis of the problems with Bayesian filtering.

Bayesian filtering, in a nutshell, breaks down an email into words and or phrases, and then assigns a spam probability to each, based on the word's previous penchant to be spam. For example, let's assume that spammers are sending messages advertising a product called "Bradley". Over time, as those messages are categorized as spam, a Bayesian system would give an increasing spam score to the term "Bradley".

This type of system began to be deployed widely starting 3 or 4 years ago, and was very effective for a couple of years.

Bayesian Poisoning

As seems to always be the case, the spammers switched tactics. They began sending out their spam with a large number of (typically incoherent) words stolen from news sources or literature. This throws off the Bayesian system by

• Adding to the non-spam score (since there are "good" words in the mail), and
  
• Putting the good words in the spam list
  

Beginning about a year ago, I started receiving spam that only had the good text, no advertisements. This was a deliberate attempt just to poison Bayesian systems.

The ongoing issue with Bayesian systems is that spammers have fairly effectively figured out how to confuse them (either by falsely calling acceptable mail spam, or letting spam go through). Fortunately, the state of the art in spam detection is being pushed forward as well.
Read more...

Ron Paul Spam an attempt to block campaign email?

After writing my previous article about the email spam advertising the 2008 Ron Paul presidential campaign, I was left wondering: why?

Why would someone use a method so polarizing as spam in a popularity contest? The tinfoil beanie wearer in me came up with a very insidious reason.

So let's say that you are an opponent of Ron Paul, and you want to limit his reach. Where is his campaign most effective? That's right, online. You can't block his campaigners on digg, et al., but maybe you can prevent email.

No really, and here's how it works:

Send out all of Ron Paul's campaign ideas, use his bumper sticker phrases: "Ron Paul has never voted for a tax increase", and make double sure that the email will get caught as spam.

About Bayesian Filters
Now, most people who have used anti-spam tools have probably heard of a Bayesian Filter, but what you may not realize is that Bayesian Filters are subject to poisoning. You've seen this before, in spam that seems to have unusual strings of semantically incoherent words, or direct quotes from the news or literary sources. This is an attempt to "trick" a bayesian filter.

What you may or may not remember is that most of this was preceded by mail that looked similar, but contained no actual advertisement. This was an attempt at pre-training the bayesian filter to accept the later spam.

How does this effect legitimate mail?
Now back to the Ron Paul spam: if these mails are caught as spam (and they are a very obvious form of spam-- any spam filter should catch it, and MessageGate certainly nails the headers), then the phrases contained in the email get added to the bayesian spam bucket" and are henceforth used as indicators of spam. Then, when the Ron Paul campaign sends out a legitimate mail (say, one that you actually asked for), it will be categorized as spam by the bayesian filter.

That would be particularly devastating to a campaign that seems to be almost entirely dependent on the Internet.

I'm not saying that this is actually what happened, but it's interesting to think about.
Read more...

Ron Paul Spam

I wrote about the Ron Paul fan club earlier.

I've been around since Canter and Siegel offered me a chance at the green card lottery, but this is novel:

Message-ID: <000701c81a53$0156f27e$3360289e@yeibw>
From: --obscured--
To: byoung
Subject: ***SPAM*** Ron Paul Eliminates The IRS! XqvMlJY
Date: Mon, 29 Oct 2007 15:56:37 +0000
MIME-Version: 1.0

Hello Scott,

Ron Paul is for the people, unless you want your children to
have human implant RFID chips, a National ID card and create
a North American Union and see an economic collapse far worse
than the great depression. Vote for Ron Paul he speaks the
truth and the media and government is afraid of him. This is
the last honest politican left to bring this country out of
this rut from the War Profiteers and bush Administration has
created. Get motivated America, don't believe the lies of the
media he has also WON the GOP Debate On Sunday! Value Freedom
and Liberty instead of corporate lies and corruption. Bypass
this media blackout they are doing to Ron Paul, tell your family
and friends and get involved in a local group at meetup.com make
your voice heard! He will end the War In Iraq immediately,
He will eliminate the IRS and wasteful government spending, and
eliminate the Federal Reserve and restore power to the people
and the only person not a member on the CFR. Can any other runner
make these claims or give Americans the true freedom we were all
raised to believe? We are all economic slaves to the banks and the
illegal federal Reserve. This is why our currency is worth nothing
because of Hidden Inflation Tax and the IRS taking everything
you make!

** RON PAUL WILL STOP THE IRAQ WAR IMMEDIATELY! **

He has NEVER voted:
* to raise taxes
* for an unbalanced budget
* to raise congressional pay
* for a federal restriction on gun ownership
* to increase the power of the executive branch

He HAS voted:
* against the Iraq war
* against the inappropriately named USA PATRIOT act
* against regulating the internet
* against the Military Commissions Act

He will eliminate the IRS, Wasteful Government Spending &
Stop The Iraq War Immediately!

Most importantly, he voted NO on anything in Congress that
is not allowed by the Constitution. And he Despises any
politican that does not do their job for the people and lives
up to the constitution!

Google.com & Youtube.com Search: "Ron Paul"
Join The Revolution!

***************************************
We Need A Real President That Will Restore And Protect
Americans! Stop The War! Protect Our Borders!
*********VOTE RON PAUL 2008************
RqWjdM

Editors note: I work at a company that produces anti-spam solutions.

Now from the purely technical point of view, this is a rather obvious spam-- there are definite telltales in the received lines (which is why I didn't get this message at my work account-- we screen for false headers). This came in on my "canary in the coal mine" account, which isn't listed anywhere, and I never use, except to receive spam.

Dr. Paul's supporters are well known at this point for being very active on digg and the like (to the point that many have accused them of spamming), but this is really a new low.

I hope that these political spams get a little more sophisticated (since we all know they will continue, now that they've started), starting with: please don't send mail with 500 words in one paragraph.
Read more...

Receiver Initiated Authentication

Over the weekend, I read a proposal for a new method to combat spam, called receiver initiated authentication.

Read on to learn the pitfalls that the author missed.

First, it depends on changing client code. The author suggests that three companies control 70% of the email clients, which is basically true. What he does not account for is that users also have some choice in this-- they don't have to (or may not be allowed to by IT policy) upgrade to the latest version of Microsoft Outlook, for example.

Second, it assumes that "legitimate" companies would implement it. The spammers implemented SPF faster than any of the "legitimate" companies.

Third, it assumes a database of "authorized" domains. This is a popular anti-pattern to many integration problems, so I got a good laugh. (The anti-pattern is, "Let's build a big database!", and is fraught with scalability issues.)

Fourth, it uses captcha. This is supposed to block spammers, but creates a pain for legitimate users.

Anti-spam solutions are about two things:

1. Convenience for the user (put another way: productivity for the user increases when they don't have to delete 100+ spam messages a day)
   
2. Security (preventing phishing scams, viruses, etc.)
   

When the solution puts burden on the end user, it can never be successful.

Editor's note: I work for a company which is in the Email Governance space (including anti-spam).

Full article here.
Read more...