How not to Monitor Email

This is inspired by an article over at Worse Than Failure.

If you really want to fail at email monitoring, here are some ideas:

1. Don't tell the employees.
2. Don't create any policies that outline acceptable use.
3. Don't create any policies that outline acceptable monitoring.
4. Make sure the monitoring is as intrusive as possible, bonus points for trying to be covert, but telegraphing everything to the people being monitored.
5. Cause mail delays or blockage.
6. Be inconsistent.
7. Use people to review every email-- better yet, outsource it to somewhere that doesn't care about your intellectual property.
8. Implement it without much thought towards what you wish to accomplish, a vague feeling of "we should be monitoring" is enough to start.
9. Don't talk to the HR department.
10. Don't talk to the Legal department.
11. Don't talk to IT, just foist it upon them at the last possible moment.

Read more...

Bayesian Filtering: Why Not?

The earlier pieces I did on Ron Paul spam (here and here) point to some problems with Bayesian filtering. Read on for some further analysis of the problems with Bayesian filtering.

Bayesian filtering, in a nutshell, breaks down an email into words and or phrases, and then assigns a spam probability to each, based on the word's previous penchant to be spam. For example, let's assume that spammers are sending messages advertising a product called "Bradley". Over time, as those messages are categorized as spam, a Bayesian system would give an increasing spam score to the term "Bradley".

This type of system began to be deployed widely starting 3 or 4 years ago, and was very effective for a couple of years.

Bayesian Poisoning

As seems to always be the case, the spammers switched tactics. They began sending out their spam with a large number of (typically incoherent) words stolen from news sources or literature. This throws off the Bayesian system by

• Adding to the non-spam score (since there are "good" words in the mail), and
  
• Putting the good words in the spam list
  

Beginning about a year ago, I started receiving spam that only had the good text, no advertisements. This was a deliberate attempt just to poison Bayesian systems.

The ongoing issue with Bayesian systems is that spammers have fairly effectively figured out how to confuse them (either by falsely calling acceptable mail spam, or letting spam go through). Fortunately, the state of the art in spam detection is being pushed forward as well.
Read more...

Ron Paul Spam an attempt to block campaign email?

After writing my previous article about the email spam advertising the 2008 Ron Paul presidential campaign, I was left wondering: why?

Why would someone use a method so polarizing as spam in a popularity contest? The tinfoil beanie wearer in me came up with a very insidious reason.

So let's say that you are an opponent of Ron Paul, and you want to limit his reach. Where is his campaign most effective? That's right, online. You can't block his campaigners on digg, et al., but maybe you can prevent email.

No really, and here's how it works:

Send out all of Ron Paul's campaign ideas, use his bumper sticker phrases: "Ron Paul has never voted for a tax increase", and make double sure that the email will get caught as spam.

About Bayesian Filters
Now, most people who have used anti-spam tools have probably heard of a Bayesian Filter, but what you may not realize is that Bayesian Filters are subject to poisoning. You've seen this before, in spam that seems to have unusual strings of semantically incoherent words, or direct quotes from the news or literary sources. This is an attempt to "trick" a bayesian filter.

What you may or may not remember is that most of this was preceded by mail that looked similar, but contained no actual advertisement. This was an attempt at pre-training the bayesian filter to accept the later spam.

How does this effect legitimate mail?
Now back to the Ron Paul spam: if these mails are caught as spam (and they are a very obvious form of spam-- any spam filter should catch it, and MessageGate certainly nails the headers), then the phrases contained in the email get added to the bayesian spam bucket" and are henceforth used as indicators of spam. Then, when the Ron Paul campaign sends out a legitimate mail (say, one that you actually asked for), it will be categorized as spam by the bayesian filter.

That would be particularly devastating to a campaign that seems to be almost entirely dependent on the Internet.

I'm not saying that this is actually what happened, but it's interesting to think about.
Read more...

Ron Paul Spam

I wrote about the Ron Paul fan club earlier.

I've been around since Canter and Siegel offered me a chance at the green card lottery, but this is novel:

Message-ID: <000701c81a53$0156f27e$3360289e@yeibw>
From: --obscured--
To: byoung
Subject: ***SPAM*** Ron Paul Eliminates The IRS! XqvMlJY
Date: Mon, 29 Oct 2007 15:56:37 +0000
MIME-Version: 1.0

Hello Scott,

Ron Paul is for the people, unless you want your children to
have human implant RFID chips, a National ID card and create
a North American Union and see an economic collapse far worse
than the great depression. Vote for Ron Paul he speaks the
truth and the media and government is afraid of him. This is
the last honest politican left to bring this country out of
this rut from the War Profiteers and bush Administration has
created. Get motivated America, don't believe the lies of the
media he has also WON the GOP Debate On Sunday! Value Freedom
and Liberty instead of corporate lies and corruption. Bypass
this media blackout they are doing to Ron Paul, tell your family
and friends and get involved in a local group at meetup.com make
your voice heard! He will end the War In Iraq immediately,
He will eliminate the IRS and wasteful government spending, and
eliminate the Federal Reserve and restore power to the people
and the only person not a member on the CFR. Can any other runner
make these claims or give Americans the true freedom we were all
raised to believe? We are all economic slaves to the banks and the
illegal federal Reserve. This is why our currency is worth nothing
because of Hidden Inflation Tax and the IRS taking everything
you make!

** RON PAUL WILL STOP THE IRAQ WAR IMMEDIATELY! **

He has NEVER voted:
* to raise taxes
* for an unbalanced budget
* to raise congressional pay
* for a federal restriction on gun ownership
* to increase the power of the executive branch

He HAS voted:
* against the Iraq war
* against the inappropriately named USA PATRIOT act
* against regulating the internet
* against the Military Commissions Act

He will eliminate the IRS, Wasteful Government Spending &
Stop The Iraq War Immediately!

Most importantly, he voted NO on anything in Congress that
is not allowed by the Constitution. And he Despises any
politican that does not do their job for the people and lives
up to the constitution!

Google.com & Youtube.com Search: "Ron Paul"
Join The Revolution!

***************************************
We Need A Real President That Will Restore And Protect
Americans! Stop The War! Protect Our Borders!
*********VOTE RON PAUL 2008************
RqWjdM

Editors note: I work at a company that produces anti-spam solutions.

Now from the purely technical point of view, this is a rather obvious spam-- there are definite telltales in the received lines (which is why I didn't get this message at my work account-- we screen for false headers). This came in on my "canary in the coal mine" account, which isn't listed anywhere, and I never use, except to receive spam.

Dr. Paul's supporters are well known at this point for being very active on digg and the like (to the point that many have accused them of spamming), but this is really a new low.

I hope that these political spams get a little more sophisticated (since we all know they will continue, now that they've started), starting with: please don't send mail with 500 words in one paragraph.
Read more...

Receiver Initiated Authentication

Over the weekend, I read a proposal for a new method to combat spam, called receiver initiated authentication.

Read on to learn the pitfalls that the author missed.

First, it depends on changing client code. The author suggests that three companies control 70% of the email clients, which is basically true. What he does not account for is that users also have some choice in this-- they don't have to (or may not be allowed to by IT policy) upgrade to the latest version of Microsoft Outlook, for example.

Second, it assumes that "legitimate" companies would implement it. The spammers implemented SPF faster than any of the "legitimate" companies.

Third, it assumes a database of "authorized" domains. This is a popular anti-pattern to many integration problems, so I got a good laugh. (The anti-pattern is, "Let's build a big database!", and is fraught with scalability issues.)

Fourth, it uses captcha. This is supposed to block spammers, but creates a pain for legitimate users.

Anti-spam solutions are about two things:

1. Convenience for the user (put another way: productivity for the user increases when they don't have to delete 100+ spam messages a day)
   
2. Security (preventing phishing scams, viruses, etc.)
   

When the solution puts burden on the end user, it can never be successful.

Editor's note: I work for a company which is in the Email Governance space (including anti-spam).

Full article here.
Read more...

News Flash: Veterans Administration security is shameful.

Film at 11.

Among the great quotes:

As the VA was rolling out the e-mail filtering software, the software caught about 7,000 e-mails containing Social Security numbers in just one month

and

The VA had only completed two of 22 recommendations from its inspector general following the breach

Here is the full article.

Let me get this right...

The Veterans Administration (I'm a vet, so maybe I'm a little sensitive) has been sending 7,000 emails a month with SSNs?

Pause, drumroll, please--

SEVEN THOUSAND?!

Notice that that wasn't the number of SSNs, but the number of emails, so an excel spreadsheet with 1000 SSNs counts once.

I don't even want to know what they are doing with my medical records. Maybe they are putting them up on LED readerboards across the nation?


Read more...

Non-business email messages are not public records in Florida

At least for the State of Florida.

Follow the jump for an interesting opinion on the question of whether non-business email constitutes a public record.

The opinion Makes the following assertions:

..."official business communicated by e-mail transmissions is a matter of public record." In re: Amendments to Rule of Judicial Administration 2.051 -- Public Access to Judicial Records, 651 So. 2d 1185 (Fla. 1995). However, the court has also recognized that e-mail messages may include transmissions that are not official business and which, consequently, are not public records." id. at 1187. Thus, the Supreme Court has already given us some guidance in this area. Non-business e-mail messages are not public records and need not be retained.

...

Experience in our office indicates that many e-mail messages consist of one or two lines dashed off electronically because, at any given time, it may be the most expedient means of communicating a simple message: "your meeting is at 2:00, don't be late"; "remember to order a new copier cartridge this afternoon"; "please let me know when you will have the project finished." These communications are the electronic equivalent of communications that under different circumstances would take place verbally -- either by telephone or directly.

It seems that there is a prevailing legal mantra, "save everything". Including every joke of the day, every email from social networking sites, every chain letter, every everything.

I'd like to suggest that it is possible to manage your archive with a little less extreme-ness. There are definitely items that can be safely removed: do you really need the 65,000 low toner notices?

I think it is time to really discuss what constitutes a business email. What are your thoughts?
Read more...

E-Discovery Searches are Inadequate

Many e-discovery efforts focus on two things: date range searches and searches for email addresses. I'd like to suggest that these are inadequate, and what you can do to really find the messages you're looking for.

The main problem with searching addresses is that they are not normalized. They come in myriad formats:

• bradleyy@example.com


• young.bradley@example.com


• </O=EXAMPLE/OU=SERVICES/CN=RECIPIENTS/CN=BRADLEYY>

These are just examples-- there are others. The point is that these all refer to the same user. On the other hand, you might end up with different users sharing the same address (which Joe Smith were we referring to during the three year period covered by the e-discovery?).

Dates in email are really completely random, unless you are referring to dates in the received lines. Alternately, you could keep metainformation about the email, i.e. the date that it was delivered to the journal, etc.

Emails need to have the current contextual information applied at the time of archive insertion. At a minimum, I would suggest looking at inserting unique identifier for the user (something like an employee id), what department the user is in, whether the user is an executive, whether the email contains potentially proprietary information, and whether the email is potentially privileged.

It would also be a good time to set retention policies and flag non-business mail, but that's a discussion for another day.
Read more...

Schadenfreude and Bacn

Schadenfreude is one of my favorite words, precisely because of the almost universal reaction received upon defining it. "There is a word for that?!", is the incredulous refrain.

Now there's a new term making the rounds of the noosphere, and it is bacn.

Bacn is the term for mail that isn't spam, isn't personal email, and isn't business email, either. Think about the newsletters you get from companies you purchase from, automated notices from internal systems, etc.

The reason I find this term interesting is that it is something I've been talking about for some time, but just never had a good word (words are power!) to properly describe it quickly. Even though it has "hip", "Web 2.0", "look I dropped a vowel, how creative I am!"-ness to it, I suspect it is going to make it into the common lexicon.

Here's why:

• It talks about a productivity problem with email.
  The modern information professional has 10-30% of their email composed of these types of email (newsletters, automated notices, your order has shipped), and every interruption to check email takes 15 minutes to properly resume from.


• It talks about an infrastructure problem.
  These emails are going in the archive. 10-20% of the emails in an email archive are bacn. They are especially likely to be saved by users, because, "I want to read them, just not right now."


• There are tools to help deal with it.
  Here at MessageGate, we use our software to automatically tag email as bacn, and the end user can set up rules to file these emails appropriately. A more intensive approach could auto-file these mails for the user (without them setting up rules).


• The volume of email involved has a very negative impact on e-discovery efforts, especially since the current (broken) methodology of searching by address and date has basically no false-positive filtering.

Companies are starting to look at the productivity and infrastructure burden of bacn; I'm just glad to have a word to describe a topic I've been working on and thinking about. Read more...

Email statistics

Chances are that you have some decidedly unexpected behaviors happening on your email network. You probably expect that there are jokes and video files being emailed around. But did you know about that user on you network that only forwards mail?

That's right. In an enterprise of any substantial size (over 500 users), there is virtually a 100% chance that there is at least one user that for every 10 mails sent, 9 are forwards (and it isn't uncommon to see 100%). No, I don't know what they do all day.

Have you ever measured which account sends the most emails in a given unit time? I'll take odds on it being a printer (or copier, or database, or application). This certainly points to enterprises using email as a generic messaging platform, and that enterprises consider it acceptable to use email as a method for applications and devices communicating their state.

Of course, the law of unintended consequences rears its ugly head when you realize that the printer sent 65,000 emails last month lamenting the loss of toner. To a distribution group. With dozens of users. And all of it gets archived.

Of course there's the 25-40% of your email (by volume) that is entirely Office documents, and the 20-40% that is non-business email (non-business images, video files, jokes, chain letters).

Here's an idea for blocking chain letters: just block any email with more than one exclamation point in the subject line. While this is meant to be tongue in cheek, it is pretty accurate. Read more...

Three (more) things you can do today to get email under control

My friend Robert posted an article entitled three things you can do today to get your email under control. I'd like to propose my own list of three items after the jump.

1. Implement an Acceptable Use Policy (AUP)and educate your users.

   This is the starting place for all email governance efforts. You must offer your users guidance on what constitutes acceptable uses of the email system. Have you actually educated your users about policies regarding non-business email?
   
   It surprises me how many organizations either do not have an AUP (or one that is hopelessly out-of-date), or do not properly educate their users on it. If, God forbid, an employee termination is necessary, proof that the user was educated on the policy is more than just nice to have. Every webmail service has an AUP that must be acknowledged before an account can be set up, why shouldn't all organizations implement this?
   



2. Block proprietary content from leaving your organization.

   This can be as simple as searching the email and attachments for terms like "proprietary and confidential" or "internal use only", or as complex as fingerprinting specific documents, and flagging emails containing subsections of the documents.
   
   Whatever you do, you need to look at the types of files that constitute your intellectual property. Some suggested starting places:

   
   
   1. Office Documents (including Adobe PDF)
   
   
   2. Source Code (VB, Java, C, C++, Perl, COBOL...)
   
   
   3. The files that support your business, e.g.: AutoCAD, Matlab, specific reports, etc.



3. Perform an audit of your traffic to see what is really going on.

   Okay, so admittedly I copied this from Robert's article. But it bears repeating, since the vast majority of organizations do not know what exactly is moving through their email network.
   
   If you are concerned about privacy, have the report anonymized. This is something that MessageGate does regularly. It provides a great value, and I can assure you that there will be unique and interesting information in the audit. It will help you understand the metrics of your network, and will, I dare say, offer insight into the character of the organization as a whole.
   
   Just like a financial audit, it is most helpful to do the email audit on a regular basis, allowing you to track to particular goals.

Implement these, and you'll have a much greater understanding of your email, a lower risk of information leakage, and better control over your email network.
Read more...

Email and Stress

There is an interesting (although not unexpected) study published by researchers from Glasgow and Paisley universities about email stress.

Interesting findings:

The participants in the study were checking mail up to 40 times per hour.
They thought that they were checking 4 times per hour.

Links to the articles after the jump.

The article is on the Ars Technica: link.

Another interesting study was conducted in 2003, with American workers: Overwhelmed by Email. Since this study is from 2003, keep in mind that email volume has been climbing 10-20% per year (which implies that time consumed by email may have as much as doubled since then).

Consider the costs of this "always on", "pressured to respond immediately" mindset:

It takes up to 15 minutes for an information worker to properly resume the task that was interrupted responding to email. Meanwhile, the average user receives 10-20 emails per day. Even accounting for the mail coming in pairs, that would account for 1 hour 15 minutes of lost productivity, per employee, per day (and going up by 10-20% per year).

In a 1000 employee organization:

1000 users * 1.25 hours * 250 days per year = 312500 hours per year in lost productivity.

Multiply that by an by a fully loaded cost of $35/hr, and you're at $11M per year in lost productivity.

Something to keep in mind when you are analyzing the costs of email governance.
Read more...