EMail, the only real killer application on the Internet has a big problem: a spam problem. After a brief decline in the first eight months of last year, spam is back. The New York Times reports that the situation is getting worse by the minute.
Ironport, a spam filtering firm, and unsolicited junk mail now accounts for more than 9 of every 10 e-mail messages sent over the Internet.
According to Softscan, the percentage of email which counts as spam is 89.73%, which is close to the IronPort data.
Image spam used to just contain straight text on a white background, now increasingly spammers are trying to obfuscate scanners by introducing more complex images and colours. Text has also been disguised by changing its colour throughout the image and is frequently distorted
There are new kinds of spam, like the image spam which is making the rounds, and is causing problems. It is not the only kind, but it is the kind which has become a major issue for us here at GigaOM. These image spam emails continue to confound our email servers, and also our Apple Mail inbox, even though we have been patiently teaching it to behave like an over eager parent.
No dice! Some of us simply have set up elaborate rules where we route emails to our Google Mail account, and then route it back to our main email account. That helps, since Google has good spam filters, but in reality it is a band aid fix. (Please share your tips and tricks with rest of us, and help make our email lives better!)
Our personal spam trials and tribulations aside, the email spam is a massive productivity drain and I bet it is a big drain on the overall economy. What is more confounding is that the bad guys keep winning, and the technology industry, with all its brains has failed to outwit them. Back in 2003, Bill Gates had declared war on spam, but that hasn’t helped.
So what we really need to do is rethink the whole concept of email and its client-server underpinnings. We need a new kind of an email application, be it for the web or for the local machine.
Om,
Try this fix which should work for image related spams:
http://lifehacker.com/software/gmail/how-to-filter-gmail-image-spam-197955.php
It has helped me quite a bit. Atleast in Gmail.
Regards,
Rushi
Om, We use Postini and it grabs everything. Full disclosure: we invested in Postini so ostensibly some interest in that recommendation. Not sure if that is what you are looking for, but I hope that is helpful.
the biggest problem with email nowadays is “false positives”, when legitimate emails get snagged in overzealous spam filters. i recently tested this myself, sending 10 blank emails directly from my school account to my hotmail account. only 8 made it through.
while viagra ads are annoying, imagine the communication breakdown that results when 20% of the emails you send never arrive.
I’d recommend a visit to http://www.acme.com for a good read on effectively handling spam.
We’ve been saying for some while – http://www.egoboss.com/pdfs/egobossprnewsoct2006_1.pdf – that email is no longer viable; it’s become the new Fax: use RSS/tailored blogs for business communications, IM for personal contacts and other blogs/social networks areas for everything else. if all else fails, use VoIP, of course.
so, R.I.P., email … it’s ironic, as in the past year or so email client appllcations have become really nice to use. Too late.
Just look at the new demographic – youngsters who have caused the surge in social networks such as myspace, etc: rarely do they use email – it’s perceived as being as arcane/irrelevant as letter-writing for communication. they want to communicate via IM, myspace pages/blogs, text/SMS messaging – not email.
with new legislation regards email archiving and the knowledge that most of our email is being monitored (especially in the context of business accounts) the appeal of email has diminished to the point of an irrelevance.
the killer app has been killed.
Email as the only killer app of the internet? Maybe the FIRST killer app but not the only one…
Apps I would classify as Killer (today)in order of appearance
On the horizon:
1. Gaming (Almost there)
2. VoIP (not mainstream yet, for good reason)
3 ???
we run 100s of sites and have the catch all mail for all of them so end up getting 1000s of spam mails per day
The solution is Cloudmark. Cloudmark works and for me has a 99% positive identification rate.
It just rocks and it saves so much time…I love the product. I should work in their marketing department 🙂
I am the project leader of EmailXT, a new public, patent-free protocol for email.
EmailXT is a proposal for a new email protocol that solves its current problems. It makes your email private, and defeats any non-authorized bulk emailing, and stops viruses from spreading. It is based on relationships and has a self-updating address book. With its automatic return receipts you always know if your message reached its intended recipient. Check other features at the EmailXT website.
This is a work in progress (currently at v0.1). Upcoming features are forms, photo albums, tasks and calendars, file sharing.
Along with the protocol, an official email client named InfinityXT is made available, currently at a pre-alpha stage (v0.13).
For those who want to learn more can visit the EmailXT end-user website at http://www.emailxt.com
We were using our own mail hosting until we switched to the Google hosted apps last week. It is amazing how good their filter is. I went from just about pulling my hair out to disappointed that I don’t get mail anymore. I find myself constantly scanning the spam folder searching for a false positive, but have yet to find one.
Or perhaps I am just lonely and miss my spam.
Hey Om.
I recently tried “Spam Sieve”:http://c-command.com/spamsieve/, which works with Apple Mail, and it rocks.
You train it with 1,000 messages, 2/3 SPAM and 1/3 good.
After that, my SPAM problem is basically 100% resolved, with a very few early false positives (yikes!) and now and again a false negative.
Image spam is gone.
sorry, but that gmail spam filter sucks ass and i’m not sure who they keep catering to, but it would appear that they must be manually removing spam from the inboxes of select users and groups…the rest of the hoi polloi are looking at image spam, vitamins and all sorts of crap that any normal isp gateway would catch…
It’s a bummer to have to spend so much time dealing with such a tedious problem. I’m going to try some of these solutions but I agree that over time we will probably form something different to communicate with eachother because email doesn’t work well anymore.
I’ve been very happy with the way my webhosts (pair.com, which uses greylisting and SpamAssassin) antispam has been working for me lately. A few weeks ago I realized that more spam was getting through, so I uploaded a chunk of it to help train the spam filter and the spam has disappeared again.
Unfortunately, the process of collecting the spam messages and submitting them for training isn’t as simple as it should be. It would be nice if there was a standard for client side spam flagging to be reported back to server side spam filters.
I have used Vqme.com for a couple of years and I don’t really see spam any more. They do catch stuff that they shouldn’t, but I can usually pull it out easily. They give senders an option to identify themselves as real people, which works for most of my contacts.
That said, their interface is the pits, they tend to run slowly sometimes (I guess they get overwhelmed). Apart from that, I just don’t see any spam and it makes me happy!
How do spammers get paid? Is it possible and feasible to follow the money? Who has authority to cut off spammer’s access to whatever funding mechanisms keep the economics of their trade alive? And whatever happened with the bounty, from Microsoft I think, for successful spam procescution? I wonder if there’s an economic solution to a problem driving by such a low barrier for entry and a high rate of return (compared, at least, to the potential for getting punished).
What Tom said. SpamSieve has the problem solved 99%.
Being on a Mac, I’m surprised you don’t use it.
http://c-command.com/spamsieve/
I have been using it for a long time and it just keeps getting better. The developer is very responsive, which helps a lot. It goes on all of my Macs before I even open Mail. It works with Entourage as well.
Hope this helps.
There is a simple, quick, and sure-fire solution to the spam problem.
Every day, someone tracks down a spammer (it isn’t that hard) and kills them. Seriously. Send a photo of the body along with details of which spam campaign they are responsible for and post it on a public web site.
It’s a serious problem and while the nice Postini people probably do have image spam licked, there will be a new one coming along tomorrow. Plus we will soon be seeing voip, audio, video and other forms of spam. One of the interesting things to me is that people seem to be able to tell the spam on sight without opening it – so we ought to be able to teach machines to do it.
Anyway – Ironport has a live webinar [http://w.on24.com/r.htm?e=31330&s=1&k=4F3C9E916ADB0DC0C1C2A628F11D20BC&partnerref=blog]
with a live Q&A specifically about spam along with an email security expert coming up on Thursday, 1PST for anyone who wants to get a chance to dig into this issue with a couple fo experts. (fair notice – I am editorial director for the parent company that owns ITSecurity.com, the site running the webinar)
One more testimonial for Spam Sieve. I do all my business via email, and write between 100 – 200 emails per day. I receive about 300 valid emails a day, and about 2,000 spam emails. Spam Sieve has saved me about 30 minutes a day. After one month, I get maybe one spam email in my inbox each day, takes about 2 seconds to train it again, and it is gone. I have only gotten one or two good emails put in the spam box, so I still check, but I can check them all in about a minute.
I was in a conference earlier this year with Matthew Seely, president of Cheetahmail, and he stated spam would be gone by 2008. I nearly fell off my chair. Not only is it email spam, but I have to fight forum spam, comment spam and any other kind of spam you can think about every day. I doubt it will get better any time soon.
We use and resell Postini and it does not grab everything. It does ok, but I still get lots of SPAM and lots of image based SPAM. I would estimate that it gets catches about half of my SPAM.
Spamsieve and MediaTemple’s hardcore spam filter software does a 100% job catching all this crap, we haven’t had a Spam mail in our inbox for months!
;o)
gmail has learned how to block image spam..
99% anti-spam accuracy ISN’T GOOD ENOUGH.
I am surprised no one mentioned services like http://www.SpamArrest.com – that first requires manual verification from sender. In all honesty, all those real senders who ignore such request obviously don’t require my attention that much in the first place.
I manage around 10 sites and receive 1000s of spam messages per day. I have been using SpamArrest for 2 years and have only occasionally (once a week or so) see a couple of spammers verify their email addresses, after which they are blocked forever at source.
Sure it costs $50/year, but the pleasure of having a functional and working email exceed the cost.
Best,
Artashes
As long as everybody tries to filter on things the spammer can control it’s a loosing game. Filter on syn packets, for the technology challenged. That’s the first packet any client has to send to you before you answer, or just drop it.
Filter on location.
Filter on connection behavior.
Filter on DNS name.
Filter on Company information.
/* So far the spammer can’t control this info.
* IFF they use bots.
*/
Filter on text, pictures.
Bingo no spam. All bots a weeded out. Mail servers that send spam are weeded out.
Anyhow, before Thanksgiving I saw a few hundred bot connection attempts at any given time. Just checked it’s down to 14.
Looks like the wave is gone anyway.
Don’t listen to those experts who just want you to buy a new BIGGER SPAM FILTER/SERVER. One packet is all you need to weed out most bots.
Maybe somebody tell CISCO.
Om,
Gmail’s spam filters have been causing a number of false positives for me recently. Having to perue spam everyday just to keep up is getting out of hand. Emails life is limited. I’ve been following push protocols being worked on including LLUP and SENA that may go a way to solving part of the problem. Identity and reputation services filling the rest of that gap.
I would welcome all the email spam in the world if someone could figure out how to stop all the junk mail that shows in my real world mail box. i’ve called all the numbers, but it just seems like every time i even step into a new store, i get a damn catalog within 2 days.
it’s a heck of a lot easier to just hit the delete button then it is to balance my bags, stop all mail from exploding out of my mail box as soon as i open it and then go through it all to figure out the 99% of it is complete garbage
I use fastmail and it does a decent job with its proactive filtering. It does get some false positives but since it’s IMAP, just takes a click to sort it out.
Email has sucked for a long time even without spam. It traps info in an unusable way in your inbox. I think people also hide behind it.
Here’s an image spam fix for Mail.app (no SpamSieve required):
http://www.hawkwings.net/2006/08/01/mailapp-rule-fix-for-image-spam/
You can also set a rule to filter e-mails with .gif attachments, since nobody e-mails gifs to each other.
The FuzzyOCR module for SpamAssassin works pretty well at reading image spam but spammers are getting increasingly sophisticated with colored/distorted images.
I have to agree with divendra … Cloudmark kicks butt! You can get Cloudmark’s Safetybar (Outlook & Outlook Express) thru its partnership with PayPal (free spoof protection forever, free spam protection for 30 days). Click here for more info http://www.cloudmark.com/paypal.
With Cloudmark, SpamArrest is a little overkill. But it does a great job too.
What I don’t understand is that if no one ever acts on spam or pop-ups (as in buys something, clicks on their links, orders Viagra, etc.) spam and pop-ups should have died years ago. It must be working, so do we have only ourselves to blame?
Couldn’t agree more Om. Something needs to be done. The SMTP protocol was designed to be flexible and allow email servers to easily pass email to each other, but it needs a major overhaul with some sort of unbreakable authentication scheme. Perhaps something like digital certificates through a trusted 3rd party (Verisign)
This is why I recently surrendered my 13 year old email address to the spammers.
I surrender my email address to spammers
This post inspired me.
I’ve now forwarded all my other emails accounts to Google. This means all email 1) gets read in one place and 2) has ben spam-killed twice, at the originating server and by GMail. After a day, there’s practically no spam.
My blog thoughts on it
http://polizeros.com/2006/12/08/killing-spam/
I’ve found the best way to deal with spam is white list filtering. This is where you only accept email from known senders, and it blocks over 99% of spam.
No matter how smart the spammers are, they can’t know the email addresses of all your friends!
Spammers rely on the fact that no filtering system is perfect, especially when their message is in an image. You can avoid having to read their lousy messages however, by reading your emails in text instead of html format.
Please post comments to http://hiphil.cashhosters.com/blog/index.php
In our startup we use Gmail as hosted spam filter. This is works great and we have 0 false positives. We use the setup as described at http://www.iopus.com/guides/gmail-spam-filter.htm
I know the post is a bit old, but the thing I find most annoying about SpamArrest is that Gmail seems to always (or very frequently) mark SpamArrest’s verification requests as Spam.
Therefore, Gmail users who mail me usually don’t see those verification requests until they paw through their spam folder, if they ever do.
Not SpamArrest’s fault, but it’s irksome nonetheless.