ISP Operator Barry Shein Answers Spam Questions

Barry mentions his "sender pays" spamfighting plan more than once in his answers to your questions, and discuessed it at length in an InternetWeek.com article published on Feb. 20. Is Barry's plan workable? Do you have a better idea? Or should we all just get used to spam as part of the online experience, and learn to live with it and block it as best we can?

1) Back to the 90s
by gylz

If you had known back in the early 90s that spam was going to be the problem it is now, what steps would you have taken then to protect yourself and others from it?

For instance, what changes would you have advocated in the mail protocols and what standard procedures would you have told other ISPs to use to prevent spammers from getting a foothold in the first place?

Barry:

When The World began selling the first commercial dial-up internet accounts in 1989 one question we were frequently asked by the privileged few who had internet access was: How are you going to control them? To be honest, we never had a good answer other than developing what everyone thought was a pretty good AUP (Acceptable Use Policy) and promising to enforce it as best we could.

But even as the net developed, in the early-mid 90s, there were similar problems with system cracking and break-ins. Back then there were more open holes to just walk right through, get a privileged shell, or just cause mayhem. To a great extent spam can be viewed as a form of system compromise and similar to malicious cracking in many ways.

One of my pleas back then to other ISPs was to make some sincere effort to know to whom you were giving accounts. Many of the ISPs with big funding and marketing departments to match would just give out new accounts to anyone with a drink coaster and worry about it later, oftentimes much later only when the bill wasn't paid.

I think practices like these gave rise to the sense of anarchy and lawlessness on the net that came from the easy abuse of anonymity which persists today. At The World we were careful about not enabling new accounts until we were pretty sure we had valid information. Many ISPs did not do this and tracing problems back to an account on their service would lead to a dead end; the info they had on the account would turn out to be obviously fraudulent.

Also, and this isn't a regret but more of an observation, some early internet advocates wanted only end-to-end services which basically meant that every single computer on the net should be a mostly autonomous client and server. Dial-up made this impractical; you couldn't really run a web site or even a decent mail server over a part-time connection. But I think some of that ambivalence over goals contributed to inaction on issues which might have helped with problems we see today.

2) Acting Locally, Effecting Globally
by merlin_jim

Many posts talk about proposed changes to society, government, and technology to lessen the spam problem. However, an ISP has more insight into the problem than many others, and I thought I'd ask a question to tap that insight:

Given today's society, technology and infrastructure, what can an individual do that would be effective in reducing not only the personal strain of spam, but also lessen an ISP's burden.

What kind of strategies have you seen work. For instance, in particularly bad instances I'm prone to send an e-mail to [e-mail], [e-mail], or [e-mail], but usually never even get a response. Is there a better thing to do? Are there things that are absolutely the wrong thing to do (such as replying to a spam)?

In short, what would you like to see users do in response to spam today?

Barry:

Pressure your legislators to enforce the laws already on the books! Hijacking others' systems, identity falsification, and fraud are already illegal. These aren't legitimate business people who send all this bulk mail, they're crooks.

Even if a spammer can sneak around the laws making it clear that the activity is illegal, this prevents a spammer from getting investors, incorporating, taking out bank loans, obtaining legal indemnification against liability, buying business insurance, registering with their state or owning intellectual property (e.g., trademarks), etc.

Something else everyone can do is install spam filters. And help others install spam filters. Ultimately, I believe it's an arms race between the filters and the spammers so other forces need to be put into play.

But my reasoning is that utilizing filters now will make the internet experience more pleasant and productive for many which is a good thing. Their wide-spread use will also serve as a wake-up call to those companies who are deluding themselves into thinking they're "white-hat" spammers so ought to be exempt. The filters throw their stuff away also.

The so-called legitimate advertisers need to get to the table with the ISPs and figure this thing out and stop thinking the status quo serves them.

At this point my thinking is that there isn't much difference, from the point of view of an ISP, between companies whose spam you don't hate and those whose spam you do hate.

When it's paper mail you have to put a stamp on a letter whether the intended recipient asked for the mail piece or not. I think we need to move in the same direction on the net with all bulk e-mailers. They need to start paying for the infrastructure they're exploiting.

The current situation is that people tend to define "spam" as e-mail which promotes products which they don't want others to think they want. We need to get beyond that because you're paying for any e-mail you receive, even if only indirectly.

3) why not whitelist?
by Aviancer

Why hasn't any large ISP or enterprise seriously considered whitelisting mail? The traditional blacklist idea -- when I see spammers I'll no longer accept their mail -- is so easily overcome that many spammers don't even wait one generation to change addresses. Instead, bounce all mail you don't recognize, with a note to the sender on how to inform the system that you are a real user. Nearly all spammers loose their incoming account immedately, so this seems the natural choice. There's some more detail on this method at the TMDA project.

Barry:

The easy answer is that the target moves too fast. How could we begin to keep up a whitelist at the ISP level on behalf of thousands or even millions of customers?

And how exactly do you propose to "inform the system that you are a real user"? Right there is the crux of the matter. What you're suggesting is one of those techniques which works pretty well for individuals but is unmanageable at the ISP level.

Something from the TMDA site I do agree with is:

Spam will not cease until it becomes prohibitively expensive for spammers to operate.

We just have slightly different approaches to making spam prohibitively expensive. Let a thousand flowers bloom!

4) Is there a reasonable solution?
by PincheGab

Given that junk mail in the regular mail is more acceptable (and I will mention that my wife (specially) does like to know when there's a sale on), and given that e-mail is the next big thing, what do you see as an acceptable solution/accord to spam?

I certainly am tired of deleting the penis enlargement and Nigerian bank deposit e-mails, but where is the balance and how do we attain it, if ever?

Barry:

I believe the only approach which will work is a "sender pays" model for bulk e-mail advertising. Such a model corrects the current situation on several levels:

a) Sender pays can provide an economy to enforce its own rules.

Most proposals I've seen to deal with spam are workable on paper but fail in this regard. If, when considering yet another spam proposal, you ask yourself who will pay for this or that solution, how will it be enforced (e.g., if it requires lawsuits who will pay the lawyers?) generally no answer comes to mind.

However, if we create a (bulk) sender pays model through some sort of trade association then that organization would have a revenue stream which can be tapped to enforce its revenue model, and a monied interest in defending that revenue model.

b) Sender pays creates a conduit of control between the sender and the ISPs.

Right now spammers can use an ISP's facilities to firehose any spam they want, to anyone and everyone they like, at almost zero cost. For example, kids' accounts are flooded with explicit pornographic come-ons. There's no ability to control that sort of thing.

What business allows its facilities to be used to offend its customers?

In a sender pays model one could also refuse to be paid and, hence, refuse the advertising. Spammers are trying to send their spam to the ISP's customers. I think the ISP has both a right and an interest in controlling that so as not to drive customers away. It's not reasonable that an ISP such as myself has no control over what sort of advertising is placed in my customers' mailboxes yet is left responsible for the quality of that experience.

c) Sender pays clarifies the legal situation without a need for new legislation.

Sending, and not paying, would become simple theft of service, wire fraud, etc.

5) ISP Tools
by feenberg

Do ISPs have the tools they need to prevent outgoing SPAM from their own customers? I look at Sendmail and don't see anything that would allow you to throttle mail volume, check outbound messages for SPAM, restrict new customers etc. There isn't even anything built in that would warn you about a customer sending a million messages. It would seem that a few tools like that would be a big help to an ISP too small to develop its own.

Barry:

I think the best tool is knowing who your customer is and having a clear and effective policy if a customer spams such as clean-up costs which should also include intangibles such as public relations costs.

But you're correct, better tools at that level might help if ISPs were inclined to use them. Many ISPs do use tools such as you describe, others obviously don't care.

6) RBL's
by sabri

One of the few measures that can be taken against spam is the use of blacklists (for instance via DNS). There are a lot of pro's and con's for the use of DNSBL's. How do you feel about these? Should DNSBL's be governmentally regulated? Do you use any DNSBL? Should an ISP enforce certain RBL's (let say, of open relay's) on its customers?

Barry:

I've always resisted using these blacklist services at the ISP level. There are several reasons why but the most important is control.

If the blacklist suddenly began blocking some site, such as a major university or corporation because it was the source of spam the night before, that might cause a big problem with our customers. Even if it could be worked around it'd be just another out of control detail which might send one into fire-fighting mode suddenly.

Another problem I've had with blacklists is that some have become rogue and gone power-mad, blacklisting addresses for reasons completely unrelated to their stated purpose such as personal politics.

Also, the blacklists I've looked into were volunteer efforts which meant the people involved often felt they could paper over any mistake or oversight or staff unresponsiveness with the excuse that they were unpaid volunteers so what do you expect? You can't have your ISP be dependent on organizations with that attitude. And what if I don't like a blacklist's policies or implementation of their policies? If I'm not paying them I can't vote with my wallet.

I suspect that anyone attempting to run a blacklist in a professional, paid manner would go broke; the service isn't worth what it'd have to charge to stay in business. The legal costs alone can be daunting. With legal issues even if you're right it can be expensive getting there. And customers of any service don't want to pay for your legal bills as the major cost of such a service. So we're back to problems with the economic models.

I don't think government regulation would help with blacklists, per se, except in very general ways (they can run the courts for the lawsuits!) The only analogy I can think of are credit bureaus but most of the government regulation in that area is to protect consumers. I don't think we want the government stepping in to protect spammers!

Finally, yes, just about all ISPs blacklist (block) offending sites. Doing it in-house gives them the control they need. It's not great to have to take this on but it's the only choice right now. Unfortunately it's becoming a major burden, and the results are not altogether predictable.

7) What would be the minimum actual cost?
by jamie

What would be your actual dollar cost of spam, if you didn't spend much time and effort fighting it?

Let me explain...

I sometimes hear that spam has significant costs in bandwidth and storage but I don't believe it. As far as I can tell, SMTP traffic is at most 2-5% of net traffic. And a quick calculation shows that an ISP's costs for storing its users' spam are fractions of pennies on the dollar. (*)

You've likened spam to a DDoS attack on your mail servers. Stories about being flooded with traffic sound impressive but computers are so fast now, it's hard to put anecdotes into context. So I'm looking for dollar amounts. For a customers paying b dollars per unit time, an ISP like yours has to spend c dollars per unit time on servers that can handle those customers' incoming SMTP traffic. If this is significant, I'm looking for c over a times b :)

Obviously admins to run the servers are an important cost. But for purposes of this question, suppose you wanted to do the bare minimum. Say you set up the SMTP servers to use just a few of the less-intrusive DNSBL lists, like sbl.spamhaus, relays.ordb, or list.dsbl, and then ignored them as much as possible.

The next most common argument I hear is that customers will abandon ISPs that don't fight spam. But every ISP has the same problem, so this is really a competitive advantage issue except for the small percentage of users who are actually driven off the internet by spam.

Then there's outgoing spam but I don't imagine that's too hard to recognize and stop quickly.

Let me know what I'm missing...

(*) Thumbnail calculations of spam storage follow. Let's say J. Average ISP Customer gets 20 spams a day at 10K each, and deletes them only every 30 days. That's an average of 20*10K*15 = 3 MB of storage. If the ISP replaces hard drives every two years on average and its total storage costs are ten times the actual medium costs (for labor, backup, redundancy, downtime), then at today's hard drive prices, that spam storage will cost the ISP 0.003 * 10 / 2 dollars, or about a penny and a half. Over that same year, J. Customer pays the ISP $100+.

Barry:

Your figures for the percentage of bandwidth which is spam are far too low. Others have put the numbers much higher. NewsFactor cites studies putting the figure somewhere between 17 and 38%. See http://www.ecommercetimes.com/perl/story/19803.html.

As to computers getting faster, that's not a primary issue in my mind. But addressing even that point, how rapidly should I have to amortize and replace my equipment just to accommodate spammers?

And what about the intangibles? They're becoming the major factor in all this. E-mail is the "killer app" on the net. Yet spam is fouling that e-mail experience.

People reading Slashdot might be sufficiently committed to e-mail that they'll wade through all the spam and tweak spam filters even if it takes hours per day and a clothes pin on their collective noses. But what about the many millions of people who aren't so committed to this technology?

As an ISP I can tell you they're giving up on the internet, to them the cost/benefit is just not worthwhile. That's not a good trend.

Another cost is that spam is undermining the standardization of protocols on the net, and thus introducing a pervasive chaos. Every ISP and many other sites are scrambling around implementing mostly different "solutions" to the spam problem. Some of these in-house solutions might be ok, others can be pretty bad.

One result is that e-mail is becoming less reliable as a communications tool. Your mail might get through, it might be kicked out or filtered as spam, you might be able to figure out why and get the message through on a slightly changed subsequent attempt, or maybe not.

Who needs this kind of craziness? How can this situation possibly be productive?

How productive is it to have millions of people installing and customizing spam filters? Or having really bright people writing spam filtering programs? And where is this all going?

In my opinion, if unchecked, I think the current trend is very destructive to the entire idea of a public network.

P.S. I realize in another answer I recommend installing spam filters, but I see that only as a temporary measure.

8) Collateral Damage
by aridhol

One of the greatest problems with spam-prevention techniques has to do with collateral damage. Can you see any solution to spam that either prevents or minimizes the damage to innocent bystanders, such as other users of a spammer's ISP?

Barry:

Yes, the solution I favor is going to a sender pays model aimed at bulk e-mailers.

Other approaches, in particular technical solutions, are prone to causing collateral damage. Inevitably as the arms race heats up, and spam filters have to take bigger and bigger risks to have any effect, collateral damage will become more common.

And it's already worse than you might imagine. Spam and similar are causing severe operational problems on the net and undermining standards as ISPs and others invent new ways to avoid the spew.

As one concrete example, right this minute there's a network provider who was just assigned most of the 69.0.0.0/8 IP address space. Unfortunately, this was formerly a spam and DOS (denial-of-service) cesspool so many sites out there just block the whole 69.* address space.

So the new owners are making appeals to firewall managers asking them to please remove their blocks in the 69.* space on the NANOG (North American Network Operators Group) list.

But NANOG is not a particularly big or influential mailing list. At best it's only aimed at North America while the blocking exists world-wide. But how do you communicate with so many sites and undo the problem? In a nutshell, you can't. I suspect their customers who get space in 69.* are going to find themselves blocked by many sites for many years to come.

See what a mess spam is causing? It's like asking how much can such a little tiny termite eat? And then the house falls down.

9) Spam Lawsuits
by ca1v1n

Do you think new laws that allow ISPs and end-users to collect damages from spammers on a per-message basis can be effective tools to reduce spam?

Barry:

Although it should be part of the picture I think this sort of litigation would be ineffective as a primary attack on the problem.

What we need to do first is stop the insanity!

To do that I say introduce sensible economics into e-mail advertising. You may find network TV commercials annoying, but imagine if just anyone could break into a station's signal at any time and insert advertising! That's what we have right now, and it's crazy.

If we were subjected to a few, well-paid and placed ads it might be annoying to some but others might even find it beneficial like the person in the previous message whose wife likes to know about the good sales. Or we could just pay a premium and not see another ad, analogous to premium cable TV. Or find ways to block them via our personal mail clients, analogous to what people do with PVRs. It'd just be a matter of economics and marketing and taste.

But right now it's complete anarchy, only the introduction of a viable economic model can tame the situation.

Also, I'm not optimistic about any legalistic approach so long as there's no scalable revenue stream associated with e-mail or its abuse.

Currently the general consensus on the net is that we don't even want sales taxes on e-commerce, which might be a reasonable point of view, but then we're going to ask that billions should be spent on courts and enforcement of new spam laws? Where is that money supposed to come from? Cut the fire dept? The schools? Not-growing corn subsidies? Without additional revenue something has to give.

Given a sender pays model money could be earmarked for private enforcement, such as investigation and litigation. And the case could be more realistically made as to the exact economic cost of spam. If an ISP was supposed to get paid for ads going through their system then anyone evading that is simply guilty of good old fashioned theft of service, no new laws needed. And legislators, who presumably would be getting their usual business tax cut of such revenue, could begin to see the logic in returning some tax money to defend these revenue streams.

There would still be challenges to be worked out internationally but it wouldn't be the first time a revenue model had to work on a global scale. Obviously international telephony and postal mail works well enough to combat fraud. But only with some sort of concomitant revenue stream attached to the activity could you possibly begin to tackle the problem, domestically or internationally.

10) Kill 'em all
by Lord_Slepnir

If you could meet a spammer, what would you say? What would you do? What caliber would you use? Would you want someone to do it for you? Is $10,000 a head too much?

Barry:

I would tell the spammer in no uncertain terms that spammers' days are numbered, just like junk faxers and other scam artists who exploited a brief window of vulnerability.

Situations like this don't last long.

Of course, then the spammer would laugh in my face because that's what sociopaths like to do when confronted. But, as the expression goes, we'll see who laughs last.

One thing is clear, however, spammers will not listen to reason. So any change in their behavior will have to be the result of force.

Simple Solution

[Anonymous Coward]

There is a simple solution (or at least a starting point): Prosecute Forged Headers. If someone is going to send spam, sending it from someone elses account or server SHOULD BE illegal, and it should be prosecuted. This would cut down on large amounts of spam and make all other forms of combatting much easier.

throttle an smtp server

[tinla]

"I look at Sendmail and don't see anything that would allow you to throttle mail volume"

ISP's offering dialup services generally know the CLID and maybe the name & address of a caller.. but its too much hassle to do anything about bulk mailers that use the service. If I go and sign up with a free isp I can send a huge volume of spam before I get banned and there is a very low chance of any comeback.

What tools are available for SMTP gateways (such as sendmail, exim etc) that let you trottle mail based on the sending address / user (maybe tied into radius)? So i can allow normall users to send thier 20 messages per connection by automatically make it unattractive to people sending 1000's. If each subsequent message from a user has a longer and longer transmission time (insert some arbitary delays etc) then they won't relay through the isp server.

Any ideas? I was talking to a friend recently that works at a small isp and he has the exact problem above. They give out "free" accounts (earning off the call revenue) and spammers clog up the smtp server with really vast volumes of junk in the mail queues... after all - most addresses on spam lists are duds.

thinking about sender pays

[j0nb0y]

The problem this has is that people don't want to pay to send email. I think the solution to this may be for each account to get so many free emails a day. For example, you can send ten free emails a day, but after that you pay 10c each. No spammer would get an account at such an ISP.

Another solution may be to have a ten cent "deposit" every time you send an email. If it's legit, you get it back. If the end user rejects it, you lose your ten cents.

The problem with the first approach is that it wouldn't work unless every ISP did it. It would make more sense to charge at the incoming mail server.

Re:Sender pays is a bad idea

[dattaway]

I know a better idea. And the reason why I love local ISPs. One our LUG mailing list got spammed. Within the hour, I got an address and it was from someone here in the city. Let's say it was an interesting experience getting to know this spammer and observing her habitat. Yes, it was a internet luser wanting to exploit the masses.

Get to know your spammer. Field trips are more entertaining than sitting on the couch watching television.

My Spam solution - worth thinking about?

[MrFredBloggs]

How about if people had 256 character email addresses? Or 512 bytes? Or 1k? You wouldn't care, as you`d be using your address book. A spammer can use a dictionary to create email addresses and spam them, and be sure a lot exist. But not this way - chances are it would be a non-existant one.

Re:throttle an smtp server

[Anonymous Coward]

postfix has this built in.

Re:White list problem

[mark_lybarger]

why? because it would mean the mr. ISP would have to get up and provide a competitive, usefull proudct to consumers.

i believe the first few ISP's that step into this space will pave the way for the rest.

Problem with sender pays

[leviramsey]

Is how sender is defined. Is any mail of a commerical nature the definition? Would an email from my stockbroker (ie one I hire) suggesting that I sell MSFT qualify? If it's limited strictly to bulk, where is "bulk" defined.

Yes, certain varieties of spam may be eradicated, but the spammers will simply move on to other varieties that aren't covered.

White list white list white list!

[jonniesmokes]

Putting a price on sending mail will make it difficult to get email from peoples who don't have similiar economies. It'll stratify the net based on the dollar value of the people. That's not a good thing.

White lists would require a spammer to spend a small amount of his or her time to get a valid send-to address. But it wouldn't cost a penny. Just time. And if you don't send spam on that valid email, it'll be good for a long long time.

Come on - Isn't this a good solution?

setup a trap open relay??

[AwesomeJT]

Well, could someone open a modified sendmail relay that only logs connections and attempts to send spam -- that way you'll have a good idea who is sending the spam (at least the ones dumb enough not cover their tracks before it gets relayed). Then you could DOS or hack that system and disable it. Or at least find the owner or ISP of the IP address. Could be a fun experiment.

Sender pays

[DonkeyJimmy]

I'm not so sure I trust this idea. Who are we paying? If the sender pays, then I want the receiver to get paid. Either by everyone getting reduced ISP rates, or just getting to send a free email for each one they receive.

Then, for someone to spam 50,000 people, they'll want to find away to get 50,000 e-mails sent to them, probably from other spammers, and spammer will be fighting spammer in the arena of the web, while us "innocent bystandards" will be making money. More likely, spam will just stop and everyone will send about as many emails as they receive.

Re:"Sender pays" should be universal...

[rgmoore]

Of course it should be universal. But the brilliant part of sender pays is that you can set the payment low enough that it won't be much of a factor for ordinary users, but will be terrible for spammers. For instance, $0.01 per message is unlikely to break the bank for any ordinary net user, but it's enough to significantly raise costs for somebody who's depending on sending out millions of emails for results. An interesting twist on the basic concept that I've considered is the logical conclusion of the sender pays argument: sender pays and recipient gets paid. That way you'd be fine as long as you receive more emails than you send. As a practical matter, most ISPs would probably implement a policy that you wouldn't get a rebate if you received more mail than you sent, and they'd only pay you if they got payment from the original sender, but it would let the process trickle down to ordinary users without greatly inflating their monthly ISP charges.

This would also potentially be able to save mailing lists. One obvious problem with sender pays is that it would make it prohibitively expensive to run an ordinary mailing list. By giving the money to the recipient, though, you could let the lists recoup most of their sending expenses; users would just return a blank message (or a return receipt) every time they got a message on the mailing list, which would send the penny that it cost the list to mail them back to the person running the list.

How is e-mail different then snail mail?

[xtheunknown]

I am not one to take sides with the spammers, but I have one question? How is e-mail different then snail mail?

Recently I moved to a town where there is no home mail delivery. I had to get a PO box. After a few weeks, I found that I was getting more direct mail then mail addressed to me. I asked the post office not to deliver this mail, but they refused saying that those companies paid to have mail delivered to me and therefore they couldn't stop delivery of junk mail. The only way I could get them to stop was to write each junk mailer and request them to stop sending me unsolicited mail.

Well, I don't think I am going to send each one a letter so I just put up with it. How is this any different than spam?

Comment removed

[account_deleted]

Comment removed based on user account deletion

Mailing lists?

[pdawson]

Here's one I didn't see an answer to: How would a sended-pays system handle mailing lists such as majordomo or yahoo/egroups setups? Any ideas?

Spammers with slashdot accounts (mod down parent)

[ergo98]

I am absolutely, positively convinced that spammers hold Slashdot accounts and come here to try to convince us all that it's no big deal, and we should all just live in peace with messages with forged headers, fraudulent subject lines, web bug images intentionally intending to circumvent processes to avoid them, criminal "removal" processes and activities, often pornographic content that could get one fired or put under surveillance (nothing like a big set of tits appearing on your screen when you open the "FW: Budget proposal" message. This ignores that these are sent out to anyone and everyone, including minors), a business that is almost entirely supported by pump-and-dump schemes, bogus snake oils, and outright illegal pyramid (or similar schemes), and a illegal use of other people's hardware and bandwidth resources. How else could someone be so unbelievably stupid to go running around ranting about how it's no big deal? No big deal indeed. Shall I bend over?

Hilarious "irony" alert: Isn't it funny when people specifically go into discussions ABOUT SPAM to complain about how they're sick of "hearing people complain about spam": Hell, you don't even have to "hit the delete key"-DON'T COME INTO THE BLOODY DISCUSSION. Unbelievable. Then again, I'm just sick of reading messages complaining about spam in discussions about spam...

Re:Simple Solution

[gilesjuk]

Can't mail servers become more intelligent, after all who sends 100 emails a day in one big batch from a hotmail account?

If a hotmail server for example looked at the age of an email address/account when deciding to accept an email then it might prevent some spam. If an email account is minutes or hours old and the user attempts to send 1000 emails from it then you know you're dealing with a spammer. Also if you check for replies to that message you also can validate if the mail was potentially a valid one.

I personally receive very little spam since installing Popfile, in the the true tradition of networking and the internet it's free. Works hundreds of times better than any commercial tool I have used. Best thing is it's cross platform too, if you can run perl on an OS you can run popfile.

Spammer pays...

[YrWrstNtmr]

...who? The ISP? Nice revenue stream. Some would actively solicit spammers for the increased revenue.
And then some ISP's would simply put it on a flat rate. Or advertise "Send free emails!", and charge an extra dollar on the base price.

The recipient? hmmm.....there would end up being some kind of reciprocal agreement between spammers. "I pay you, you pay me."

The poor user whose box gets cracked and becomes the sender of a million emails is in for a fight getting his money back or account reinstated.

And then of course the valid bulk email senders would wither and die.

Who pays, and who gets 'paid'?

Sorry, but there are far too many loopholes and traps for paid email to actually work.

I don't buy spam costs estimates either

[groomed]

I'm getting quite fed up with all the anti-spam rhetoric around the 'Net. All kinds of figures fly around as to the cost and magnitude of the spam problem, but most of them are obviously biased and the methodology by which they are obtained is genrally fuzzy at best. It reminds one of the figures quoted by the BSA for software piracy, or the figures quoted by the RIAA for music piracy: that is, they factor in all kinds of "intangible" costs, are based on questionable assumptions, and are impossible to verify.

It is clear that spam is a nuisance. But spam filters work miracles, and they don't have to be fashionable Bayesian classifiers either. Simple treshold or trigger based filters work extremely well for individual mail accounts. Such as junkfilter [zer0.org], or SpamAssassin [spamassassin.org].

Now some people will argue that filters don't solve the problem: by the time the mail arrives in somebody's inbox, the damage has been done, the network resources have been wasted and the CPU time has been spent. But that argument is meaningless without a means to quantify the costs. And again, where are the figures? How can we even reliably estimate the figures?

It stands to reason that many people benefit from inflating the costs of spam. Meanwhile nobody questions the figures because everybody hates spam. Notice how Barry manages to almost, but not quite, evade question #7 in this interview.

Spam: the non-issue that everyone loves to hate.

Re:Money talks

[Eric Savage]

Your message is somewhat of a contradiction:

"If no-one ever responded to SPAM, it would die out pretty rapidly."

"There is an endless supply of spammers who have yet to realise that it doesn't pay"

The first is false. The second is true. What people don't realize is that spam is NOT about someone sending mail through an ISP's server! This is easy to stop, and most ISP's have had countermeasures in place for years. Unfortunately every hacker out there thinks he/she has the solution through limiting these things.

Most spammers don't use open relays, and they don't use their ISP's mail server!

So if we could please move on, the problem is that spammers are doing the same thing as many people on slashdot, they are running their own mail server off their cheap (often free thanks to parents, and yes I am asserting that much spam is from teenagers) broadband connections, or as Barry mentioned, setting up a colo and blasting out 10 or 20 megabit of spam until the place gets enough heat to shut you off (and rent the box to the next spammer).

If we could start thinking about this problem in terms of 2003 and not 1998, we might make some progress.

Preach on!

[The Bungi]

Another problem I've had with blacklists is that some have become rogue and gone power-mad, blacklisting addresses for reasons completely unrelated to their stated purpose such as personal politics.

Those of you who run one or more domains from which spam NEVER generates, and who's had some asshole wacko submit said domain(s) to a RTBL, raise your hand.

If you think RTBLs are great, wait until you get on one of them. It's almost Orwellian. Amazingly simple to be in one, incredibly difficult to get out.

Granted, some RTBLs are run responsibly. But a few are nothing more than power trips for the operators - people who've become essentially hysterical about spam and are quite glad to condemn anyone with minimum proof. An IP address in a forged header, fer fuck's sake.

Police, prosecutor, judge, jury and executioner, all rolled up in a wonderful little bundle of joy.

People who buy into spam

[phorm]

How about if ISP's had some way of tracking those that respond to SPAM-sent offers, and had a clause making those customers liable. After all, it's the 1 in 1000 users who respond to spam that make it profitable.

Re:Sure

[AssFace]

To be fair, I'm not sure that is what drives spam revenue.

It is true that television and print ads are driven by the desire to sell the product that they advertise.

But I think spam has evolved to the point where it is not about what is being "sold" and instead about creating the address base.

If you look at a large percentage of the ads, there is no real way to buy what they are referring to. If you click on the links, they bring you to a page that then lets them confirm the address that they sent to (yours - and by clicking on that link, they know that you exist and it is a live address). Can you actually buy anything on that page? Not usually - usually it is a way to unsubscribe, or it is a series of ads for other things, or it pops up a ton of other crap - eventually leading to porn.

The point of spam might have initially started as an attempt to get people to buy crap - like an annoying informercial or a qvc that keeps coming back at you.

But the money making part of it now is to generate a larger list of users with more real users in it and then sell that list to other spammers. As well as generate revenue from popups and click thrus.

If you genuinely look at over 90% of the spam you get, you will see that were you to want to buy whatever it is that they are talking about - there isn't a way to get to it - you will be diverted many times along the way.
There are obvious exceptions to the rule - Netflix, porn, and geroge foreman grills come to mind.

I am down to about 100 spams a day now and I go through them frequently to try and think of more ways to get rid of them (I use the most recent spamassassin), and have come to notice this trend on my own.

Regular web advertising failed because nobody was buying the product - banner ads and the like never really caught on - but they remain out there - people trying to get your attention.

Whereas spam has grown the entire time - because they have found a different way to make money - not selling you the product, but by selling your address.

Spam and mail percentages

[jamie]

Hi Barry,

Thanks for doing this interview :)

I'm not really satisfied with your answer to my question about dollar cost of spam, but that's OK, you don't have to satisfy me :)

I did want to clear one thing up. I had written:

"As far as I can tell, SMTP traffic is at most 2-5% of net traffic."

And you responded:

"Your figures for the percentage of bandwidth which is spam are far too low. Others have put the numbers much higher. NewsFactor cites studies putting the figure somewhere between 17 and 38%."

I totally accept that spam is about 17-38% of SMTP traffic, that sounds roughly correct to me.

My point there was that SMTP traffic is a very small fraction of total net traffic.

I haven't found any recent statistics on this -- partly because I don't think anyone publishes these numbers anymore, and partly because it's a real pain to try to find with Google. (Do a search on "SMTP NNTP HTTP bandwidth backbone" and you turn up a zillion ISPs bragging about all the protocols they support and how many backbones they're connected to.)

Here's one example of the crappy data out there, a six-year-old report from a link near a backbone showing that SMTP traffic totaled 2.2% of all network traffic:

http://www.nlanr.net/NA/Learn/popular.html [nlanr.net]

Here's another survey of a backbone, this one five years old, showing SMTP traffic as 3.3% of all network traffic:

http://traffic.caida.org/Reading/Papers/Inet98/ [caida.org]

My point was just that if we're trying to assign a dollar figure to what spam costs an ISP, we might as well ignore connectivity charges, because SMTP itself uses so little bandwidth.

As for what all the other costs add up to... I still don't know.

I wonder why he dodged question #7.

[tempshill]

What would be your actual dollar cost of spam, if you didn't spend much time and effort fighting it?

He didn't actually answer the question. It's too bad that we don't see more actual analysis rather than opinions. Are we really concerned about people in our society who are "on the fence" about e-mail and might decide not to use it, at all, because of spam?

Like the poster of question #7, I am also skeptical of the actual cost.

PS: I don't like spam

Re:Money talks

[Eric Savage]

Do you have any facts or specific experience to back this up, or are you talking out of your ass?

The experience of looking through thousands and thousands of spam headers and source info. Almost all large spam attacks come from colo or broadband.

Spamming directly from your own broadband connection is retarded; that's why most spammers don't do it.

I'm sorry but I beg to differ. Most ISP's don't shut spammers off unless they get actual phone calls from enough users/admins. Typically they stall for 24 hours or so at which point the spamming has stopped and they say "Sorry, its stopped, there is nothing we can do". The ISP you worked at obviously took it more seriously than just about every one we have dealt with.

Shutting them off sounds like a plan, but that's not the practice. Even if it got to the point where they get shut off, they just call the next provider. In major areas there can be a dozen or more broadband providers. After a few mistakes the spammer gets wise to policies and becomes even harder to detect, or has moved up to colo, of which there are hundreds of choices available to them.

Don't just cancel their account, but make them pay

Again, sounds good, but remember that this activity isn't illegal so if the fines become too much the spammer will take the ISP to court and likely get the fine reduced or eliminated (re: adhesion contracts). Also, spammers aren't exactly the most honest businesspeople, so the credit card they gave you, if it was legit, was probably cancelled and your legal department will laugh at you if you want to pursue anything more than standard bill collection on a few hundred bucks.

I don't think he understood question #3

[mjh]

I think this guy really understands TMDA [tmda.net] when he answered question #3 like this:

And how exactly do you propose to "inform the system that you are a real user"?

The answer to his question is, "By using TMDA, of course!". TMDA is an automated whitelist management program. I agree that manually managing whitelists is next to impossible, even at the individual level. But that's why TMDA exists, to automate that process.

And it's currently being tested on a large scale. GMANE [gmane.org] is using TMDA [gmane.org] as a mechanism of blocking spam for some 3500 mailing lists.

I wish I could rewrite the original question so that it was more clear that TMDA is an automated whitelist management program. Cuz I don't think the guy understood that. And he answered as if the question were suggesting that the ISP manage all their user's whitelists.

$.02

Re:throttle an smtp server

[psyki]

In my experience working for an ISP there are three main avenues of spam an admin should be worried about.

1) Spam sent through the customers open-relays, or hijacked mail servers. This is truly the worst of the worst, and almost always unsolicited.
2) Customers who send from their own SMTP servers. We kept an eye on these guys and generally allowed people to do it unless we got complaints or if the mailings were illegal (forged headers or missing/non-functional unsubscribe link).
3) Customers who use the ISPs SMTP server. Of the three this was the least of our concerns by far. This was prohibited in our TOS and we were pretty strict. One warning and you're gone.

So given these three types of spam delivery, what could an ISP do to prevent it? Since only a very small fraction of the spam that passed through our network went through our SMTP server, throttling it back or monitoring it wouldn't have done very much. And tracking down our customers open relays before spammers do was never really effective, because most people were either too incompetent to understand what an open relay was (uhh, I'll call our "consultant", he set up "the server"), or didn't think it was a serious enough issue to do anything about it. Most of the people that understood what an open relay was and realized its potential for abuse had either already secured it properly, or secured it after our discovery.

Economic vs. Technical vs. Educational solutions

[quikgrit]

[Anti-rant Disclaimer: This post is knowingly U.S.-centric in its discussion of legislation proposals and user attitudes, just an FYI]

I think it's important to address the potential solutions that are being discussed from a slightly higher level than is being done.

Personally, I tend to categorize the solutions I've heard into one of three concepts. Here they are, with a few examples:

Economic:
- fine spammers
- implement a 'pay for email' system
- legislation (usually economically punitive)

Technical:
- spam filters
- blackholing ISPs/open relays/DNS blocks

Educational:
- teaching people the consequences of handing out their email addresses. (OK, this being /., I should probably say "lecturing" instead of teaching.

Obviously, there are more solutions, but those few examples should illustrate what I'm getting at. I have yet to hear a proposed solution that doesn't fit in one or more of those categories.

The problems with these categories tend to be:

Economic - restriction of freedoms
Technical - cost
Educational - cost

Ideally, we'd just use an educational method. We know that the only reason that people spam is because they believe it is profitable to do so. They believe it's profitable to do so because they believe that people will respond to their spam. As much as I hate to think it, it's hard to believe they'd just continue to spam without some proof that these beliefs are accurate. The educational method would seek to do one or more of the following:

1. Educate users not to let their email addresses get in the hands of spammers.
2. Educate users about how the spammers get their email addresses.
3. Educate users that responding to spam (whether to buy, or to just tell the spammer off) is not a good idea.
4. Educate the spammer that ... well, educating the spammer is probably not a worthwhile goal.

The problems you run into with this method are that:

1. It costs a lot of money to educate enough users such that it will not be rewarding for a spammer to continue to spam.
2. Some users (believe it or not) actually don't mind receiving spam.
3. Some users (believe it or not) actually *like* to receive (certain) spam, and purchase goods and services based on the spam.
4. Not all users are on the same end of the bell curve.

I don't know if the roadblocks allow us to pursue education as an option. My guess, after a few years of reflection, would be "no".

The latest fad in addressing spam seems to be economic solutions. Most often, this is in the form of either legislation with punitive results (See the $500/per unsolicited email stories), or ideas floated about taxing email, either publically (taxes) or privately (fees).

I call these fads because they are generally politically motivated or under-researched ideas. Legislation is fine and dandy, but the technical aspect of spam makes it very difficult to enforce. I am referring to header forging, spam-and-run tactics, identity falsification, sending from other countries, etc. The fact that the spammers have such amazing technical weaponry at their disposal makes legislation seem almost silly. Due to the sluggish nature of the legislative process, spammers will *always* be a step ahead, thanks to technology.

Eventually, of course, spammers will destroy themselves, thanks to this same technology. We will eventually reach a point where the technology exists for anyone to very easily spam everyone else, and the signal to noise ratio of SMTP traffic will approach 0 as a limit. At that point, email will be useless without filtering. But we don't want to go through this period before emerging on the other side of the rainbow. I am flabbergasted that Joe User isn't receiving far more spam these days than he is. If you had asked me a year ago, I would have said that I would not be surprised in March 2003 if Joe User averaged over 500 spam a day, instead of the more common two-digit numbers bandied about.

The other economic option of taxing or levying fees against email traffic falls victim to the same technology curve. It's just too hard to keep up. Even if it were, you then run into the labyrinth of *how* to implement such a solution, let alone enforce it. Should it be based strictly on the quantity of emails sent? Should the size of the email play a factor? Should people only have to pay past a certain threshold of mail sent? The questions go on and on.

The final category is technical solutions. End-user software, blacklists, DNS blocking lists, the 'block' button at hotmail.. This is where the vast majority of effort has been spent so far, and even with the rampant spamming that exists today, I think it's fair to say that a lot of these technical solutions have had a good effect. They've also had a lot of bad effect, in that there has not been a unified approach to resolving the problem, but at least people are addressing it, even if in a differentiated manner. These methods have a host of their own problems, as well - false positives (blocking non-spam), not catching everything, spam-and-run spammers, and so on. Any end-user based blocking device still leaves a ton of bandwidth consumed by every network involved, right up until the packets get dropped at the doorstep. Any non-end-user based device runs all the other risks.

So, what do we do?

I don't know.

Sorry, I'm not trying to cop out here, but I am trying to be honest. I don't know what the best solution is. Personally, I like the idea of whitelists (deny all unless explicitly permitted), wherein users don't receive email from anyone unless they already told their mail system that it's ok to receive mail from that address. Here's a walkthrough of my ideal end-user solution:

1. User implements whitelist, with set of "allowed" addresses/domains.
2. User receives email only from addresses on the whitelist.
3. If mail arrives and is not on the whitelist, it is bounced back to the sender, with a message from the user.

The way around false-positives (rejecting email you want - in this case, from senders not on your white list - let's say you handed out your email address to a new friend, or need to receive an invoice from a company) is to set up the bounce message with information indicating how the send can get on your whitelist. This could include an alternate email address, and agreement to a Terms & Conditions document (hells yes, send it right back at the companies!), or something as simple as a alphanumeric string to include in the subject line that would bypass the white list.

The alphanumeric string solution would look like this:

User sets up whitelist to deny all email unless either:
a. the sender's address/domain is in the whitelist.
b. the subject contains 'MYPASSWORD733'
c. bounce messages includes the password.

This prevents automated spamming (unique passwords per individual, in customized bounce messages), yet allows a sender to re-send their message immediately with the new subject to bypass the whitelist.

Can spammers get around it? Sure, if they read every single bounce message by hand. Or, if they get a program intelligent enough to understand and respond to the bounce message - but that's much harder than you think, if everyone customizes their bounce message differently. Worried about a spammer adding your email and whitelist password to a database and selling it? just change your whitelist password, and thier gathered data is useless. You never need to remember it, and it doesn't need to ever stay the same, because it's sent out in each bounced message.

Of course, this solution has problems of its own:

1. It greatly increases the amount of SMTP traffic.
2. It doesn't stop the spammers traffic from getting to the last mile, and consuming all the bandwidth in between.

That being the case, all I have to say is - bandwidth is cheap. ;)

Thoughtful replies encouraged..

quikgrit

Re:Sender pays is a bad idea

[njdj]

I get full color spam in my mailbox. Bulk advertisers have no problem paying a few cents per spamee, when one gullible shmuck in 1000 orders

You don't understand the economics of spam email. The response rate is much less than the 1 in 1000 that physical junk mail attracts, more like 1 in 10,000 or more. To make money, the spammer needs to send millions, not thousands, of emails. Currently that costs nothing (whether it's full color or not) per message. But charging even half a cent per message would completely destroy the spammer's business.
Furthermore, the effectiveness of charging would not be in any way impaired if 1,000 free emails per month were included in every paid subscription to an ISP. 1000 emails is far too few to be of any interest to a spammer.

Re:"Sender pays" should be universal or it won't w

[lar3ry]

I've been reading the comments, and it seems like the biggest objection to "Sender Pays" goes like "The Linux Kernel Mailing List will disappear if this happens."

But this objection is unnecessary: the truth is that mailing lists are no longer necessary!

Why does anybody have to send emails to 100 or 200 people? This is the stuff of horrific waste!

Instead, let's get the purveyors of email programs the ability to IMPORT messages into the mail queues. Simply connect to a server and download the latest stuff. Do it once a day, once an hour, or whatever. People with PDAs do this (think "AvantGo" and "Mazingo" and a few other services that hot-sync news from the web to a PDA for offline perusal). People with POP and IMAP accounts do this as well (connecting to a server on some regular basis to get email). It's not a big change from the current email GUI model.

For instance, let's assume there's a "Slashdot Mailing List" that basically feeds all the stories that appear on Slashdot. Your mailer can connect via some "well known protocol" (ftp, http) to the server that provides the mailing list. The mailer then imports the messages into its message queue, augmenting the messages it has received via IMAP and POP3.

How would this work? The mailer sends the last received message id, and the server sends all messages on the list that appeared after that id. This could be either by generating a transaction on the fly from a database, or just concatenating all messages id's from the one that was last received. Add compression (which email doesn't provide) and you have now provided a benefit--less bandwidth required compared to bulk email.

And the mailer KNOWS that it's not receiving SPAM, since the user has actually OPTED IN by definition by giving the email program the name of the mailing list that the user is interested in.

(Some mailing lists do a lot of this processing already, by incorporating messages in a digest format that is emailed or posted to the web, so we're not asking for too much additional functionality from them.)

Now, an ISP can intercept ALL DATA going across port 25, and examine the envelope. If there is more than a few recipients ("few" determined by the ISP's AUP), the ISP can generate a bounce ("Too many recipients") and drop the mail, or even save the mail, and in the bounce, give a URL that points the user to a "Click Through" agreement to pay the surcharge for sending email to multiple users.

For Spammers that try to circumvent this by sending one message per recipient, the ISP, which now has the equivalent of a "taxi meter" on port 25, can detect this as well, simply by aggregating the number of emails that a single address is sending out (simple database application).

Spammers that use open relays will get hit by the same problem, again assuming that the open relay's ISP has filtered port 25.

This should cut down spam.

The person whose wife wants to know about sales, simply OPTS IN to a server that will feed her the latest prices from the local grocery or the mall down the road. Advertisers are happy (they know how many people are downloading their bulk stuff), and the recipient is happy. If the mailing list isn't what the wife wanted, tell the email program to no longer download the stuff.

The only problem I see with such a solution is the possible invasion of privacy that could happen if it wasn't just the envelope of the message that is inspected (think: Carnivore). I'd leave this up to the implementing parties to come up with safeguards against this.

Any opinions?

RTBL - been on there, done that.

[mbourgon]

Actually, a couple of years ago, I got email from a sales rep's home account. The person who emailed the rep couldn't email their the rep's company account. Apparently we had managed to make it onto either RBL or ORBS (I don't remember). After having a laughing fit (so apropos... at the time one of our marketing guys was trying to figure out the best way to spam our potential customers), I did some research. Sure enough, open relay. Nice to see the better-than-thou corporate IT doing such a good job. Anyhow, I got ahold of someone, made them fix it. Re-submitted it to the RBL, and two days later it worked.

Oddly enough, our people weren't willing to fix it until I told them that the vast majority of our clients were probably subscribed (via their schools), and so our customers couldn't reach our sales slugs.

So, NBD to get off of one, at least over here.

What about spammers with their own mail server?

[BabyP]

I've seen a lot of ideas thrown out in this discussion (whitelists, blacklists, sender pays, etc) but I have yet to see one which would solve our current spam problem.

You see, a few days ago, someone started sending out porno spam with randomized return addresses from our company's domain. We first found out about this when we started reciving angry email from recipients of the spam, then looked in the mail log and saw an assload of bounces hitting our mailserver.

Well, we got someone to finally send us one of the spam messages with headers and saw that they were originating from an IP which was registered to a provider in Beijing (but announcing that it was the mailserver for [mycompany].com). We fired off an email to the anti-spam address that they had posted. We never got a reply from the Chinese provider, but we did notice the number of bounces in the mail log trickling down after a day or so. We were getting up to 5 per second, now we were getting 1 every 20 minutes.

But they were still sending. We got a few more headers forwarded to us (thank goodness some people research a little before they fire off an angry letter and delete the spam permanently). the emails were almost exactly the same, only the originating IP had changed to a provider in Rio de Janeiro. We sent an email to their abuse address, but the next headers we recieved pointed to a Swedish ISP...

And this is still going on. We'd love to sue these guys...what they are doing is clearly illegal. Hell, I'd love to kick their asses, personally. But it seems all we can do is complain to an isp that they are using and hopefully cut them off for a while before they continue with a new IP. We can't get any info from the ISPs they are using...they won't even respond to let us know they care, much less give out contact info for the offenders.

I really don't see any way of bringing this to a stop, once it starts. Do we have to change our company's name just because of some anonymous assholes?

Re:Sender pays is a bad idea

[davidj0228]

telemarketing has taken a hit with "no call" lists. now (in the us at least) if you dont want to get called you wont, but spam isnt as easy to regulate as telemarketing. snail mail advertisements are common, but my mailbox isn't stuffed with junk mail like my email box is everyday. spam is a much more serious problem

hashcash

[billstewart]

So whitelist mail from IP addresses you trust, and do hashcash on unknowns, and do mild teergrubes on suspected spammers (don't even need to slow them down to zero, just pretty slow), and set your DNS to give open relays the email addresses of other open relays. Most legitimate mailing lists don't have more than a few thousand subscribers, many more like a few hundred, so a few seconds of hashcash won't kill them - but it's a lot more annoying to a spammer who's trying to send 90 million messages, while the hashcash makes this a slow enough process that he's bothered a lot fewer people before his ISP gets enough complaints and blows him away.