Ask ISP Owner Barry Shein About the Spam Wars 610
Spam sucks. But it's worse for ISPs than for the rest of us, because they get bounces and complaints and other behind-the-scenes spam-caused messes the rest of us don't see. AOL talks of spam as "public enemy number one." Barry Shein, who started (and still runs) the world's first full-service dialup ISP, likens spammers to organized criminals, and calls spam "an organized, vicious, sociopathic thing" in this article, which spurred an interesting Slashdot discussion. So what should we do about spam? Ask Barry. One question per post, please. We'll post his answers to 10 of the highest-moderated questions sometime in the next week or so.
What's your e-mail address? (Score:4, Funny)
Collateral Damage (Score:5, Interesting)
Re:Collateral Damage (Score:3, Insightful)
If an ISP is willing to sell bandwidth to a known spammer and ignore complaints for months on end, then a network owner such as myself is perfectly free to regard that ISP as rogue and block all traffic from that ISP's network.
If that inconveniences other customers of that ISP, then either (a) they convince their ISP to change their ways or (b) they find another ISP.
This is exactly what SPEWS does, and it's remarkably effective. The analogy is much the same as having a crack house open in your neighbourhood. You either take action on the crack dealers or move out...
has destroyed the usefulness of email though (Score:5, Insightful)
Re:has destroyed the usefulness of email though (Score:4, Insightful)
scripsit Trepidity:
There's another, more insidious effect. I have caught myself almost deleting important, legitimate e-mails because subject lines looked ``spammy'' on first glance. Something like 80% of the e-mail in my inbox is spam, so I delete more than I read. Eventually, something important is going to get deleted instead of read; heck, it may have happened already and I just don't know it yet.
Consequently, I never assume e-mail to be totally reliable.
Re:has destroyed the usefulness of email though (Score:3, Insightful)
Sprint knowingly null routes spam complaints, and the various services that re-sell bandwidth from them don't even give you a bot reply. If we broadended the black list to every single sprint network subcriber (including susidiaries) immediately it may solve the problem domestically. Fact of the matter is sprint's poor management and greed -- spammers pay lots of money for their connections and typically in the past some isp's have played the 'we don't like spammers' lip service game, while raking in the cash.
Destroys the usefulness of email? That's a little melodramatic. Means as consumers we have to choose wisely perhaps, but caveat emptor is no different with computers than it is with any consumer good. Worse than spam? Never not in a million years. Wasting wads of your bandwidth getting joe jobbed is far far worse than losing one message from a contact on said network.
If it's that important, then it's foolish to trust it to anything but a courier, with a delivery receipt. There's never been any guarantees with email delivery, nor should there be. Blacklisting hasn't affected that basic design decision made long ago when email was first envisioned.
I'd say you're being paranoid, email works just fine null routing or not. If someone I need to correspond with is on a spam infested network, there are alternatives.
In fact, I lose no important traffic, just maybe the odd useless email from spam infested domains. Or a mass forwarded joke, but who cares? I'm better off without that.
SPEWS is a BAD operation. (Score:3, Insightful)
This is preached on email abuse newsgroups as gospel but I have yet to see anything other than anecdotal proof. What I do see are a lot of innocent ISP customers whose business is being interruped, not by spammers, but by SPEWS' vigilante blocking policies.
The analogy is much the same as having a crack house open in your neighbourhood. You either take action on the crack dealers or move out...
My $Deity, where to begin...
To correct your analogy the spammer is the crack house operator. What SPEWS does is start blowing up all the houses in the neighbourhood that surround the crack house in the hopes that the neighbours will complain to the authorities (The ISP)to take action.
What this farcical pretext misses is that spammers can move from ISP to ISP daily and as soon as you shut down one account they have opened a new one either on the same or a different ISP. The number of spammers and their mobility precludes an ISP permanantly blocking a spammer and thus the chances of getting off SPEWS once an ISP are on are minimal.
SPEWS has no posted policies as to what the timeframe is between an ISP complying with their blackmail blocking and the removal from the SPEWS list. 24 hours?, 2 weeks? who knows, SPEWS doesn't tell you. How often do they check? What criteria is applied during a check? Why don't they block the large ISPs like AT&T? Why don't they announce listings/delistings anymore? Why is there no direct method for applying for delisting? Why are postings from innocent ISP customers asking for reasons for listing met with scorn and accusations that sound make the customer is a nazi sympathizer?
There are far too many questions about SPEWs' practices.
Re:SPEWS is a BAD operation. (Score:3, Interesting)
This compares maybe something more to the tune of people going around door to door, asking for money. It's not illegal, but it can be annoying, but it's not that bad as I only see maybe 1 person a month. But if you apply this to spam, the cost for "going to door to door" is really cheap, so you can get hundreds of "visits" a day. So how do you stop them? You can't arrest them, it's not illegal (in most states). If you can think of a better way to convince "spam friendly" ISP to not allow spammers, I'm all ears.
This kind of blocking has been done in the past (but with warnings first), and has been met with similar outlash. usenet udp [stopspam.org]. I'm up in the air about the issue. I hate spam friendly ISP's with a passsion, but on the other hand, if there was only one high-speed ISP in town and they were spam friendly, then I'd be screwed.
SOMETHING needs to be done, no doubt about it. Spam Assassin works to an extent, but it's more of a hack, and doesn't actually directly address the problem at it's source, where it needs to be addressed.
Re:Collateral Damage (Score:5, Interesting)
Anyway the server looks at the from line in the header which now has a IP-key pair to see if it is valid. The server appends it's own daily key saying that it has checked the IP for validity.
On the recieving side, the server looks to see if the sending mail server is using this system, and does it's own filtering based on the IP addess (i.e. no 192.168.* or 172.28.* or other addresses reversed for special purposes)
Once this sytem becomes widely available, incoming servers can just ignore mail that does not conform to this system.
Ensuring IP address validity will be a big step in keeping spammers honest. If people could directly respond to spammers then we have sovled almost all SPAM abuse problems. Other valid SPAMs can be effectively filtered out on the client side. These steps will reduce SPAM effictiveness to a negligable level, while preserving valid emails from mailing lists and such.
Re:Collateral Damage (Score:3, Informative)
The IP addresses in email headers are valid, until you trace back past one that can't be trusted (it's malicious, or it's misconfigured, etc). Now, if this one can't be trusted to make sure it only forwards mail it's supposed to, why should it be trusted to correctly enforce any other new scheme?
It seems like what you actually want is some sort of end-to-end scheme where the sender and the recipient are sure of each other's identity. You can do this already, using software like PGP - the sender signs the message, and encrypts it using the recipients public key.
Bayesian Filtering (Score:3, Interesting)
I use Popfile at home. It seems like the perfect answer to spam. What's your take on Popfile and other Bayesian filtering methods?
Re:Bayesian Filtering (Score:5, Insightful)
By the time it gets to your inbox, it has already cost your ISP money (time/effort/bandwidth) to deliver it. You just see what leaks through your ISP's filters, despite their best efforts.
I ask for mod-love for the first time ever here. (Score:5, Insightful)
While in the short term I concur, in the long term I must cry au contraire.
If Baysean filtering makes its way to the general public -- or is introduced at an ISP level, then it will reduce the amount of spam that gets through to potential customers, and hence make each spamming less profitable.
The least profitable of the spam messages will dissapear, thereby reducing the loads on our mailboxes and on the ISP as a whole. Therefore, perhaps a better question is:
what's your opinion? (Score:4, Interesting)
Kill 'em all.... (Score:5, Funny)
Collateral damage (Score:3, Informative)
If I was the president of the company that makes Viagra I'd be nervous.
Nigerian spam killing in Prague (Score:5, Informative)
If I ran an ISP... (Score:2, Interesting)
As an ISP, you shouldn't have to be the front line of defense for some of the people who want to use your networks to deluge the email boxes of the world with their emails about penis growth, diets and discount shoes.
Craenor
Windows + L (Score:2)
Re:If I ran an ISP... (Score:2)
Re:If I ran an ISP... (Score:5, Insightful)
So if your best friend is infected with klez (or the latest variant) and sending messages that appear to be from you, if three people call to complain that you are sending them junk, you are outta there? Those are three complaints about the content of your email, and your policy says no matter the circumstances.
What if I don't like your political views that you've espoused on a political discussions mailing list and I call up your isp and tell them that your opinions about certain PICKWHATEVERPARTYYOUHATE Senators constitute a terrorist threat. After 3 of those complaints, you get dropped.
I wouldn't use an isp that didn't have some intelligence behind its decisions or didn't have an appeals process if I feel I was mistreated.
But that can be abused too (Score:3, Insightful)
Then it becomes the isps responsibility to investigate otherwise they could face legal libability for cutting off someone account wrongly.
Fine, I'll ask (Score:4, Funny)
Re:Fine, I'll ask (Score:5, Funny)
Actually, only one of the above is true
How to learn to ignore spam.? (Score:2)
Re:How to learn to ignore spam.? (Score:3, Funny)
Spamming as a crime (Score:4, Interesting)
What kind of changes would you make to the way email is handled to facilitate the elimination of spam?
SMTP (Score:5, Interesting)
What do you think should enhance/replace it?
Article suggestions (Score:5, Interesting)
Liberty and Security (Score:5, Interesting)
If spammers are like organized criminals, one might make an analogy to protecting ourselves from orgnized criminals (say, terrorists) and possibly having to sacrfice liberties. I don't really endorse this myself, but lots of our lawmakers talk about needing find the "balance" between liberty and security.
Specifically with regard to spam, every proposed solution I've seen bothers me, in that it will restrict what I do myself now (legitimately) with E-mail. I already feel like collatoral damage in the spam wars, in that I can't just use my computer as a SMTP host to send mail (which is the Unix way to do it); rather, I had to configure my SMTP server to forward through an ISP SMTP gateway, as my cable modem is on a blacklisted range of IP addresses and some of my mail wasn't getting delivered. Well, OK, I was able to work around it, but it is already unfortunate that we can't just use whatever computer to send E-mail. More rigorous schemes will only make this worse-- for instance, requiring that the From: line match the SMTP host id where the message originates. Alternatively, a "per message" charge for E-mail, which yes would hamper spammers, will also hamper people who just happen to use E-mail a lot-- and,I guarantee you the spammers have easier ways of getting around that than the legimiate E-mailers do.
Is there any way we can cut back on spam without hampering legitimate use of E-mail even more than it already has been hampered in the battle against spam? Or are we just going to have to accept that we won't have the freedom to fully use E-mail the way it was intended in order to stop spam?
-Rob
Re:Liberty and Security (Score:3, Interesting)
I know that many have been trying to come up with a viable solution to stop "unwanted" spam...
What about a whitelist that would would use a combination of a registered email addresses list in combination with a reverse dns lookup? In other words, make it a law that if you are a business, or send emails to anyone for monetary purposes, you must register with a particular DB (so that we know who they are) and if you send unwanted spam, you can automatically be fined. Also, I would think that you would want to force businesses to use a double opt in type of format. If you want to use someone else's list, THEY mus send the email for you (so that the check with the DB and reverse DNS lookup still works).
If you are a personal user, perhaps there could be a separate list, if only to declare that your address is not a spammer address.
I know there are plently of holes in my idea. Also, Methinks we must have outgrown SMTP and maybe the spamming problem can only be fixed once we move to a mailing protocol that takes spam into account.
coordinated attacks (Score:2)
Laws (Score:5, Interesting)
Do you have any thoughts on these laws? I know that, as a non-lawyer, you probably can't do much for the actual wording, but what content would you have if it were totally up to you?
What would be the minimum actual cost? (Score:5, Interesting)
What would be your actual dollar cost of spam, if you didn't spend much time and effort fighting it?
Let me explain...
I sometimes hear that spam has significant costs in bandwidth and storage but I don't believe it. As far as I can tell, SMTP traffic is at most 2-5% of net traffic. And a quick calculation shows that an ISP's costs for storing its users' spam are fractions of pennies on the dollar. (*)
You've likened spam to a DDoS attack on your mail servers. Stories about being flooded with traffic sound impressive but computers are so fast now, it's hard to put anecdotes into context. So I'm looking for dollar amounts. For a customers paying b dollars per unit time, an ISP like yours has to spend c dollars per unit time on servers that can handle those customers' incoming SMTP traffic. If this is significant, I'm looking for c over a times b :)
Obviously admins to run the servers are an important cost. But for purposes of this question, suppose you wanted to do the bare minimum. Say you set up the SMTP servers to use just a few of the less-intrusive DNSBL lists, like sbl.spamhaus, relays.ordb, or list.dsbl, and then ignored them as much as possible.
The next most common argument I hear is that customers will abandon ISPs that don't fight spam. But every ISP has the same problem, so this is really a competitive advantage issue except for the small percentage of users who are actually driven off the internet by spam.
Then there's outgoing spam but I don't imagine that's too hard to recognize and stop quickly.
Let me know what I'm missing...
(*) Thumbnail calculations of spam storage follow. Let's say J. Average ISP Customer gets 20 spams a day at 10K each, and deletes them only every 30 days. That's an average of 20*10K*15 = 3 MB of storage. If the ISP replaces hard drives every two years on average and its total storage costs are ten times the actual medium costs (for labor, backup, redundancy, downtime), then at today's hard drive prices, that spam storage will cost the ISP 0.003 * 10 / 2 dollars, or about a penny and a half. Over that same year, J. Customer pays the ISP $100+.
Re:What would be the minimum actual cost? (Score:5, Informative)
For a start, ISP's get hit every day with repeated dictionary attacks where a spammer tries thousands of common usernames for each domain the ISP hosts. The sending hosts (usually a number of raped proxies) pipelines the SMTP sessions and doesn't wait for a response. Every single one of those emails chews up CPU, memory and disk space. It's a non-stop attack on your mail server queues.
When they get a miss, sendmail bounces the email to the postmaster and tries to deliver a bounce message to the forged FROM address, so your queues and disk fill up for days with this crap.
When they get a hit, it's even more disk space chewed up until the user downloads them. Some spammers are embedding HTML and graphics in their spam as well, so they are getting larger and larger.
I don't know where you saw 2-5% spam content. Most ISP's are seeing ten times that, unless they employ agressive filters which may be ideal for people who run their own domains but can be problematic for ISP's.
Re:What would be the minimum actual cost? (Score:5, Insightful)
The 2-5% he guesstimated was total usage of bandwidth by SMTP. I say guesstimate because I've searched for bandwidth usages by protocol and haven't been able to find (recent) data. Unless we can have reasonably accurate numbers from backbone segments it's going to be difficult to estimate just how much Spam really does cost.
I mean, if the OP is correct and SMTP only chews up 2-5% of the backbone, then it's not nearly as big of a problem as if it's chewing up 20% or more.
Even so, if SMTP only takes up 5% of the bandwidth and 80% of that usage is Spam, consider just how much cost savings could be realized from dropping SMTP from 5% to 1%.
Look at it in another way (Score:3, Insightful)
If the average genuine mail to spam ratio on your system is 1/10 (ie: for each genuine message, you get 9 spam messages) this will have the inevitable effect that your infrastructure has to be capable of processing a load which is 10 times higher than would be required if there was no such thing as spam.
Given that 1/10 is probably a very conservative estimate (escpecially for big ISPs with a lot of J. Average Customers), you can imagine that this can have a huge impact on the systems required to handle this.
Also when a spammer is using a fake (or real) address at the ISP as a return address, a lot of bounces get directed there in very short period of time (which in fact is very much like a DDoS).
While silicon speed is still increasing at a mindnumbimgly speed, disk platters haven't. It's not costly to get a lot of storage (73GB disks are 'affordable'), but it can cost a lot to build a storage subsystem that can cope with the load and is relatively solid (raid / backup).
On top of that there are the hidden costs, eg: customer support for dealing with customer issues related to spam, system administrator time spent extra on dealing with spam-related problems.
I don't think it's so simple as to stating that "bandwidth is cheap" (which simply isn't true for a very big part of the world) and "storage is cheap" so spam can not cost much.
Re:What would be the minimum actual cost? (Score:3, Interesting)
Incoming (From the internet into our network) SMTP traffic is closer to 30% of all bandwidth used.
The next largest chunk is web traffic (between customers and the internet) which is about 50%
Another 10% is POP3 to customers and the internet (Only the latter being really noticable)
Then another 10% or so of other things like ssh/telnet, games (well, random high ports, im just assuming) and the like.
I setup SpamAssassin in a global way for the customers and run stats on the data captured.
For around 3000 email accounts or so (I am rounding up) spam is held in a quarenteen for 5 days and then deleted. (This is so users can go to a web control panel and deliver mail that was flagged as spam incorrectly and add it to a safe-list)
The 5 day queue stats are
Total size of SPAM spools : 1.4G
Total number of SPAMs : 154502
This compared to (for reference)
Total size of quarantine spools : 33M
Total number of e-mails with viruses : 237
in the same time period.
This added to the fact customers STILL get spam, still complain, still threaten to leave to go to our competition which still gets as much spam as we do, and it really turns out that any amount of money we spent to fight spam is a loss. We get nothing for it other than knowing a few of our customers are slightly less pissed off than without our efforts.
On top of that we have customers that complain when an ad that they signed up comes in and they have to *gasp* go to a little effort to safe-list the emails they want. From them we get "How dare you!" and they still threaten to leave.
Damed if you do, damned if you dont, and everyone blames the ISP.
This is always a finantual loss (everything costs money, it doesnt make us a dime), and on top of that paying staff to deal with it costs.
And none of this takes into account spam complaints from customers that we have to look into and deal with, which in reality Does cost us money (losing a paying customer) and all it gains us (finantually speaking) is the privlige of not being blacklisted.
While I cant say I like the fact there are some ISPs out there that totally ignore spam complaints, I fully understand why.
RBL's (Score:5, Interesting)
One of the few measures that can be taken against spam is the use of blacklists (for instance via DNS). There are a lot of pro's and con's for the use of DNSBL's. How do you feel about these? Should DNSBL's be governmentally regulated? Do you use any DNSBL? Should an ISP enforce certain RBL's (let say, of open relay's) on its customers?
Technological versus lawful. (Score:3, Interesting)
Bayesian filtering (Score:3, Interesting)
Spammer Crackers (Score:5, Interesting)
ISP Tools (Score:5, Interesting)
at Sendmail and don't see anything that would allow you to throttle mail volume, check outbound messages for SPAM, restrict new customers etc. There isn't even anything built in that would warn you about a customer sending a million messages. It would seem that a few tools like that would be a big help to an ISP too small to develope its own.
Re:ISP Tools (Score:3, Informative)
We were using postfix ( http://www.postfix.org ) with PCRE support.
Incoming mail was first passed through a few DNSBLs (ordb, wirehub.nl and relays.osirusoft.com)
We had all mails checked through a simple regex body check, which looked for a few keywords like US1918 and phrases like "To unsubscribe please click". These were bounced (at that time Postfix didn't have the lovely discard keyword).
We used regular expressions from the postfix site to additionally capture spam.
We blocked outbound port 25/tcp to force dialup and cable users to relay through our servers (so that we had logs).We had no limits on how many iusers you sent mail to, but if you were complained about with proof as headers, then the logs got checked and the user terminated.
Also, you could use any identity you liked (we didn't have the from user@our.domain limitation).
Additionally, I had log parsers which watched for connections coming in from ips and notifying me when they would go above limits within certain time periods. This was mostly useful in catching virus infected machines.
Today, I would run amavisd-new with clamav (http://clamav.elektrapro.com) for this.
I had about 5-6 false positives with this for about 9000 users with about 20K mails/day.
Mail to postmaster@our.domain, abuse@our.domain was exempt from all checks.
Using blacklists saved about 5% of our bandwidth, which in USD would have been about 50000 USD.
It should be easy enough to write a Perl script to read your logs and throttle customers down to a maximum rate.
Is there a resonable solution? (Score:5, Interesting)
I certainly am tired of deleting the penis elargement and Nigerian bank deposit e-mails, but where is the balance and how do we attain it, if ever?
ISP's duty? (Score:2)
ISP's are in the best position to pursue spammers and demonstrate to courts the financial burden of dealing with spam.
With very few exceptions, we don't hear about ISP's taking spammers to court. What's up with that?
Permission Based Solutions (Score:5, Interesting)
I am currently using a permission based solution to block spam, called Choicemail [digiportal.com]. It works great since I know that there are no filters trying to guess what is spam and what is not. People on my white list get in, people who aren't get sent a message asking them to identify themselves.
The only drawback is that some people may possibly feel slighted that they are forced to go through such a process. But so far no one has complained. In fact, most people seem to be intrigued by the concept. If this type of spam blocking catches on, people will begin to expect it. Sort of like having to knock on someone's door before entering their house. It is a custom so pervasive, we feel strange just walking into someone's home, even a friends, without first knocking.
Sorry for the length of this post, and now to the question: How do you feel about this type of spam blocking?
Re:Permission Based Solutions (Score:3, Insightful)
For instance, when you buy something online most companies will send you a confirmation email. If I haven't bought from that store before I have absolutely no idea what addrss that's going to come from, and thus have no way to whitelist it. And it's impossible for the automailer to respond and whitelist itself, since any method that's auto-parseable will simply be co-opted by spammers.
Sure, you can have an alternate mailbox for this kind of mail that isn't behind a whitelist, but it doesn't really solve the problem then.
Re:Whitelists - just say no (Score:3, Insightful)
And a good whitelist will pay attention to outgoing mail, as well, and authorize replies.
The people need to know! (Score:2, Funny)
(Disclaimer:of course, this is said firmly tongue in cheek, I don't approve or condone physical violence against spammers, etc. etc. yadda yadda yadda)... =)
when spam doesn't work (Score:2)
why not whitelist? (Score:5, Interesting)
Re:why not whitelist? (Score:3, Informative)
Re:why not whitelist? (Score:3, Informative)
Except that it won't work if I whitelist my mail. I'd have to add the site to my whitelist before they send me anything, and I don't know where the mail will be coming from. Since it's an automated system, a response from a whitelist system won't be seen by a real human.
Blacklisting SMTP servers? (Score:4, Interesting)
Or alternatively blocking whole ip-ranges of ISPs which deny to cooperate on this issue?
Can tech solve this? (Score:5, Interesting)
Re:Can tech solve this? (Score:3, Insightful)
Tech solution followup: Do you think that recasting the email system would help? A micro-payment tariff per-email sent is suggested every now and then here. Could that work given that if it isn't uniformly adopted around the world it may not help that much?
How about law based solutions? Are the efforts of (West coast state - CA I think) to combat spam as unsolicited email destined to failure, or might that be the right approach? Can local (eg statewide) efforts work when dealing with the international operation which is mass-emailing?
Finally, how about the community based approaches? By this I mean efforts that emphasis the stigma of spamming or facilitating spamming, for example the black-listing groups who publish ISP's that allow mailing relays or direct spamming through them. It sounds like your ISP uses blacklists, is blacklisting an effective solution, or does it entail too high a false-positive rate?
More interestingly perhaps, does it knock out enough spam to be considered effective? Does simple blacklisting stop more than 50% of incoming spam? Are there really a small hand full of channels through which most of the spam is routed? I find the approach appealing because it allows a relatively fast punishment to those who propagate the problem. In a sense it's a bit like focusing on the drug-dealer not the drug-user. On the other hand it is a fast response system, which is highly open to abuse, in a sense it's a form of vigilante-ism. It also raises the question of what a service would have to do to get themselves removed from the black-lists. Speaking as someone who runs an ISP, what do you think of the black-list approach?
national "do not email" list??? (Score:4, Interesting)
All spammers would have to (by law) query the "national do-no-email" database before sending out their crap.
I'm just wondering if something like that would be an effective way to cut down on the noise out there?
Spam Lawsuits (Score:5, Interesting)
Spam and whitelists (Score:4, Interesting)
Acting Locally, Effecting Globally (Score:5, Interesting)
Given today's society, technology and infrastructure, what can an individual do that would be effective in reducing not only the personal strain of spam, but also lessen an ISP's burden.
What kind of strategies have you seen work. For instance, in particularly bad instances I'm prone to send an e-mail to spam@isp.net, abuse@isp.net, or admin@isp.net, but usually never even get a response. Is there a better thing to do? Are there things that are absolutely the wrong thing to do (such as replying to a spam)?
In short, what would you like to see users do in response to spam today?
What legal pursuits? (Score:5, Interesting)
MTA Identification? (Score:4, Interesting)
Sure it'd be a short term hit on the number of hosts you could exchange mail from, but eventually I think anyone who wanted to talk to anyone would have to get on.
isn't email filtering dangerous? (Score:5, Interesting)
And can you selectively give up common carrier status? If you block some email but host anyone's web page, for instance, can you be sued successfully for objectionable content on those web pages?
Worst Practise (Score:5, Interesting)
Reply-to impersonation?
Embedded hypertext identifiers?
I'm sure it's much worse than that.
What would you do to stop that evilest of evil practises?
SPAM as a weapon? (Score:5, Interesting)
Barry, are you saying that SPAM might be being used as a WEAPON against smaller ISPs that can't afford the manpower to try and limit it?
This is a conspiracy theory which would certainly explain alot about smaller mail servers getting 200 pieces of SPAM per user per day (as some accounts I am familiar with do) while accounts with some of the "big boys" seem to get amazingly FEW SPAM.
some quick stats (Score:3, Informative)
Total Messages: 155,610
Bytes: 1,198,581,670
Messages that got looked at: 91,140
Messages that got blocked: 76,657
% of bytes blocked: 53.0%
so 91k of 155k messages got filtered- the rest are people who turned it off cause they dont trust the system. It's perhaps safe to presume that the percentage is roughly the same if the rest got filtered, too. 76% of our email is spam! We've had our domain since 1994. Spam is defined as anything blocked by the postini filters, which each user can control a threshold on, so the definition of spam is subjective.
real problem (Score:3, Informative)
do you think there is anything that can combat this?
simply telling people that nothing will add three inches may not work as well as we hope for... stupid people are gullible.
Back to the 90s (Score:5, Interesting)
For instance, what changes would you have advocated in the mail protocols and what standard procedures would you have told other ISPs to use to prevent spammers from getting a foothold in the first place?
Bandwidth consumed? (Score:5, Interesting)
To Bounce or Not To Bounce? (Score:4, Interesting)
I used Mailwasher for a while, which gives users the options of generating bounce messages while filtering. There is some personal gratification in making it look like my email address doesn't exist. But does it actually help, or does it just add to the ISP's bandwidth requirements?
I can't publish my email address anymore (Score:3, Interesting)
I don't give out my email address to anyone I don't know well, and I change it every year. I tell people who need to get in touch with me to call.
All this is because I started getting 50 spams a day. Right now, it's impossible to post to a newsgroup, put an email address on a web page, or have an email address that's listed in any sort of a directory without getting tons of spam each day.
I agree with that article that email is a failure. Important/busy people just don't have time for it.
A friend of mine finished looking for a new full-time job. He sent out some resumes by email to the listed addresses, and some by Fed-EX. Only the Fed-EX ones got answers. Companies get so much spam that they miss good resumes coming to them!
Internet Mail 2000 (Score:5, Interesting)
DJB claims that with this system bounce messages will be eliminated (if I read correctly).
false negatives vs. false positives (Score:3, Interesting)
Ie. are you attempting to stop all spam, with the possibility of false positives an acceptable risk, or is there some sort of calculation that your organization uses to balance the false positives (mail rejected as spam that wasn't) against the false negatives (mail that was accepted, but was spam)
Best software solution? (Score:3, Interesting)
cleetus
Should a new email protocol be created? (Score:4, Interesting)
Specifically, if we created a second protocol that required that all email be digitally signed by the person listed in the "from:" clause and that the originating ISP guarantees this identity, wouldn't that solve most of the problems? The true identity of people who use the bandwidth I pay for to communicate with me seems like a fair thing for me to be able to insist on. I might even be willing to pay a little more to have such a system, although I would think such a system would be cheaper for my ISP, since the cost of carrying 33% garbage isn't there.
I should be able to say I want to filter email from Alan M. Ralsky of West Bloomfield, Mich or from any that passed through any ISP that cannot guarantee me that I can determine this. The problem is that Mr. Ralsky can send me email and I have no hope of identifying that it came from him. All that is required, it seems to me is for the leading ISP's to get together and create and enforce a standard that says your new-style email will be digitally signed with your legal name and that only ISP's that comply with enforcement practices will be allowed to use the new email protocol.
Re:Should a new email protocol be created? (Score:3, Insightful)
Does that mean I can't send e-mail without my real name attached? What if I prefer to maintain some level of anonymity in my online communications? Sure, my ISP can know who I am, but I should be able to send someone mail that doesn't have my real name on it, to someone whose real name I don't know.
I think it's also important for children - someday I'll probably have kids, and I certainly plan to teach them about basic safety rules, which includes not giving out your last name or address to anyone online, including by sending them e-mail with your name on it. Goes along with not taking candy from strangers.
Claimed Opt-In Spam Lists (Score:3, Interesting)
My question is, do you see this as an effective method? Do spammers really quit after seeing their packets are being dropped? Why do they not?
Is a legal solution possible? (Score:3, Interesting)
Where do you draw the line? (Score:4, Interesting)
Most of the emails we sent out we're from internal, registered customers of the company. I would call these 'opt-in' emarketing messages that ranged from pitches to buy new or upgrade products, customer satisfaction surveys and automated replies for visiting a website and signing up.
There were, on the other hand, spammers. That is the only way to describe the quality of the emails they sent out. When I could query their databases and find email addresses of 'abuse@someisp.com' and other, similar non-customer addresses, there is no other way to classify it.
In either case, we never tried to hide or run away. We always used real email addresses and kept the same domain names. So, my challenges were, "How to I keep the 'good' customers from impacting the 'bad' customers?" I dealt a lot with CAUSE, the MAPS RBL and other organizations to keep the emails flowing.
So, here is my question: How do you, at the ISP level, differentiate between legitimate email marketing and Spam?
What is current SOHO SMTP server "best parctice"? (Score:3, Interesting)
What are the current "best practices" and state-of-the-art for the little guy (enterprise, small office/home office, little ISP, etc.) who:
- has some need or desire to directly serve inbound and outbound SMTP and
- has SOME time to sysadmin, but
- does not have the resources to throw several full-time-plus-pager sysadmins into the spam wars?
Recommendations for the small guys? (Score:4, Insightful)
What are your technical recommendations for us, to make your life easier?
For instance, I usually argue to require valid FQDNs in the HELO and MAIL FROM command, and reject anything claiming to come from myself or one of the RFC1918 reserved IP addresses. This is entirely content-neutral - I just see no point in accepting any message from somebody who can't be contacted in turn if there's a problem delivering the message.
But I generally don't bother with RBLs, and am philosophically opposed to IP redlining since it could easily lead to a world where a few corporations act as gatekeepers.
I know what impact this has on my sites, but does this cause problems for the large sites? Or does it help you as well?
Misguided efforts (Score:5, Interesting)
- Boucing messages with Mailwasher
- Having munged addresses where the "NOSPAM" is in the user part rather than in the domain part (that is, "bozoNOSPAM@isp.net" instead of "bozo@NOSPAMisp.net"), so your servers get hammered with invalid harvested addresses.
- Using often broken tools such as SPAMCOP to LART other ISPs?
- Does a significant number of problems from your user always come from the same users, or is the problem widespread?
are having a negative effect towards your own efforts at fighting spam, either by diverting ressources or simply being a nuisance?How much of the SPAM complaints do you do receive are properly done (that is, with headers and sent to the proper ISPs)???
Quota systems... (Score:3, Interesting)
My technical proposal: people/companies purchase SMTP message-sends the way they purchase cell-phone-minutes:
- spammers who use open relays would saturate that relay's quota, and most of the spam thus
relayed would fail to go out, thus the owner of the
relay would have incentive to fix it, so they
can send their own mail.
- spammers who send directly from ISP accounts would have to purchase large numbers of them in order to send a given volume of mail.
To enforce such a system, you would need to build a smart firewall that knew just enough SMTP protocol to read the RCPT To: lines, and count recipients. When a given sending host exceeds its counter for the week, poof! the firewall blocks further SMTP activity (or even all activity) from that host until someone clears it.Backbones could limit individual ISP's with such a system, and ISP's could in turn limit individual customers; indeed they would basically have to, so that one customer can't ruin their SMTP quota. If the ISP doesn't enforce such a rule, their backbone tap enforces it for them.
If such infrastructure became widespread, the only way a spammer could send large numbers of messages would be to get large numbers of ISP accounts, which would hopefully cost them enough money to make it not worth their while anymore.
How about a "no filter day"? (Score:5, Interesting)
Regarding your MIT Spam Conference appearance (Score:3, Interesting)
I am a security technician and sysadmin for a research institution. My clients, who are scientists, are not interested in being paid to watch advertisements, or in having our institution funded by advertisements shown to them in email. We don't want to be paid to receive spam; we just want not to receive it. We just want the spam attack, the theft of our resources and our people's time, to stop. Do you see any way this can be reconciled?
Conflicts of interest... (Score:3, Interesting)
Q: If ISPs are really all that upset about spam, why haven't they done anything about it?
It's patently obvious that ISPs could eliminate spam simply by blacklisting individuals who engage in the practice (and other ISPs who don't follow it). This is how credit ratings work, an area in which there is both a greater monetary incentive for misbehaviour and much lower (technical) barrier to entry.
Properly implemented, such an individual blacklist would eliminate most worldwide spam - since only a couple dozen individuals are responsible for more than 90% of the phenonema.
It seems to me that the real reason ISPs don't stop spam is due to base economics: spam houses pay money. So spam elmination has become a classic games theory problem - money you spend to search for spammers on your own network is wasted; you just have to respond enough to keep off the RTBL.
And because detection is always someone else's problem, spammers will continue to thrive in the time it takes to process the request.
Spam, Viruses, and Filtering (Score:5, Interesting)
How would you grade the effectiveness of current filter techniques, and blacklists etc.
What filters/blacklists do you use, and how could they evolve so that you would feel comfortable using them? When choosing blacklists or filters, how do you measure the gains of blocking x% of spam against not-blocking y% of legitimate emails.
How do you regard the threat of spam in opposition to some of the major viruses. That is, viruses like "sapphire" that generate huge disabling traffic netwide, or like "code red" that - to this day - is still making attempts to access "cmd.exe" on my own linux box.
And lastly, as we all want to know, what do you think can be done to spammers to strongly discourage them from continueing their immoral practices.
Whose responsibility is false positives? (Score:5, Interesting)
As a World customer, I found last year that I was getting removed from several mailing lists I was subscribed to beause so much of their traffic was being bounced by World spam filters.
When I contacted customer support, they said that the messages must have contained strings that triggered the filters, and that the solution was for the lists to avoid using those strings in the future.
What strings would these be? Customer Support couldn't say.
So, if I wanted to use my World account to recieve my list mail, I would have to persuade all other list members to not use the filter-triggering words. And I would have to do this without telling them what those words were.
It seems to me that strong filtering of customer inboxes is one thing, but doing so with no provision for opt-out or whitelists interferes with the individual's right to get the internet servide he's paying for. Do you disagree?
HashCash? (Score:3, Interesting)
I see much more promise in technologies like HashCash [cypherspace.org] which force sending machines to burn CPU cycles in order to send their message. My question to you is, are you aware of this type of technology? Do you think it would be effective? And what do you think it would take to get such a technology deployed (standardization, ISP acceptance, MTA/MUA integration, etc)?
What if it were outlawed? (Score:3, Interesting)
Are spammers too hard and too numerous to track down to be worth it (and too poor to pay the fine even if caught)? Would spammers just move offshore and continue to spam?
550: 5.7.2 No mail for you (Score:3, Interesting)
Using the spammer's last SMTP protocol leg, before your mail server closes it, why not do the following:
By not letting go of the (would-be spammer's) SMTP connection, one can consult the mail recipient white list. From an unknown sender, instead, save the entire email in a holding queue and send back the following SMTP error message:
With a marriage of sendmail [sendmail.org] MILTER [milter.org] and Tagged Message Delivery Agent [sourceforge.net], one can shift the burden of automating the mail recipient white list back to the sender (like ICQ does).
With a tweak of the last leg of SMTP protocol, we, the email users, will have control over what is 200 and what is 5-f@cking-50.
What say you?
- Shamelessly ripped from the Seinfield TV episode "Soup Nazi."
Can you put a number on it? (Score:3)
DNS Blacklists (Score:3)
I've been an active anti-spammer for quite a while now and am quite proud of the knowledge I've acquired in the fight against spam. I even make good money off of filtering spam for others. As an anti-spammer I'm sure you've encountered folks that simply don't understand the purpose for a DNS blacklist. They claim it's prone to false-positives, dated information, legality issues, informally administrated, submission information isn't verified, hard to get removed from a DNSBL, or just plain silly (I actually had a person tell me this once). Most of these people make such claims due to a bad experience they personally had with a DNS blacklist at some point. It might be that they didn't get a newsletter they'd signed up for, when it reality the sender might actually use spam as a marketing tool. It could also be that they no longer get yahoogroups.com mail, when in reality they harbor spammers and take no action on abuse complaints. It could also be that they themselves had a MTA listed, when in reality they were incompetent mail admins and their MTA was an open relay. The last one is the worst of all. Unfortunately a large number of the people that have said these things somehow manage to call themselves mail administrators.
As a mail admin, I'm sure you have a better understanding than most about how much spam can hurt a business and can see the usefulness in DNS blacklists. How do you make the case for DNS blacklists when faced with the misguided biasness from those that simply don't understand?
How do you define spam? (Score:4, Insightful)
There seems to be a lot of disagreement between spammers and their victims on what exactly is "spam". Lots of spammers claim that it's not spam as long as [it's not commercial | it's not porn | I bought an opt-in list | etc]. Some users don't mind diet pill ads but hate herbal viagra.
What do you consider spam? Is it unsolicited commercial email? Unsolicited bulk email? What about chain letters forwarded to you by your Aunt Ethel? Any successful legal solution will depend on a good definition.
Spammer spoofing with my email address (Score:3, Interesting)
I'm currently getting hundreds of bounced, undeliverable messages from various organizations because a spammer is using my email address to spam others. The web site he's advertising is located in China, and I seem to have no way of finding the individual much less taking action against him.
What are my options?
criminalize relay rape (Score:3, Insightful)
Upwards of 90% of the spam hitting our servers is relay raped off innocent 3rd parties. When you report the criminal trespass to law enforcement, they shrug their shoulders and say "there's no law against it" or "there's not enough fines to make it worth our time to prosecute".
Well, there should be.
What would you ask of legitimate opt-in marketers? (Score:3, Interesting)
The question:
Currently, a company that follows all of the "guidelines" and does everything right, still stands a good chance of getting listed on SPAMCOP and other RBL lists based on a handful of complaints from clueless customers.
BCDE.COM maintains an nation-wide network of high-volume web sites. Access to the most basic site features is free, but all value-added features require that the user register -- The registration page includes very clear notice that that the "cost" of registration, of access to advanced features, is that the user will receive marketing email from BCDE.COM.
If you choose to "unregister", BCDE.COM will stop sending you email, and you will no longer be able to access the advanced site features.
Filling out the form on the site is just step one -- based on the form, an email is sent to the email address supplied, re-iterating the terms on the form, and providing a URL to "confirm" opt-in. The URL includes a secure hash to prevent spoofed confirmations. Once an address has been sent a registration request, it cannot be sent another request for a week (to prevent using the form as a flood attack).
Daily, BCDE.COM and their ISP(s) receive complaints from users and from SPAMCOP about the confirmation email, about the marketing email, about the "spamvertised" sites hosted at A.BCDE.COM which are promoted in the marketing email.
99.999% of the user base has no problem with this business model, and would prefer this approach to actually paying a subsciption fee for access to the "value add" site features.
How can an ISP known that a sending site that their customers complain about, or a customer that other ISPs complain about, is a legitimate business that is following all the "rules"?
Re:What new tactics... (Score:3, Interesting)
SPAMMERs disregard the rules of SMTP fair play (falsified headers, for one), so we should have the tools to deal with these miscreants.
1) Allow users to reply to SPAM with "User unknown" message as if the administrator issued the message.
2) ISPs should allow users to report SPAM and falsified headers, which are then compared to the spooled email messages. E-mail issued from offending domains are rejected with a "Please Resubmit" message. This could be an Opt-in service to allow community policing for SPAM. Imagine the flood of Resubmit Messages back to offending (or falsified) domains. Even if the headers where hacked, the SPAMMERs would not reach their audience, and the postmasters would shrug off the "Please Resubmit" requests. Shouldn't swamp any email server.
3) ISPs should allow users to delete, ignore, and read email messages without informing the entire mailing list of your current status. AOL does this, and I can just imagine SPAMMERs elisting people to parse through email status - Who reads them, who deletes them, and who ignores it.
Re:He's severely over-reacting (Score:3, Insightful)
He's right, though -- it's not his job as an ISP to fix it on an individual basis. We need a change in the whole infrastructure.