- Jason

Link | Posted on Jul 09, 2009 12:52 PM PDT by Jason Livingood

Vhlo does not require cooperation of the whole world. Although it aims at recovering the reliability that mail had in the 90s, it only operates in transmissions between trusted parties.

There is no such thing as a trusted party unless you are willing to accept the consequences of misplaced trust. Trust me, I know.

What happens to your proposed system when the sending MTA is subverted and stops filtering? Of course, that would never happen.

Where Dave speaks of trust models I start from a position of distrust. Must be the anti-abuse background in me coming out. It really is quite simple. Emitting hosts which cause problems (why get hung up on the word "SPAM"?) will find remote MTAs unwilling to accept mail from them or throttling throughput. Dynamic ranges of IPs increasingly are blocked from the start.

Before throwing another acronym into the hopper I would prefer to see the impact of wider deployment (both sending and receiving) of DKIM/ADSP as well as receiver side respect of strong (-all) SPF assertions. I will grant that these approaches do not directly attack SPAM but do make it easier to sort the wheat from the chaff with regard to good actors from bad actors based on authentication.

Link | Posted on Jul 09, 2009 6:40 AM PDT by Michael Hammer

Had the specialty domains been established before most of their logical registrants and customers had well-promoted name identities somewhere else, the story might have been different.

The problems of those established organizations and their domain names is then compounded by search engines that rely on popularity of references.  While they retained their original domains, all references to them used the same links and hence got them priority.  If they transition some of their activities to the new TLD, they split the counts.  Search engine algorithms might be smart enough to compensate for that, or might not, but why take the risk?

That logic means that only one type of registrant goes primarily into the newly-added domains: those who haven't had much of an Internet presence before and who hope that the new TLD will become sufficiently established to reinforce their presence.  The establishment of the TLD "brand" itself depends on the major actors, and those are already established elsewhere: if they have any incentive to migrate, it is out of support for a community of which they perceive themselves to be a part—the very opposite of giving themselves competitive advantage.

The net result of this is that I agree (this time) with John's conclusion, only I would go a bit further: the only significant beneficiaries of adding a lot of new TLDs are going to be ICANN's revenue stream, TLD operators who get paid whether there are significant numbers of useful (not redundant) registrations or not, those who are in the business of writing applications and marketing material, and the bad guys.

There is one other aspect of the situation that seems to be under-appreciated. Single-component domain names are problematic in many applications.  To a web browser, "foo" is much more likely to be a request to call a search engine or an abbreviation for "foo.com" (or "foo.current-ccTLD") rather than part of a URL.  To many mail clients, "user@foo" is more likely to be a local name that cannot be transmitted over the Internet rather than a TLD with mail address records.  And so on.  Those problems can be "fixed", but the fixes, whether with special syntax or "try one thing, then another" searches, will have problems of their own including creating new vunerabilities for attack.

Link | Posted on Jul 08, 2009 4:29 PM PDT by John Klensin

Link | Posted on Jul 07, 2009 5:03 PM PDT by Daniel R. Tobias

If the model sounds plausible in terms of that trust and operations overhead, then it might be worth considering the technical details for instantiating it.

/d

Link | Posted on Jul 05, 2009 10:41 AM PDT by Dave Crocker

Certification: sponsored TLDs are supposed to ensure that all of their registrants meet specific requirements, so you know that a domain in, say, .COOP is an actual co-operative. [...]

That reminds of John K.'s argument about whois records, that commerce sites (ideally .COM) should not conceal their identities. I can understand .COM, because it was very common until the 80s to design databases where critical attributes played the role of search keys.  Afterward, that habit has been stigmatized and considered a design error. Thus, it seems more difficult to understand .COOP, which has has been launched in 2002. However, the feature that triggered that change, secondary indexes, is still missing from the DNS.

Link | Posted on Jul 05, 2009 10:40 AM PDT by Ale

Link | Posted on Jul 05, 2009 8:18 AM PDT by Suresh Ramasubramanian

FUSSP = http://www.rhyolite.com/anti-spam/you-might-be.html

I'd have appreciated if you mentioned which item in Vernon's list would best identify vhlo as a FUSSP. It's not. Actually, it is neither final nor ultimate. To wit: its list of authentication/reputation mechanisms is extensible (not final), and its informative references point to further work to do (not ultimate). It is a possible solution to the spam problem, though.

The current state of affairs is such that, just like it's not possible to agree on a definition of spam, it is also not possible to find a solution for it. The anti-anti-spam paradigm enforces preemptive rebuttal of any solution proposed. Thus, it turns out we need to find a solution to the anti-spam problem before we can proceed. I knew that, that's why I asked for readiness. Thank you for your answer.

As for running filters at the sending MTA - outbound filtering is widely recommended as a best practice.

So is the submit protocol. However, the advantages of running them are confined within the environment where they operate. An outbound MTA wishing to distinguish itself from unreliable transmitters has no verb in SMTP to express that it has authenticated the sender, scanned the content, found all fine, and takes accountability for what it is about to transmit.

Then - if the entire world was following best practices we wouldnt have the sort of spam volumes that we're seeing. So .. you're left with inbound filtering.

Vhlo does not require cooperation of the whole world. Although it aims at recovering the reliability that mail had in the 90s, it only operates in transmissions between trusted parties.

The relevant alternative to inbound filtering are whitelists. For example, an office having much activity with a given company, may whitelist their outbound MTAs. By the same argument, one who trusts the outbound filtering operated by Gmail may wish to whitelist them. However, whitelisting can be hardly afforded by medium/small organizations because it requires heavy maintenance: by the time one has built a significant list of hosts, it has to be rebuilt. In facts, most mail domains do not publish the IP addresses or FQDNs of their outbound MTAs in such a way that they can be retrieved automatically. Of course, as Brett noted, it is difficult to reckon if whitelisting by domain is relevant enough to deserve standardization.

The authentication/reputation mechanisms considered in the current draft are flexible enough to allow for fine grained definitions of the set of domains one wishes to whitelist, including the explicit definition of a list of vouched mail domains. Those mechanisms are standardized already. The adjective weak used to characterize them is meant in a precise technical sense, similar to other acceptations it has in computing and science, not in the generic sense of something that doesn't work well.

Spam filtering to clean up a forwarded mail stream is not a bug, its a feature.

Since clean up means dropping messages at random, or according to non-specified and seldom predictable ways, that feature heavily affects reliability.

Link | Posted on Jul 05, 2009 1:45 AM PDT by Ale

My point, poorly expressed, was that it will be *far* easier to gain widespread acceptance of an ESMTP/EHLO extension than it will be to gain widespread acceptance of another SMTP verb.

I have reservations about the design of VHLO, but this particular objection strikes me as odd. An SMTP extension is an SMTP extension: the question of whether the extension is best expressed as a new verb or as parameters to an existing verb is driven by the semantics of the extension. In this case, the identity being conveyed is potentially relevant to the entire SMTP session, not a single mail transaction within the session, so a new verb seems more appropriate than a parameter to MAIL. On what basis do you say it is so much easier to deploy parameter-based extensions than verb-based extensions?

The work of classification still needs to be done at some point. If you do that work during the SMTP session, you will either accept and deliver the message, or the SMTP client will get an unambiguous rejection indication, so the sender will know that their message was not delivered, and they can arrange alternative means of communication.

This is a bad thing when the sender is a spammer. I don't know what the current trend in spammer ratware is—at one point it was notorious for ignoring the final post-DATA response code, and if it does, then post-DATA rejects are a good idea (so long as the software used by your preferred senders isn't similarly ratty). If spammers pay attention to such response codes, however, it gives them data on their delivery rates and whether changes to their text are improving delivery rates or not. Ultimately, if you tell a spammer you've rejected his mail, it's just an invitation for him to try, try again. The ideal way to treat spam is to accept and drop it. Explicit rejection is a better strategy in cases where the classification is less certain, however.

This is the entire reliability point that the OP was claiming that VHLO partially solves. But it can never be solved with post-acceptance spam classification schemes.

Given that the point of VHLO is to facilitate whitelisting, thereby eliminating some receiver-side classification efforts entirely, I'm not sure I see the substance of your complaint here. Perhaps the point you wanted to make was, "mail wouldn't get lost in the first place if recipients rejected spam in-line." Overlooking the issues I've already raised with explicit rejection, VHLO can still contribute to the mail process by facilitating whitelisting, thereby reducing server load and the incidence of false positives.

You might argue that it is difficult to arrange to do that work in-line during the SMTP transaction, but that would be incorrect.

It is not technically difficult, but it results in greater resource requirements than the alternative. In-line checking increases the per-session resource burden and reduces your maximum SMTP concurrency accordingly. If you take the checking out of line, then it need only keep pace with your average rate of message arrival, not your peak rate. How much difference this makes depends on message arrival patterns, of course.

Link | Posted on Jul 05, 2009 1:33 AM PDT by The Famous Brett Watson

As for running filters at the sending MTA - outbound filtering is widely recommended as a best practice.

However, most spam out there is sent through compromised hosts, botnets etc - that may not even be "regular" smtp servers at all. Or they might relay through the smtp server of their broadband provider etc - which is where the outbound filtering best practice comes in.

Then - if the entire world was following best practices we wouldnt have the sort of spam volumes that we're seeing. So .. you're left with inbound filtering.

You dont need VHLO, fancy maps, wild theories from Paul Graham etc (and by the way bayes is not the cureall - doesnt even scale all that well across a distributed mail flow, as I and a few other people had the "pleasure" of telling him back in late 2003 when he was a guest speaker at a SciAm meet the expert type session, at some pub or the other near the MIT campus)

Based on what I see here - I would personally never deploy VHLO with a bargepole.

Spam filtering to clean up a forwarded mail stream is not a bug, its a feature.

----
This memo defines an extension to the SMTP service that provides
protocol support for weak authentication of SMTP clients.  Weakly
authenticated clients enjoy an intermediate level of trust: they have
no relying privileges, but can attempt to deliver mail to local
users, are whitelisted from some filters, and may receive DSNs as
needed.

Note that this treatment is what SMTP recommends for all clients.
However, most servers operate filters to limit spam, thereby
affecting the reliability of the mail forwarding system.  Verified-
Hello recovers that reliability by providing for uncensored mail
transmission in a framework where authenticated domains are
responsible for the messages they send.  In addition, support is
provided for an extensible set of authentication mechanisms, so that
they can be managed and branded.
-----

Link | Posted on Jul 04, 2009 8:27 PM PDT by Suresh Ramasubramanian

My point, poorly expressed, was that it will be *far* easier to gain widespread acceptance of an ESMTP/EHLO extension than it will be to gain widespread acceptance of another SMTP verb. We already have ESMTP which allows the receiver to advertise extensions that it understands.  Any data that you might want to pass with VHLO could be passed with something like "MAIL FROM:<user> VHLO=arguments".

"and there are other points to be made in favour of accepting all mail (with post-classification of some kind)"

The work of classification still needs to be done at some point. If you do that work during the SMTP session, you will either accept and deliver the message, or the SMTP client will get an unambiguous rejection indication, so the sender will know that their message was not delivered, and they can arrange alternative means of communication. *Any* system that does that classification work after accepting the message *will* be unreliable unless the classification mechanism is perfect. When it makes a mistake, and classifies a wanted message as spam, then the sender will think it has been delivered, and the receiver won't see it. This is the entire reliability point that the OP was claiming that VHLO partially solves. But it can never be solved with post-acceptance spam classification schemes.

You might argue that it is difficult to arrange to do that work in-line during the SMTP transaction, but that would be incorrect. The sendmail milter mechanism allows one to run arbitrary code of your choice during the SMTP transaction. If the MTA of your choice does not allow that, then perhaps you need a more flexible MTA. My point is that *any* computation that you could do post-acceptance can also be done before you send the answer to the DATA command. Why not be polite and let the SMTP client know the results of that computation?

Link | Posted on Jul 04, 2009 9:45 AM PDT by Carl Byington

You will need some other property of that MTA (other than the fact that it issued VHLO rather than EHLO) to trigger the acceptance of mail from it.

VHLO provides a means for the sending system to assert association with a particular domain name, providing information as to how the recipient may verify this assertion (through SPF records, PTR records, etc). A message is accepted without further filtering only if the domain identity in question is both verifiable and whitelisted (either locally, or by the recommendation of a trusted third party).

In particular, the AUTH extension seems to cover almost all of this case.

The AUTH extension is for authentication by prior arrangement. VHLO offers a weaker kind of authentication at the level of the domain name which does not require explicit prior arrangement or PKI certification.

If that MTA is "trusted" and has a static ip address, then you could simply use a local whitelist on the receiver to get the same effect.

True, particularly if third-party whitelists allow reference by IP address instead of domain name. The advantage of using a domain name identity is that it decouples identity from network addressing, which simplifies management: it means you don't run into trouble every time you fiddle with your outgoing pool of mail hosts. Whether or not this benefit is worth the effort of developing and implementing VHLO is open to question, of course.

You should *almost never* actually accept (2xx response to DATA) a message and then make some spam/ham decision about it.  Modern systems make that decision *before* returning the status code to the DATA command.

That's an ideological stance. I used to agree with it. I'll agree that accepting and then bouncing a message is a bad idea (unless you can somehow verify that the owner of the bounce address has authorised the bounce—a very special case indeed). As for classification during SMTP being "modern", however, Gmail begs to differ.

I've done a more detailed analysis of spam classification and acceptance/rejection strategies in my PhD thesis (due for submission this month), and there are other points to be made in favour of accepting all mail (with post-classification of some kind). If there's any interest, I may publish an extract of that work as an article here. (Use CircleID's "send message" facility to express interest or see contact details here.)

Link | Posted on Jul 04, 2009 12:47 AM PDT by The Famous Brett Watson

.com isn’t sold or advertised by VeriSign. It is sold and branded implicitly by virtually every major corporation using it day in day out across all their communications. It’s that simple.

The success of the internet is because it delivers efficiency to the market place. It enables anyone to reach an unfathomable number of people simply by buying a domain name. Cost - $10 + hosting per year. Mind blowing!

ICANN’s new gTLDs for corporations and brands changes this by taking advantage of the efficiencies afforded by the original design of the Internet over non internet models. And in doing so creates a super league the cost of entry to which is $185,000 and $25,000 + hosting per year.

Recent trends in brand evolution have led to many websites both on and off line using a more and more minimalist form

http://www.brand.com
www.brand.com
brand.com
.brand if allowed may follow

If this happens and is reinforced worldwide in corporate communications day in day out users will quickly come to recognize that a brand to the right of the dot is a major player and therefore by implication a brand to the left of the dot will be perceived as a lesser brand.

The level playing field of the internet is destroyed and a super league created.

The Creation of a Super league

There has to be serious concerns not least because a single layer model to the right of the dot can never replicate the complexities of businesses around the world.
Whilst initially appearing to offer more freedom for new domains it actually offers less freedom.

For example if there is .dell .ibm what about brands like .hp? HP is seriously disadvantaged simply because its brand is 2 letters and 2 letters are reserved for country codes.

Or the fact that it offers a system where there can only be one organization to the right of the dot - ever! This is a step backwards from the existing system which by careful management of competing open generic gTLDs allows multiple totally separate entities to each enjoy a similar level of branding in the second level to the left of the dot.

What about organizations whose names conflict with geographic areas? .amazon? What about organizations or brands that share a name with places that may in the future have a need for an internet presence? .moon or .saturn etc. What about companies whose brands are already taken like .cat?

But most importantly a Super league destroys the ability to compete on a level playing field. At the moment to launch some software designed compete to with Microsoft or Sun its $10 + hosting a year then it’s down to skill and innovation.

A super league changes this and medium sized players will have to consider whether it worth spending $185,000 + $25,000 per year with ICANN to enjoy the same level of branding and enter the Super league. For startups and smaller players cost of admission to this implicit branding advantage is likely to prove prohibitive.

The Creation of Private Monopolies

If day to day usage and advertising of corporate bands to the right of the dot means they enjoy competitive advantage then generic names to the right of the dot will become to enjoy a similar branding advantage.

Generic names such as .news .shop .store .music .radio and .movie will become to be perceived as superior. Their simple existence will create a series of worldwide monopolies.

What happens if Microsoft applies for .search? If they are granted rights to .search how is google.search handled? May be Microsoft would be happy to allocate it to Google especially if they can then use shopping.seach, images.search & video.search. themselves.

What happens if Rupert Murdoch purchases a controlling interest in a company which is awarded .news?


btw I always wondered who first thought of the splitting Registries and Registrars - pure brilliance! An irony is some of the people calling for relaxation of the boundaries actually have the most to loose if the protection the current model affords them is taken away.

Link | Posted on Jul 03, 2009 5:08 PM PDT by gpmgroup.com

If that MTA is "trusted" and has a static ip address, then you could simply use a local whitelist on the receiver to get the same effect.

"The receiving server runs content filtering, but doesn't know what to do in case an accepted message turns out to be spam". In that case, the receiving server is broken. You should *almost never* actually accept (2xx response to DATA) a message and then make some spam/ham decision about it.  Modern systems make that decision *before* returning the status code to the DATA command.

Link | Posted on Jul 03, 2009 3:27 PM PDT by Carl Byington

Link | Posted on Jul 02, 2009 11:46 AM PDT by Eric Brunner-Williams