Musings from a small IPP

Well you are of course.

Then there are spammers, whom we try to control with SPF records in the DNS to limit where your email can come from to the servers you use.

Then there are misguided websites which try to use your email address to send mail when someone fills in a form claiming to be you.  You didn’t send that mail, the website did, so the 4 players in the message should be set appropriately; they are

From — who he message purports to be from — you in this case
Sender — the website administrator
Reply address — again probably you
Error Reports — Probably should go to the website admin

Websites will usually fix this once they understand the problem.

Finally there are people doing stuff for you, for instance payment portal providers, or webservice consolidation sites such as  house or employment search services.  Most of the above applies, with the added wrinkle of you as the named responsable (EU French term for the person accountable) for the service.

The portal will correctly show you as the From address, but should not be using your email as the sending address that’s just simply wrong, if you do agree to it then the portal should provide a portal SPF record that can be included by your own SPF record, how otherwise are you supposed to maintain your email security with random contractors thinking they can send as though they are you vs someone providing a service to you?

One does not use another company’s headed notepaper after all. That’s fraud. Why then should you use another company’s email address, without making the ‘on behalf of‘ relationship clear?

Sometimes you have to give up, and take the hit of extra SPAM.  For me personally, one of the more valuable features of ASSP’s SPF checks was to apply the check to incoming addresses that are otherwise whitelisted, so that no-one can hijack a genuine contact’s email address to send one spam.

But there is always the wilfully ignorant customer of one’s customers who refuses to believe that their IT provider has not authorised the way they are sending mail.  “Lots of other places receive my mail”  — well yes but they’re not checking, “Well it must be your fault, and no I can’t talk to my IT department, you fix it” — phone slams down.

When the person who will not listen is controlling a significant spend some form of smiling compliance is forced.

We still have a list of domains where we force SPF checks, and let’s say a big “THANK YOU’ to HSBC, PayPal and RBS who actually care enough for their customers to protect them from phishing with SPF.

In contrast one ought to hold up to ridicule Barclays, NatWest, Santander, Bank of Scotland, Lloyds, Halifax etc who (as of today) still leave their customers open to attack.

“Ok so how do I make a big SPF record? I’ve run over the size of a DNS TXT record.”

Well as I commented elsewhere you can’t simply have a bunch of records as only the first one received will be applied, and if the software notices there’s more than one it’ll throw a wobbly. You can however split your spf record up like this:

$ORIGIN X.com.
@           IN      TXT     "v=spf1 a mx include:part1.x.com include:part2.x.com include:part3.x.com -all"
part1       IN      TXT     "v=spf1 ip4:192.168.128.0/20 -all"
part2       IN      TXT     "v=spf1 ip4:192.168.64.0/22 -all"
part3       IN      TXT     "v=spf1 ip4:192.168.192.0/18 -all"

Of course if you’re setting up for a large multinational, then it’s sensible to make the included parts correspond to your national gateways so you can use the a and mx tags, and avoid having to set up separate SPF records for national presentation.

I’ve been rehearsing that comment for a while, as people tell me they can’t send mail to my customers, and what am I going to do about it?

SPF is the Sender Policy Framework.  If a mail fails SPF that means that the sender (you)  has violated the policy set forth by the owner of the sending domain, or their agents, in an SPF record; the recipient is not an actor in this

By publishing an SPF record the sending domain’s owner has asked the world to make some simple checks that mail is coming from the correct place.  These checks help with avoiding being impersonated.

When a correct rejection happens, we never hear about it.  When a rejection hits someone who might legitimately expect to use the domain there are two possible causes, and one solution.

1. The mail is coming from the wrong place, such as sending work mail through a home ISP because the VPN to work has dropped, or perhaps there’s a new gateway at work which hasn’t been configured into the SPF.
   Another common case is trying to use an SPF protected address from public webmail providers who have not configured their servers in an SPF aware way.
2. Your domain’s SPF record is scrambled, so the receiving software can’t make a list of the places the mail is allowed to come from

The solution in both cases is to fix the SPF record, not to ask the recipient to special-case your mail.

There is a third case of websites doing the wrong thing which I’ve dealt with elsewhere

It’s amazing how easy it is to get something wrong when configuring complex systems. When it comes to networking the first thing that people notice is when their e-mail goes wrong, because e-mail is a fairly complex application these days that calls upon all of the underlying services in many unexpected ways. (continue reading…)

Shortcuts make life easier for us.  For administrators configuring DNS there is a great shortcut which tells the program reading the zone where it is.  This lets the administrator leave off the domain part of the thing they’re configuring.

OK that  sounds complicated, so let’s give an example – if in a DNS file I were to write

$ORIGIN X.com.

then a few lines later I can say “mail” and “mail.X.com.” will be understood.  Well and good, though often a source of problems when someone leaves off a terminating ‘.’ and gets the domain added on where they were not expecting it.

SPF also has a chance to get messed up here.   Today’s gem was a record with

"v=spf1 mx a:ironport a:sandberg"

which makes one think the administrator setting it up was expecting that shorthand to apply to those ‘a:’ elements. It’s not clear what they thought they were doing for non-matching source addresses, as they left off a closure element.

"v=spf1 mx a:ironport.X.com a:sandberg.X.com -all"

was of course what they meant, but the software isn’t meant to follow inferences, rather it fails their SPF validation with a permanent error.

Today a customer wrote to ask for help getting mail from a company — let’s call it X — hat was suddenly not getting through.

X has 4 SPF rrcords

• v=spf1 mx include:isp1 -all
• v=spf1 mx include:isp2 -all
• v=spf1 mx include:isp3 -all
• v=spf1 mx include:isp4 -all

So depending on which of these four records is randomly received first, one of their supporting ISPs is authorized to forward X’s outbound email, — and — the other three ISPs are not authorized.   Actually X also had a syntax error in one of their 4 records which seriously did not help.

The SPF record X should have is of course:

• v=spf1 mx include:isp1 include:isp2 include:isp3 include:isp4 -all

Today I had a designer expressing the view that  mail rejection was the recipient’s “fault” not their web form.

They were using the user supplied e-mail address as the sender of the message generated by their web form.

The difference between Sender and From is that From is part of the headers, and is used as a reply address in the absence of a Reply-to header, whereas Sender is used in the SMTP transaction as the Mail From part.

Mail From/Sender is often different from From — eg when a secretary sends on behalf of their boss, and this is reflected in SMTP email with that distinction.

It is an issue these days as there is an anti-SPAM mechanism called SPF (Sender Policy Framework) [q.v. for details], but in summary it allows the owners of a domain to say ‘Legitimate mail from our domain only comes from these places” (usually their mail server, and web server), and add, “if it comes from elsewhere discard it as SPAM’ — now the thing to notice here is it is a tag that applies to the sender address (Mail From in SMTP), and may have nothing whatsoever to do with the recipient who is being protected from SPAM, and the owners of a domain are not going to authorise any and all random websites to use their domain to send mail as their company, just so that it can reply to them.

Gmail and hotmail for instance use the SPF mechanism to protect their customers of being accused of spamming, so the constituency your form will fail for is fairly large

This is why any web form you create has to use a legitimate sending address for that server, even if you put the address of the form-filler into the From field where it will be displayed to the recipient. It also means you are taking responsibility for any abuse of forms on your server.

The scourge of the Internet is SPAM mail.  So much rubbish arrives each day — it is a major part of the service we offer to our customers that we block tag and pre-file most of it for them, so they can (we hope) have uncluttered real communication.

With our efforts to clean-up what comes in goes a responsibility to keep the outgoing stream of mail clean.  In general we trust our customers to keep their anti-virus up-to-date but there remains a risk of their computer becoming infected in some way, or worse becoming part of a bot-net (q.v.). We would expect to notice if their sending pattern suddenly changed, and we require an authenticated connection before we will forward mail.

Many domestic market ISPs will intercept IP port 25 traffic — mail traffic — for just this reason — they don’t want to be seen as hosters of SPAM.  As we’re not in the local loop business we are set up to accept outgoing mail on Port 587 (submit) which lets our customers bypass the blocks on port 25 and get their mail out from the right place, and lets our anti-spam software take note of their correspondent’s addresses to facilitate the replies.  Our customer’s SPF records have a clause include:oa5.com to allow their mail to come from our servers.

This leads on to some thoughts about SPF and ISP relayed mail.  There are really three cases that spring to mind

1. The user has their mail in the ISP’s domain eg aol, waitrose, tiscali, and does not use their own domain
2. The user has their own domain, but actively uses the ISP’s mail servers to handle the outgoing traffic
3. The user has their own domain, and their own mail server, but the ISP actively intercepts outgoing traffic on port 25 to mediate for SPAM

Cases where the ISP is not involved are not interesting here.

Case 1 is just for the ISP. If the ISP is at all concerner for its reputation it will set up appropriate SPF records to prevent its own, and its customer’s mails being spoofed.

Case 2 is more interesting for the concerned domain owner.  If the ISP has an SPF record, that can be pulled into the domain’s own SPF record with an ‘include:‘ clause.  Absent that ISP record though; the domain owner has two choices:

• Do without an SPF record – or –
• Run a full mail server, and deliver mail directly

as there is no way for the customer to enumerate the ISP’s outgoing mail servers.

The third case is even more interesting in it’s own way.  The ISP is doing something that in Britain should be restricted by RIPA (The Regulation of Investigatory Powers Act otherwise known as the official snoopers charter).  By interposing itself as a relay the ISP is breaking the customer’s SPF configuration, and so needs to run the Sender Rewriting Scheme on the outgoing messages to enable them to be delivered.

I don’t think any do.

Eh wazzat? Well I suppose most of you haven’t heard about it, or if you have it’s probably in the negative context of someone’s mail not getting to you…

What is SPF then?

SPF is a nice easy way for the owners of say mybank.biz to say where they send e-mails from.  Of itself that doesn’t sound too exciting.  Well no, except it has an interesting consequence.  If mybank.biz only sends email from mailout.mybank.biz then when your ISP gets mail which says it is from security@mybank.biz on a connection from phish.ripoff.crime your ISP knows it can throw that mail away and never bother you with it.

Sounds good, so all the banks use SPF then?

Nice thought, a few responsible ones do.  Mostly they don’t yet.  You might have a good argument about negligence if you got caught out, and it turns out your bank does not, while your ISP does check for you.  Some of the (financially) important internet sites like E-Bay do  protect their customers and themselves in this way.  Others use it just to avoid being accused of distributing SPAM or viruses

So something must break, or everyone would have it

Well spotted.  When the institution the mail comes from has SPF you can’t have an alias somewhere that forwards the e-mail to you, unless the ISP offering that alias is really switched on, because the machine doing the forwarding is not on the sender’s list of allowed sending machines.  We can do it at OA5, but it is significantly more work, so we would rather not, thank you.

The other thing that affects people whose institution has SPF is a common mistake on some web sites.  Those sites think they’re allowed to use your email address as the sender when they’re sending to you.  They should of course be sending from their own address and only using your address as recipient of the mail they’re sending.  Responsible sites who know what they’re doing don’t do this to start with, or quickly change when the problem is pointed out to them.

What about my Blackberry?  That sends mail from the phone company’s machines, will I get blocked?

Blackberry and the phone companies know about SPF, and use the distinction between the Sender address and the From address to be able to deliver your mail.

At OA5.com we set up SPF for our customer’s domains