Tuesday, August 28, 2007

Sendmail + AUTH +SSL tunnel -> ATT Yahoo!

I am running a small mail server on my home network. It used to relay mails to ATT (formerly SBC) Yahoo's mail server through port 25 and everything works fine. But in July, ATT Yahoo! Customer Care sends me an email saying that they are making some security improvements and urge me to take immediate actions. Well, it turned out that they want to force me to use SMTPS (port 465) with SSL enabled for my email clients.

SMTPS was introduced by Microsoft in 90s. The problem with SMTPS is that there is never a standard (no RFC) nor it is an extension of SMTP. Alternatively, there is a standard way to use SSL/TLS with SMTP (RFC 3207) which is to use STARTTLS extension. This poses some challenge to sendmail configuration since it supports STARTTLS but not SMTPS. I dug Internet for hours and couldn't find a good tutorial to enable SMTPS for sendmail. Then I came up with the idea of using Stunnel together with sendmail. This approach matches the concept of SMTPS perfectly since it uses SSL strictly as transport layer and Sendmail doesn't have to know anything about SSL. It doesn't require special Sendmail compile-time options such as SASL, STARTTLS so the installation and configuration of sendmail are greatly simplified.

The following instructions work on FreeBSD 6.2 but they should apply to Linux and other *nixes with some minor tweaks.

Step 1: Set up S-Tunnel


Install Stunnel version 4.2. Stunnel-3.x should work fine too. Run command:

stunnel -c -d 2525 -r smtp.att.yahoo.com:smtps

This creates a SSL tunnel to Yahoo's mail server "smtp.att.yahoo.com". I used local port 2525 for tunneled SMTP traffic. Sendmail will need to connect to local 2525 port to relay emails to mail server.

Step 2: Configure Sendmail


Step 2.1: File authinfo


First of all, your sendmail distribution must have SASL complied in because Yahoo mail server requires sendmail to provide login information so it can authenticate to SMTP server. This is done by authinfo file. Since this file contains your ISP account username and password, so protect it by setting the correct privileges on this file. Do the following as root

mkdir /etc/mail/auth
chmod 700 /etc/mail/auth

Now create file /etc/mail/auth/authinfo, which is a simple text file. My authinfo file contains:

AuthInfo: "U:myname@sbcglobal.net" "I:myname@sbcglobal.net" "P:xxxxxx" "M:LOGIN PLAIN"

You may put multiple lines into this file, such as

AuthInfo:relayhost1 "U:myname@sbcglobal.net" "I:myname@sbcglobal.net" "P:xxxxxx" "M:LOGIN PLAIN"
AuthInfo:relayhost2 "U:myname@sbcglobal.net" "I:myname@sbcglobal.net" "P:xxxxxx" "M:LOGIN PLAIN"
AuthInfo: "U:myname@sbcglobal.net" "I:myname@sbcglobal.net" "P:xxxxxx" "M:LOGIN PLAIN"


I would like to point out several common mistakes when creating this file:

  • The string after "Authinfo:" is the hostname of the relay server and the key that Sendmail searches for authentication information. It must be a real DNS name instead of CNAME. For example, smtp.att.yahoo.com won't work here since it is actually a CNAME of smtp.sbc.mail.yahoo4.akadns.net. You have to use smtp.sbc.mail.yahoo4.akadns.net here.
  • You may notice that I didn't put anything after "Authinfo:" in my sample authinfo file. This is intentional. Sendmail ALWAYS use the line with "Authinfo:" if it can't find an exact hostname match. Since I have only one relay server, it is ok to use one set of username/password instead of dealing with all those DNS name changes.
  • "U:" specifies authorization name and "I:" specifies authentication name. In most cases, they should be the same.


Now don't forget to run makemap to make it available to Sendmail.

makemap hash authinfo < authinfo

This command creates a file authinfo.db.

Step 2.2: Configure Sendmail


I am using Sendmail 8.14. You should use the latest sendmail release since newer releases address important security issues. With this method, no special compile-time directive is required. The configuration file for sendmail is very complex for beginners and you may want to grab the O'Reilly book to understand it better. The exercise we are doing here is very simple so don't be intimidated.

On FreeBSD, I usually modify 'myhostname.mc' and then do 'make cf', which creates myhostname.cf. Then I copy it to /etc/mail/sendmail.cf. On your system, you may have sendmail.mc and then compile it manually using m4.

m4 sendmail.mc > /etc/mail/sendmail.cf

In sendmail.mc, we add two new lines:

define(`SMART_HOST', `[127.0.0.1]')
FEATURE(`authinfo', `hash /etc/mail/auth/authinfo.db')

That are the ONLY two lines that you need (well, i lied, read on...) in sendmail.mc if you use sendmail as client to relay server.

Now if you restart sendmail and try it out, you are going to find the traffic is not going to the local port 2525. There are two issues:

  1. We need to pass the port of relay host (2525) to Sendmail.
  2. Sendmail default rulesets prevent you from using localhost for relay, which makes sense normally but not in our case.

So, I will modify sendmail.cf (NOTICE it is the sendmail configuration file instead of .mc file since I can't find M4 macros for it)as following:

$ diff -c sendmail.cf.old sendmail.cf
*** sendmail.cf.old Tue Aug 28 18:21:37 2007
--- sendmail.cf Tue Aug 28 18:23:33 2007
***************
*** 269,275 ****
O UseErrorsTo=False

# log level
! O LogLevel=9

# send to me too, even in an alias expansion?
#O MeToo=True
--- 269,275 ----
O UseErrorsTo=False

# log level
! O LogLevel=20

# send to me too, even in an alias expansion?
#O MeToo=True
***************
*** 961,967 ****
R< local : $* > $* $>CanonLocal < $1 > $2
R< $~[ : $+ @ $+ > $*<$*>$* $# $1 $@ $3 $: $2<@$3> use literal user
R< $~[ : $+ > $* $# $1 $@ $2 $: $3 try qualified mailer
! R< $=w > $* $@ $2 delete local host
R< $+ > $* $#relay $@ $1 $: $2 use unqualified mailer

###################################################################
--- 961,968 ----
R< local : $* > $* $>CanonLocal < $1 > $2
R< $~[ : $+ @ $+ > $*<$*>$* $# $1 $@ $3 $: $2<@$3> use literal user
R< $~[ : $+ > $* $# $1 $@ $2 $: $3 try qualified mailer
! ### XXX It is OK to use localhost as relay
! #R< $=w > $* $@ $2 delete local host
R< $+ > $* $#relay $@ $1 $: $2 use unqualified mailer

###################################################################
***************
*** 1835,1839 ****
A=TCP $h
Mrelay, P=[IPC], F=mDFMuXa8, S=EnvFromSMTP/HdrFromSMTP, R=MasqSMTP, E=\r\n, L=2040,
T=DNS/RFC822/SMTP,
! A=TCP $h

--- 1836,1840 ----
A=TCP $h
Mrelay, P=[IPC], F=mDFMuXa8, S=EnvFromSMTP/HdrFromSMTP, R=MasqSMTP, E=\r\n, L=2040,
T=DNS/RFC822/SMTP,
! A=TCP $h 2525


There are 3 changes:

  • Change LogLevel to 20 so I can see some debug information in /var/log/maillog
  • Remove the rule that prevents localhost to be relay
  • add port 2525 to the parameter for 'relay' mail agent


Step 3: Test



Now we have everything set. Restart sendmail and send a test email. Inspect /var/log/maillog, look for lines

Aug 28 18:24:31 snake sm-mta[9883]: l7T1OVt8009881: SMTP outgoing connect on localhost
Aug 28 18:24:32 snake sm-mta[9883]: AUTH=client, relay=[127.0.0.1], mech=LOGIN, bits=0
Aug 28 18:24:33 snake sm-mta[9883]: l7T1OVt8009881: --- 050 ... Sent (ok 1188350673 qp 65294)
Aug 28 18:24:33 snake sm-mta[9883]: l7T1OVt8009881: to=, ctladdr= (1001/1001), delay=00:00:02, xdelay=00:00:02, mailer=relay, pri=30491, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (ok 1188350673 qp 65294)
Aug 28 18:24:33 snake sm-mta[9883]: l7T1OVt8009881: done; delay=00:00:02, ntries=1
Aug 28 18:24:33 snake sm-mta[9883]: NOQUEUE: --- 050 Closing connection to [127.0.0.1]


You can see sendmail connect to localhost (which is tunneled to ATT Yahoo mail server) and use AUTH mechanism 'LOGIN'. If you have a sendmail version without SASL you will see 530 Auth Required error.

Update (09/10/2008): Please make sure to check out this new post which deals with a new restriction that AT&T requires you to use a verified email address to send mails.

21 comments:

Larry said...

I'm not an expert in Linux. I try your very first command to create a "Stunnel" and it failed.

I'm using fedora core 5. Any idea?

Qiao said...

You probably don't have Stunnel installed. You may install it using rpm command on Linux.

a.p. said...

Got it to work in linux using inetd/tcpd.

First, stunnel behaves a little differently on linux --it requires a config file:

file: /etc/mail/stunnel_cf
[code]
client=yes
accept=2525
connect=smtp.att.yahoo.com:smtps
[/code]

Then, we need to edit /etc/inetd.conf to monitor our port of choice (2525 in your example)

line added to /etc/inetd.conf:
[code]
2525 stream tcp nowait root /usr/sbin/tcpd /usr/sbin/stunnel /etc/mail/stunnel_cf
[/code]

(sorry I dont know how to format code in blog coments)

Thus, stunnel will automagically make the secure connection upon each sendmail forwarding request to localhost:2525

Cheers!

Larry said...

stunnel -c -d 2525 -r smtp.att.yahoo.com:smtps

What are the switch in the first stunnel means? When I run this comment I got an error that -c: No such file or directory

I have stunnel 4.15-2 and 4.20 installed on two systems running redhat 5

a.p. said...

Larry, try "stunnel -help" and see what commands/options your version takes. My version (4.04) won't take "-c" either.

Larry said...

a.p.
How did you get your to work?

stunnel --help does not have any switch close to what described here "-c" "-d" "-r" <<<< what are these switches. I can't substitute them with others if I don't know what he is trying to do.

thanks

a.p. said...

Larry, see my comment above (3rd). First, I created teh stunnel.cf file as described. Then i added the config line shown above in /etc/inetd.conf

After that you need to make inetd reload the config file: "killall -HUP inetd"

Do all these as superuser (root).

a.p. said...

D'oh, forgot to mention that everything else from Qiao's step 2 on applies as he described.

Larry said...

What suppose to be in the /etc/inetd.conf file?
I dont have this file.

a.p. said...

Larry, I don't know what Linux distro you have. I had assumed yours had inetd installed, up and running. Just run the following command

/usr/sbin/stunnel /etc/mail/stunnel_cf

Substitute "/etc/mail/" with your actual path to the stunnel config file.

If this info is still not helping then I'm afraid I'm running out of suggestions, maybe you need to develop a little more Linux knowledge before taking on a task like this.

Good luck

Larry said...

a.p.
Thank you for all your reply.
just because you specified to add a line in the "/etc/inetd.conf" but that file does not exist. Do you mean the "/etc/xinetd.conf"?

I'm running RedHat Linux 5.

also do you mean /etc/mail/stunnel.cf instead of /etc/mail/stunnel_cf ?

a.p. said...

Now we're getting closer. I believe xinetd has a different conf file format/syntax from inetd. You may want to look into a few examples off the net, or at the man page.

But first off, does your stunnel command work?

Whether the conf file is called stunnel_cf or stunnel.cf doesn't really matter, just make sure you point stunnel to the correct path/file.

Larry said...

here is the error I got from the stunnel command

file stunnel.cf line 3: accept is not allowed in inetd mode

a.p. said...
This comment has been removed by the author.
a.p. said...

Good, this means you have stunnel and can run it. Inetd or xinetd is still the easiest and safest way to implement. If you want to run standalone instead, use this following cf file:

[smtp_s]
accept=2525
connect=smtp.att.yahoo.com:smtps


There may be some error messages in your syslog complaining about access to stunnel.pem. Refer to your stunnel manual for help with setting that up using the command:

man stunnel

This exhausts my knowledge about the entire sendmail/stunnel deal. It works for me under my particular distro via inetd using the configs suggested by Qiao and those shown in the 3rd post. Where there are differences in software version and syntax, man pages should be of help.

Good luck

djf said...

I've gotten the stunnel working in centos 5 with xinetd (telnet localhost 2525 gets me the at&t mail server) but nothing I've done seems to actually get sendmail to use port 2525 (maillog never seems to show even an attempt to connect there). Perhaps I've a setting wrong that's keeping it from trying to relay at all?

I did find this macro:

define(`RELAY_MAILER_ARGS',`TCP $h 2525')dnl

which will correctly set the port in sendmail.cf, eliminating one of the 3 manual edits to sendmail.cf in the main post.

This is the only post I've found in a number of searches that even tries to address mail forwarding for the new AT&T secure service. Thanks for posting this, I know I'm a lot closer than I was after reading it.

here's the parts of my sendmail.mc and authinfo that I've messed with if that helps:

define(`SMART_HOST', `[127.0.0.1]')dnl
define(`RELAY_MAILER_ARGS',`TCP $h 2525')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
FEATURE(`authinfo', `hash /etc/mail/authinfo.db')dnl


AuthInfo:smtp.att.yahoo.com "U:mylogin@att.net" "I:mylogin@att.net" "P:xxxxxxx" "M: LOGIN PLAIN"
AuthInfo: "U:mylogin@att.net" "I:mylogin@att.net" "P:xxxxxxx" "M: LOGIN PLAIN"

Thanks in advance if anyone is happens to have any ideas.

a.p. said...

Djf, does your /etc/hosts file resolve 127.0.0.1 to localhost and all other names by which you refer it to? In particular, sendmail requires that IP addresses resolve to resolvable fully qualified domain names. You may even have to use a service such as DynDNS for that purpose. Here is what my /etc/hosts file lists for localhost:

[code]
127.0.0.1 mymailer.mydomain.org mymailer localhost
[/code]

AP

djf said...

a.p.
Sorry for the long delay on answer, too many things up in the air right now.

No, it has it's real name was on 10.0.0.2 (it's eth0 address), not on the 127.0.0.1 address.

it looks like this:

[code]
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
10.0.0.2 host host.local host.mydomain.net
[/code]

we do have dynDNS setup for host.mydomain.net but it resolves to the actual ISP dynamic address that's passed through to the server, not to it's 10.0.0.2 address.

I tried changing the order on the 10.0.0.2 line to FQDN first and then I tried commenting that line out and putting the FQDN first on the 127.0.0.1 line.

when I watch /var/log/maillog for either config and send a local mail for off host delivery I see it connect locally and deliver, and the mail accepted. then I see it try and connect to localhost for relay and fail with this error:

Jan 6 05:04:05 host sendmail[10808]: o06545CP010806: to=, ctladdr= (500/500), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=120356, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection reset by [127.0.0.1]

[note that the blog software stripped out the email address on "to" and "ctladdr" for me, they are actually there and correct.]

To me that seems as if it tried localhost not localhost 2525. Do you know if it would show "relay=[127.0.0.1 2525]" or not?

If I "telnet localhost 2525" I still get AT&T's mail server without a problem, so I think it should have more than just "connection rest" if it was correctly making it to 2525

I've never gotten sendmail working on this host, and it's been years since I used sendmail instead of postfix, so it could be just about anything.

Any other ideas are appreciated. Thanks for taking the time to answer.

oh -- and if no one else has found it, I came across this for the sendmail.mc file:

define(`confLOG_LEVEL', `20')dnl


so that's 2 of the 3 hand edits you no longer need in the sendmail.cf from the original post now. You still need to comment out the "delete local host" line.

a.p. said...

DJF, I may be mistaken, but changes made to macro config files need to be compiled into rc files using M4 before they take effect.

Also, you have to run make in the /etc/mail directory so sendmail could rebuild its databases. I have never messed with the mc files simply because the entire concept is too complex for my level of expertise.

My mail logs also don't show any indication of the port it actually uses, so your log looks OK except for the "deferred" bit.

Aside from making sure you properly compile the macros, one other thing you could try is this diagnostic:

- create an email-structured text file mail.txt that looks something like this:

[mail.txt]
To: valid.recipient@domain.com
Subject: just testing
From: you@yourhost.org

Some text
[/mail.txt]

- run the following command, all in one line:

[code]
cat mail.txt | /usr/lib/sendmail -v -t > maildump.txt 2>&1
[/code]

- check the maildump.txt that was just created for any error message you might get.

AP

djf said...

I finally got back to this, and found that what was left was that AT&T needed me to register all the email accounts on the server that should be able to send outbound mail. to do that I did have to turn on inbound mail to get their confirmation messages. that took me a bit to figure out that I had localhost limitation still in the .mc file. once I got that done, it's all working now. thanks for the helpful article and comments everyone.

-djf

Bart said...

Since today i cannot send emails with stunnel anymore:
I tried to redo certs but no luck

sendmail says:

Aug 13 22:01:13 domain sendmail[5671]: STARTTLS=client, error: SSL_CTX_use_certificate_file(/var/myca/sendmail_req.pem) failed
Aug 13 22:01:13 domain sendmail[5671]: STARTTLS=client, error: SSL_CTX_check_private_key failed(/var/myca/sendmail_req.pem): 0