New Safari Extension - WasteNoTime

WasteNoTime is a new Safari Extension which allows you to block those time-wasting sites that can suck the life out of your day. All you need to do is to specify which sites to block and when to block them. You may add sites to be blocked in batch mode or do it while you are browsing. And you can configure the maximum amount of time you should spend on browsing each day, within and outside work hours separately. You may define multiple time ranges to match your work habits.



This extension is similar to LeechBlock extension on FireFox. It can be downloaded through Apple Safari Extension Gallery or WasteNoTime website.

AT&T Yahoo Email Address Verification and Sendmail

In my last post, I described how to send mail from an Unix server with AT&T Yahoo! DSL line because SMTP ports are blocked and you have to send emails through Yahoo's secure SMTP server. Now AT&T added another restriction. I received the following email from AT&T:

Dear AT&T Internet Services Member,

We have received questions regarding the following error message which is received by customers when they send e-mail using a non-verified e-mail address:

“Unable to send the message. Please verify the e-mail address in your account properties. The server responded: 553 From: address not verified.”

Ah, it requires that the sender's address to be verified. The hostname of my FreeBSD server is not in public DNS server so it can't really receive emails so there is no way to verify the email address. Fortunately, Sendmail allows you to masquerade sender's address easily. You just need to add the following three lines into your sendmail.mc (Don't forget to apply the patch to the result sendmail.cf if you want to use sendmail with stunnel on localhost, as described in my last post)

FEATURE(masquerade_envelope)
FEATURE(genericstable, `hash -o /etc/mail/genericstable')
GENERICS_DOMAIN(your_domain_name)

Replace your_domain_name with the email domain that your Unix server uses. Then you need to modify /etc/mail/genericstable file. The syntax is pretty simple:

user1 verified_address

The verified_address doesn't have to be from yahoo.com. Any email address that you own and verified with Yahoo will do. The genericstable file basically says that for those users at the generic domains (defined by GENERICS_DOMAIN) the sender address should be replaced with the verified_address. Make a hash out of the new genericstable file,

makemap hash genericstable.db < genericstable

Now restart sendmail and you are all set.

Sendmail + AUTH +SSL tunnel -> ATT Yahoo!

I am running a small mail server on my home network. It used to relay mails to ATT (formerly SBC) Yahoo's mail server through port 25 and everything works fine. But in July, ATT Yahoo! Customer Care sends me an email saying that they are making some security improvements and urge me to take immediate actions. Well, it turned out that they want to force me to use SMTPS (port 465) with SSL enabled for my email clients.

SMTPS was introduced by Microsoft in 90s. The problem with SMTPS is that there is never a standard (no RFC) nor it is an extension of SMTP. Alternatively, there is a standard way to use SSL/TLS with SMTP (RFC 3207) which is to use STARTTLS extension. This poses some challenge to sendmail configuration since it supports STARTTLS but not SMTPS. I dug Internet for hours and couldn't find a good tutorial to enable SMTPS for sendmail. Then I came up with the idea of using Stunnel together with sendmail. This approach matches the concept of SMTPS perfectly since it uses SSL strictly as transport layer and Sendmail doesn't have to know anything about SSL. It doesn't require special Sendmail compile-time options such as SASL, STARTTLS so the installation and configuration of sendmail are greatly simplified.

The following instructions work on FreeBSD 6.2 but they should apply to Linux and other *nixes with some minor tweaks.

Step 1: Set up S-Tunnel

Install Stunnel version 4.2. Stunnel-3.x should work fine too. Run command:

stunnel -c -d 2525 -r smtp.att.yahoo.com:smtps

This creates a SSL tunnel to Yahoo's mail server "smtp.att.yahoo.com". I used local port 2525 for tunneled SMTP traffic. Sendmail will need to connect to local 2525 port to relay emails to mail server.

Step 2: Configure Sendmail

Step 2.1: File authinfo

First of all, your sendmail distribution must have SASL complied in because Yahoo mail server requires sendmail to provide login information so it can authenticate to SMTP server. This is done by authinfo file. Since this file contains your ISP account username and password, so protect it by setting the correct privileges on this file. Do the following as root

mkdir /etc/mail/auth
chmod 700 /etc/mail/auth

Now create file /etc/mail/auth/authinfo, which is a simple text file. My authinfo file contains:

AuthInfo: "U:myname@sbcglobal.net" "I:myname@sbcglobal.net" "P:xxxxxx" "M:LOGIN PLAIN"

You may put multiple lines into this file, such as

AuthInfo:relayhost1 "U:myname@sbcglobal.net" "I:myname@sbcglobal.net" "P:xxxxxx" "M:LOGIN PLAIN"
AuthInfo:relayhost2 "U:myname@sbcglobal.net" "I:myname@sbcglobal.net" "P:xxxxxx" "M:LOGIN PLAIN"
AuthInfo: "U:myname@sbcglobal.net" "I:myname@sbcglobal.net" "P:xxxxxx" "M:LOGIN PLAIN"


I would like to point out several common mistakes when creating this file:

• The string after "Authinfo:" is the hostname of the relay server and the key that Sendmail searches for authentication information. It must be a real DNS name instead of CNAME. For example, smtp.att.yahoo.com won't work here since it is actually a CNAME of smtp.sbc.mail.yahoo4.akadns.net. You have to use smtp.sbc.mail.yahoo4.akadns.net here.
  
• You may notice that I didn't put anything after "Authinfo:" in my sample authinfo file. This is intentional. Sendmail ALWAYS use the line with "Authinfo:" if it can't find an exact hostname match. Since I have only one relay server, it is ok to use one set of username/password instead of dealing with all those DNS name changes.
  
• "U:" specifies authorization name and "I:" specifies authentication name. In most cases, they should be the same.
  

Now don't forget to run makemap to make it available to Sendmail.

makemap hash authinfo < authinfo

This command creates a file authinfo.db.

Step 2.2: Configure Sendmail

I am using Sendmail 8.14. You should use the latest sendmail release since newer releases address important security issues. With this method, no special compile-time directive is required. The configuration file for sendmail is very complex for beginners and you may want to grab the O'Reilly book to understand it better. The exercise we are doing here is very simple so don't be intimidated.

On FreeBSD, I usually modify 'myhostname.mc' and then do 'make cf', which creates myhostname.cf. Then I copy it to /etc/mail/sendmail.cf. On your system, you may have sendmail.mc and then compile it manually using m4.

m4 sendmail.mc > /etc/mail/sendmail.cf

In sendmail.mc, we add two new lines:

define(`SMART_HOST', `[127.0.0.1]')
FEATURE(`authinfo', `hash /etc/mail/auth/authinfo.db')

That are the ONLY two lines that you need (well, i lied, read on...) in sendmail.mc if you use sendmail as client to relay server.

Now if you restart sendmail and try it out, you are going to find the traffic is not going to the local port 2525. There are two issues:

1. We need to pass the port of relay host (2525) to Sendmail.
   
2. Sendmail default rulesets prevent you from using localhost for relay, which makes sense normally but not in our case.
   

So, I will modify sendmail.cf (NOTICE it is the sendmail configuration file instead of .mc file since I can't find M4 macros for it)as following:

$ diff -c sendmail.cf.old sendmail.cf
*** sendmail.cf.old Tue Aug 28 18:21:37 2007
--- sendmail.cf Tue Aug 28 18:23:33 2007
***************
*** 269,275 ****
O UseErrorsTo=False

# log level
! O LogLevel=9

# send to me too, even in an alias expansion?
#O MeToo=True
--- 269,275 ----
O UseErrorsTo=False

# log level
! O LogLevel=20

# send to me too, even in an alias expansion?
#O MeToo=True
***************
*** 961,967 ****
R< local : $* > $* $>CanonLocal < $1 > $2
R< $~[ : $+ @ $+ > $*<$*>$* $# $1 $@ $3 $: $2<@$3> use literal user
R< $~[ : $+ > $* $# $1 $@ $2 $: $3 try qualified mailer
! R< $=w > $* $@ $2 delete local host
R< $+ > $* $#relay $@ $1 $: $2 use unqualified mailer

###################################################################
--- 961,968 ----
R< local : $* > $* $>CanonLocal < $1 > $2
R< $~[ : $+ @ $+ > $*<$*>$* $# $1 $@ $3 $: $2<@$3> use literal user
R< $~[ : $+ > $* $# $1 $@ $2 $: $3 try qualified mailer
! ### XXX It is OK to use localhost as relay
! #R< $=w > $* $@ $2 delete local host
R< $+ > $* $#relay $@ $1 $: $2 use unqualified mailer

###################################################################
***************
*** 1835,1839 ****
A=TCP $h
Mrelay, P=[IPC], F=mDFMuXa8, S=EnvFromSMTP/HdrFromSMTP, R=MasqSMTP, E=\r\n, L=2040,
T=DNS/RFC822/SMTP,
! A=TCP $h

--- 1836,1840 ----
A=TCP $h
Mrelay, P=[IPC], F=mDFMuXa8, S=EnvFromSMTP/HdrFromSMTP, R=MasqSMTP, E=\r\n, L=2040,
T=DNS/RFC822/SMTP,
! A=TCP $h 2525


There are 3 changes:

• Change LogLevel to 20 so I can see some debug information in /var/log/maillog
  
• Remove the rule that prevents localhost to be relay
  
• add port 2525 to the parameter for 'relay' mail agent
  

Step 3: Test

Now we have everything set. Restart sendmail and send a test email. Inspect /var/log/maillog, look for lines

Aug 28 18:24:31 snake sm-mta[9883]: l7T1OVt8009881: SMTP outgoing connect on localhost
Aug 28 18:24:32 snake sm-mta[9883]: AUTH=client, relay=[127.0.0.1], mech=LOGIN, bits=0
Aug 28 18:24:33 snake sm-mta[9883]: l7T1OVt8009881: --- 050 ... Sent (ok 1188350673 qp 65294)
Aug 28 18:24:33 snake sm-mta[9883]: l7T1OVt8009881: to=, ctladdr= (1001/1001), delay=00:00:02, xdelay=00:00:02, mailer=relay, pri=30491, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (ok 1188350673 qp 65294)
Aug 28 18:24:33 snake sm-mta[9883]: l7T1OVt8009881: done; delay=00:00:02, ntries=1
Aug 28 18:24:33 snake sm-mta[9883]: NOQUEUE: --- 050 Closing connection to [127.0.0.1]


You can see sendmail connect to localhost (which is tunneled to ATT Yahoo mail server) and use AUTH mechanism 'LOGIN'. If you have a sendmail version without SASL you will see 530 Auth Required error.

Update (09/10/2008): Please make sure to check out this new post which deals with a new restriction that AT&T requires you to use a verified email address to send mails.

Some quirky points when deal with shell 'test' program

Or when you use if [ -n "string"]; then ... in shell.

> test -n string
returns 1 if string is not empty and 0 otherwise.

> test -n
returns 1

> test -n ""
returns 0

This brings up an important point, you need to quote the string in shell otherwise 'test' will think there is no argument for empty string and returns 1 (wrong result).

if [ -n "$my_string" ]; then

Wrestle with xargs

xargs '-I %' vs '-J %'

Normally xargs append the input arguments to the end of utility (plus fixed arguments). But sometimes you want put the input arguments in the middle or work around spaces inside arguments, then you may use '-I' or '-J'.

Before we get to that, let's review what xargs does under the hood.

xargs [-0opt] [-E eofstr] [-I replstr [-R replacements]] [-J replstr]
[-L number] [-n number [-x]] [-P maxjobs] [-s size]
[utility [argument ...]]

xargs breaks down the input into arguments delimited by spaces, tabs, newlines and EOF. It always does that. '-L' (based on number of line) and '-n' (based on how many arguments) controls when xargs should call the utility. xargs does NOT
do a second pass of shell parsing of the new arguments (Obviously it uses exec() facility).

In normal scenario, xargs put the extra arguments in the end. Each argument is considered as individual argument by the utility.

When using '-I %', any fixed argument which includes % is replaced by the new-arguments-string (Notice that all new arguments are concatenated together with a space between them), the replaced argument is considered as ONE argument
by the utility.

When using '-J %', any fixed argument which IS % (notice it is not match if it simply contains %) is EXPANDED with new arguments. Each new argument is considered as individual argument by the utility.

For example,

ls |xargs -L1 du -s

It won't work for filename with spaces (because xargs split them into separate arguments)

ls |xargs -L1 -I % du -s %

will work fine even for filename with spaces.

Some Makefile Tips

When write a non-trivial program, you may need to write some libraries and then applications. If you are using static linking, make sure that you specify dependencies clearly in the Makefile otherwise you are going to waste lots of time during debugging.

You modify the library code, then type 'make', everything seems fine. But
somehow your changes didn't go into the binary.

Normally you put the library code into a separate directory (which is a good practice) and the Makefile will produce a libx.a library file. Then, the application will use this library using '-L'. A common misktake is that Make doesn't know the target program depends on the libx.a file.

Order of Evaluation of Functional Parameters

This question comes from one interview. What is the output for the following code?

int i=1;
printf("%d %d %d %d\n", i++, i++, i++, i++);

You are likely to say "1 2 3 4". Wrong! The correct answer (I think) should be we should never write code like this because the behavior is dependent on the platform.

The C programming language standards specifically allow the parameters to be passed to a function to be evaluated in any convenient order. The SUN Sparc station compiler worked left to right, which seems more natural, whereas the on FreeBSD, gcc worked right to left which may be more efficient in some circumstances, whereas on Mac OS X, expressions are from left to right, normal variables are valuated the last.

Consider the following program to prove my point.

#include 
#include 

int f1() {
    printf("f1...\n");
    return 1;
}
int f2() {
    printf("f2...\n");
    return 2;
}
int add(int i, int j) {
   return i+j;
}
main() {
  int i =1;
  int j =1;
  printf("%d %d %d %d\n", i++, i++, i++, i++);
  printf("%d %d %d %d\n", j, j++, j++, j++);
  add(f1(), f2());
}

Run it on FreeBSD (compiled with GCC),

$./printftest 
4 3 2 1
4 3 2 1
f2...
f1...

Run it on Mac OS X Tiger (compiled with GCC),

$ ./printftest 
1 2 3 4
4 1 2 3
f1...
f2...