Setting up sendmail on a firewall, Part 1

Unix Insider –

One of the greatest features of sendmail is the extreme flexibility it provides the administrator. This is also one of its worst features, the reason being that it is so often misconfigured. Usually, a system administrator inherits a sendmail configuration file that he or she is afraid to touch, lest it should break. Over time, this file becomes hopelessly confusing, contradictory, and redundant.

I began writing this column with the intention of providing a simple, straightforward, "sendmail for dummies" approach. I ended up appreciating how Bryan Costales must have felt when he wrote his handbook, Sendmail for O'Reilly & Associates: Nothing is simple in sendmail. I also learned to appreciate just how powerful and flexible sendmail really is. It's worth learning. It's also worth a few columns. I'll devote this month's column to discussing some of the background and new features in sendmail 8.9.3 as well as how to build the source code. Next month we'll look into the configuration file. If there is enough interest, I will add a third column for special situations and testing techniques.

Sendmail on the firewall?

Like most security admins, I've always been told it's a bad idea to run sendmail on the firewall. It's generally considered better to run something like smap instead. Indeed, I ran smap myself until I discovered a problem I couldn't fix with smap. The problem was that company email was forwarded from an internal mail gateway system to the firewall. The firewall could only rewrite the external header to say the mail came from "company.com." There was still an internal header that showed the mail was routed through "mailgate.company.com," complete with the mailgate internal IP address. I couldn't have the inside machine masquerade as company.com because there were other internal mail gateways it had to communicate with. (They couldn't all be company.com.) But because sendmail is extremely flexible, open source software, it's possible to add functionality to strip out the inside header. Also, many of the security risks with sendmail have to do with the fact that it's generally run setuid to root. It isn't necessary to run it this way on a firewall because there are no direct mail users on the firewall. For added security, I run sendmail in a

<font face="Courier">chroot</font>
ed cell with the program mailer using the sendmail restricted shell,
<font face="Courier">smrsh</font>
.

Preliminaries

There are certain resources you should have available before you start working with sendmail. This column isn't going to attempt to cover all aspects of sendmail. For that, I direct you to Bryan Costales's Sendmail, Second Edition. Aside from being an excellent resource, at more than 1,000 pages this book can also double as a child's booster seat!

You'll also need the latest copy of the sendmail source code (sendmail 8.9.3 as of this writing), available from Sendmail.org. Verify the PGP signature! Some very useful links are included with the latest information.

Although it should go without saying, you need to be able to build the code using

<font face="Courier">make</font>
and
<font face="Courier">gcc</font>
or
<font face="Courier">cc</font>
and have your environment variables set appropriately. It is strongly recommended that you use a system other than the firewall to compile and test the code. Remember that any system you use has to be running the same release of the operating system and have a compatible architecture. The firewall shouldn't have a development environment installed. If for some reason you have to use the firewall to compile, at least keep the development environment on a spare drive that you can unmount when you don't need it. Life with sendmail is much easier if you have the GNU utilities already installed because it is assumed in the makefile that
<font face="Courier">groff</font>
is installed. You can get the GNU utilities (
<font face="Courier">gcc, gunzip, groff,</font>
to name a few) from sunsite.unc.edu.

For the purposes of verifying the source (and not just for sendmail), you should have a copy of PGP (Pretty Good Privacy) installed (version 2.6.3 or higher). You can download a personal copy or buy one from Network Associates.

New features in sendmail 8.9.3

Sendmail 8.9x contains several new features that will probably cause your mail to break unless you configure them properly. These features mostly help filter spam and prevent your site from being used to relay someone else's mail. You have to decide how to configure them based on your site's policy. For my own network, I can rather tightly define from whom I'll accept mail. However, an ISP cannot be so selective. Please note that I haven't personally configured mail for an ISP, so I can only speculate on the special concerns of ISPs. Because ISPs have to service the general public, I would imagine they have legal issues to consider before blocking mail from a particular site.

Relaying is denied by default in sendmail 8.9.x. While this might sound desirable, this feature will probably cause your mail to stop working because it will refuse to relay mail from your own internal systems. There are several ways to fix this. The easiest is to use the feature,

<font face="Courier">relay_entire_domain</font>
. Refer to Sendmail.org for a full description of these new features.

Another major change to sendmail is that it is now picky about the permissions on the directories it uses. You can turn this off by using the DontBlameSendmail option. If you do, don't blame sendmail if you have security problems. I recommend fixing the directory permissions.

Default

Using the default options, you can simply create a file called

<font face="Courier">/etc/mail/relay-domains</font>
that contains the names of the systems for which you want to accept relaying. For example, a domain known as "company.com" and also known as "othername.com" would have the following:

<font face="Courier">
company.com
othername.com
</font>
  • Advantage: If you only need to relay mail from a few systems, this is probably the simplest solution and the one I recommend for most installations. It prevents you from acting as a relayer, while allowing your mail to go through.

  • Disadvantage: You have to keep the file updated, and you may have other requirements. If you're an ISP, you may have to allow relaying from quite a large number of domains. Also, every time you update this file you have to restart sendmail (
    <font face="Courier">kill -1 [sendmail_pid]</font>
    ).

promiscuous_relay

This feature allows relaying from anywhere (not recommended).

  • Advantage: You don't have to worry about mail being denied.

  • Disadvantage: Anyone can use your system as a mail relay. Use of this feature defeats the antirelay features of sendmail and is definitely not recommended.

relay_entire_domain

This feature allows any host within your domain as defined by the

<font face="Courier">m</font>
class (
<font face="Courier">$=m</font>
) to relay through your system. By default, this would be
<font face="Courier">*.company.com</font>
.

  • Advantage: You don't have to worry about mail within your domain being denied. Systems outside your domain can't use your system to relay mail unless they're defined in the relay-domains file or in the access database. If you only want to relay hosts in your own domain, you can use this instead of the file

    <font face="Courier">/etc/mail/relay-domains</font>
    .

  • Disadvantage: You may not want to allow other organizations in your company to route mail through your system. Also, if you're known by several domain names (popular with companies seeking product recognition), this won't buy you much.

relay_hosts_only

This allows relaying based on the individual host name, not the domain name.

  • Advantage: Fine-tunes the permission for relaying by requiring a fully qualified host name rather than just the domain name.

  • Disadvantage: Requires you to specify in either the file
    <font face="Courier">/etc/mail/relay-domains</font>
    or the access database the host name of the system you're permitting to relay. For example, if I use this, my
    <font face="Courier">/etc/mail/relay-domains</font>
    file would have:
    <font face="Courier">
    company.com<BR>
    mailgate.company.com<BR>
    othername.com<BR>
    mailgate.othername.com 
    </font>

Mail blocking

Here are the basics of sendmail's antispam features.

accept_unresolvable_domains

By default, if the sender's domain cannot be resolved in DNS, the mail is rejected. For example:

<font face="Courier">
MAIL FROM: <wkeys@nonexistent> 501 <wkeys@nonexistent>... Sender
domain must exist
</font>

Using

<font face="Courier">accept_unresolvable_domains</font>
overrides this and accepts mail from any domain or IP address.

  • Advantage: You may have to accept mail from IP addresses if the senders don't have their headers rewritten to come from a registered domain.

  • Disadvantage: You can be spammed from anywhere. You may be better off listing known sites with this problem in the access database.

access_db

To use the access database feature, your system must support at least one map type such as NDBM (standard on most commercial systems such as Solaris) or the Berkeley database (Berkeley DB) 2.X. If you want to use Berkeley DB, you can obtain it from Sleepycat Software. If you install the Berkeley version, make sure you build sendmail with the

<font face="Courier">NEWDB</font>
flag (and include the Berkeley DB libraries and include files).

The

<font face="Courier">access_db</font>
feature causes sendmail to look in a database map file (by default
<font face="Courier">/etc/mail/access.db</font>
) and decide whether to accept or reject mail from a particular user or site. You can even send a custom error message. This feature can also be used to control relaying permissions.

  • Advantage: Really allows you to fine-tune who you will accept mail from. For example, I may not want to accept mail from domains that can't be resolved, but I want to make an exception for a particular domain or address. If yours is a large site, you may want the greater flexibility this can give you -- it's rather like a firewall rule base.

  • Disadvantage: You have to build the database file and keep it updated. It can become rather complex -- it's rather like a firewall rule base. The more complex you make it, the harder it is to maintain.

accept_unqualified_senders

By default, if the sender's address isn't fully qualified, sendmail will refuse the connection. For example:

<font face="Courier">
mail from: <wkeys>
553 <wkeys>... Domain name required
</font>

Use of this feature overrides the default so the connection will be accepted.

  • Advantage: I recommend always using fully qualified addresses. However, on an internal mail gateway, you may not be able to control how the other local systems send you mail, and this will allow you to accept mail with unqualified sender addresses.

  • Disadvantage: You lose some ability to track where mail is coming from. Don't use this on a firewall.

blacklist_recipients

This allows you to block incoming mail to accounts you don't want to receive mail by listing the account in the access database. The mail appears to be accepted but is actually dumped to

<font face="Courier">/dev/null</font>
.

  • Advantage: You may want to use this for "nobody" or "guest" accounts.

  • Disadvantage: You have to set up the access database. This isn't much of a disadvantage, but it is another step.

rbl

If spam is a major problem for your site, you'll be interested in the Realtime Blackhole List. A list of known spam hosts is maintained at rbl.maps.vix.com. The

<font face="Courier">rbl</font>
feature causes sendmail to check with rbl.maps.vix.com (or another RBL server if you specify) and blocks mail if the host is on that list.

  • Advantage: You don't have to worry about maintaining your own list of spam hosts.

  • Disadvantage: You're trusting that the RBL server is accurate. This may also cause a delay in accepting a connection while the RBL server is contacted.

relay_based_on_MX

If a host has an MX record that points to your site, this feature accepts and relays mail for them.

  • Advantage: You don't have to add hosts to the access database if they have an MX record that points to you.

  • Disadvantage: This seems like a lazy way out. It's probably quicker to look up systems in the access database than check for MX records. This will allow a third party to relay through you without your permission.

relay_local_from

This allows your users to relay mail though you if the domain portion of the sender address is a local host.

  • Advantage: If you have users that belong in your domain but are routed through another domain, this will allow their mail to be relayed.

  • Disadvantage: It's very easy to fake the sender address, and using this opens you up to spammers. I don't recommend it.

Building the sendmail binary

The first step of course is to download

<font face="Courier">sendmail.8.9.3.tar.gz</font>
and
<font face="Courier">sendmail.8.9.3.tar.sig</font>
from Sendmail.org.

<font face="Courier">
cp sendmail.8.9.3.tar.sig /usr/local/src [or wherever you keep your source]
cp  sendmail.8.9.3.tar.gz   /usr/local/src  	
cd   /usr/local/src
gunzip    sendmail.8.9.3.tar.gz    
</font>

You'll want to download the sig file so you can verify the validity of the tar file. Downlaod the PGPKEYS file from Sendmail.org in directory

<font face="Courier">/pub/sendmail</font>
. This file contains the signature keys for Greg Shapiro and Eric Allman of Sendmail.org as well as the sendmail signing keys for 1997, 1998, and 1999. You must have the 1999 key for sendmail 8.9.3. You may want to include the older keys anyway if you're installing older versions of sendmail. PGP doesn't behave the same way in all versions. I used PGP Business 4.0.1 for Unix and discovered it doesn't like to have all the keys in the same file. It only wanted to install Greg Shapiro's key. I put each PGP key block into a separate file and then added them one at a time. You will want to certify the keys and check the fingerprints in the PGPKEYS file for each key against what the PGP software thinks the fingerprint is. If you're unfamiliar with PGP, you should refer to the documentation provided with your version of PGP. For Business 4.0.1, you would type the following command to add the key:

<font face="Courier">
	pgp -ka /tmp/key1
</font>

<font face="Courier">/tmp/key1</font>
is the name of the first key to add. Once you add the keys to your public key ring, you're ready to check sendmail's key. Type:

<font face="Courier">
	pgp sendmail.8.9.3.tar.sig
</font>

You must have

<font face="Courier">sendmail.8.9.3.tar</font>
in the same directory as the signature. PGP will verify the signature. If it's okay, it will tell you that the key used to sign the software is Sendmail Signing Key/1999. Once you have completed this step, you're ready to untar the software and begin installation.

<font face="Courier">
	tar xvf sendmail.8.9.3.tar
</font>

This will unload the tar file and create a top-level directory,

<font face="Courier">/usr/local/src/sendmail-8.9.3</font>
, which has everything you need to build and configure sendmail. You should review the README files for more information. In most cases, you can just run the build command without any configuration changes. Note that, by default, sendmail is compiled with DNS support. If you're not using DNS, sendmail will run very slowly until the DNS lookup times out. See the README file in the
<font face="Courier">src</font>
directory for configuring compile-time options. To use the defaults, type:

<font face="Courier">
cd  /usr/local/src/sendmail-8.9.3/src
./Build  -c
</font>

The

<font face="Courier">-c</font>
option to build ensures a clean compile from scratch. Alternatively, you can run make from the top-level directory (
<font face="Courier">/usr/local/src/sendmail-8.9.3</font>
) to build all the utilities provided with the release. It isn't necessary if you just want the sendmail binary, but you may want to use some of the included utilities such as makemap (for building an access database). Once the build has completed successfully, the binary will be in
<font face="Courier">/usr/local/src/sendmail-8.9.3/src/obj.[ostype].[release].[arch],
(obj.SunOS.5.6.sun4)</font>
.

Next month

This concludes the first installment of this series. At this point, you should go through the sendmail source directories and review the README files. Decide which features support your site's policy. Next month, we'll continue building the sendmail config file.

Acknowledgments

Many thanks to Greg Shapiro at Sendmail.org for his prompt and patient explanations. I'm beginning to wonder if he ever sleeps. Also thanks to my partner at Wizard's Keys, Jonathan Klein, for technical input and for running out to get wine when I got frustrated!

Disclaimer: The information and software in this article are provided as-is and should be used with caution. Each environment is unique and the reader is cautioned to investigate with his or her company as to the feasibility of using the information and software in the article. No warranties, implied or actual, are granted for any use of the information and software in this article and neither author nor publisher is responsible for any damages, either consequential or incidental, with respect to use of the information and software contained herein.

What’s wrong? The new clean desk test
Join the discussion
Be the first to comment on this article. Our Commenting Policies