[Comp.Sci.Dept, Utrecht] Note from archiver<at>cs.uu.nl: This page is part of a big collection of Usenet postings, archived here for your convenience. For matters concerning the content of this page, please contact its author(s); use the source, if all else fails. For matters concerning the archive as a whole, please refer to the archive description or contact the archiver.

Subject: NEW! UNIX Email Software Survey FAQ [Part 1 of 3]

This article was archived around: 9 Aug 2002 05:00:02 GMT

All FAQs in Directory: mail/setup/unix
All FAQs posted in: news.admin.misc, comp.mail.misc
Source: Usenet Version

Archive-name: mail/setup/unix/part1 Last-modified: Mon Feb 21 09:57:01 EST 2000
UNIX EMail Software - a Survey Chris Lewis clewis@ferret.ocunix.on.ca [and a host of others - thanks] Copyright 1991-1998, Chris Lewis Redistribution for profit, or in altered content/format prohibited without permission of the author. Redistribution via printed book or CDROM expressly prohibited without consent of the author. Any other redistribution must include this copyright notice and attribution. Note to the patient readers who noticed that nothing has changed in this FAQ since 1996... Email systems have changed radically over the past few years, and I'm beginning the daunting task of bringing this FAQ into the new world. I'm planning a lot of changes: - Adding POP/IMAP discussions, and common implementations - Extensive coverage of anti-spam measures resources, and packages. - Updating recommendations to include things like the phase out of UUCP, predominance of POP/SMTP/MIME etc., S/Mime, PGP etc. - Other suggestions? I've started off the ball by mostly changes in the second and third parts: - updated sendmail - dropped IDA sendmail references. - dropped EASE references - begun the deprecation of obsolete solutions (UUCP, UUCP maps etc) - added exim. - added qmail. | - updated MMDF It would help a lot if anyone wanting to add a section on their favourite email topic (UNIX please!) could write it and send a copy to me. I'll also be dredging through my archives to find previous comments that haven't yet been added. Changes are marked with a preceding "|". You can skip to them by typing g^| in (most) newsreaders. Note: this FAQ has been formatted as a digest. Many newsreaders can skip to each of the major subsections by pressing ^G. Please direct comments or questions to mailfaq@ferret.ocunix.on.ca - note Reply-to: line - automatic if you reply to this article. Many changes made in the second and third parts. ------------------------------ Subject: Introduction Configuring electronic mail systems can be quite a complicated subject. Often far more complicated than, say, setting up a Usenet news feed. This is because, unlike news, email is expected to traverse multiple types of networks using their own protocol, whereas, Usenet news tends to be a single protocol supported by hook or by crook on different networks. This document is intended for system administrators who need to know how to set up their UNIX systems for email communication with the outside world. It is intended for the email-naive SA who gets more than a little confused by the acronyms, RFC's and plethora of software. This is intended to be a general survey of the software available, so I won't spend too much time on some of the details. Most of the available software comes with documentation that can explain things much better than I can. Additional detail can be obtained from several sources, such as: Quarterman, John S.: "The Matrix -- Computer Networks and Conferencing Systems Worldwide", Digital Press 1990, (Order No. EY-C176E-DP), ISBN 1-55558-033-5. Adams, Rick and Frey, Donnalyn: !%@:: A Directory of Mail Addressing and Networks, 3rd Ed., O'Reilly & Associates 1993, Provides a good reference for people seeking information on how to access the various email networks. ISBN 1-56592-031-7. Kehoe, Brendan P.: Zen and the Art of the Internet: A Beginner's Guide, Second Edition, Prentice Hall 1992, ISBN 0-13-010778-6. Edition 1 is available via FTP on cs.widener.edu in the tar file zen-1.0.tar.Z. [I think] Krol, Ed: The Whole Internet: User's Guide & Catalog. First edition, O'Reilly & Associates Sept. 1992. ISBN: 1-56592-025-2. Very good introduction to the Internet, history, facilities, uses, services, etc. I learned a lot. Albitz, Paul & Liu, Cricket: DNS and BIND, First edition, O'Reilly & Associates, October 1992. ISBN: 0-56592-010-4. Describes in great detail everything from what a domain is, to how to install and configure BIND. A *MUST* for people setting up large networks, or connecting machines to the Internet. It has become mandatory reading for network administrators in a large corporation for good reason. Costales, Bryan and Allman, Eric and Rickert, Neil: Sendmail. O'Reilly & Associates, Nov (?) 1993. ISBN 1-56592-056-2 (ISBN from galley proof, which I've had a preview of). An absolute necessity for anyone diving into the configuration of sendmail. The material is presented in a very clear form, and is quite exhaustive in its coverage. Perhaps a bit too wordy and overlong, but that's a more than welcome contrast to previous documentation (or lack thereof) on sendmail. Further, this is primarily oriented towards UNIX email systems. This is unfortunate, because it would be nice to have a general document covering email in all of its forms. However, each operating system tends to have radically different email mechanisms, so it would be difficult to do justice to any other environment. It seems more useful to cover one environment well here, and have companion documents for other environments. Speaking of which, why hasn't anybody else stepped in to do FAQs on other environments? Like DOS, Mac etc. And finally, this document is not intended to be pedantically correct. Knowledgeable readers will know that I'm glossing over a lot of detail, and absolute precision has been balanced against readability and effectiveness in helping people get going. ------------------------------ Subject: Layout This FAQ is laid out in the following sections: + An overview of how mail systems go together. + A glossary of the important terms to know. + A list of general do's and don'ts of mail systems. + Configuration Issues + Several suggested mail configurations. + General overviews of specific software. ------------------------------ Subject: Electronic mail - A General Overview of Structure Electronic mail generally consists of three basic pieces: 1) The link level transport - which could be UUCP, TCP/IP, or a host of others. We'll call this the "transport medium" (TM) 2) the "Mail Transport Agent" (MTA) which is responsible for transporting mail from source to destination, possibly transforming protocols, addresses, and routing the mail. The MTA often has several components: - Routing mechanisms - Local delivery agent (LDA) - Remote delivery agent Many MTA's have all of these components, but some do not. In other cases, it is possible to replace certain components for increased functionality. 3) The "User Agent" (UA) is the user interface - the software that the user uses to read his mail, sort things around in folders, and send mail. Sometimes called "Mail User Agent" (MUA). ------------------------------ Subject: Glossary Rather than alphabetic, this glossary tends to group terms referring to similar functionality together. Transport Medium: UUCP (Unix to Unix Copy Program): Back in the mists of time, UNIX systems communicated only over RS232 serial lines, usually over modems. UUCP is a suite of programs developed back in the early 70's to provide this communications link. All that UUCP does is transfer files from one system to another. There is an additional mechanism where one system can direct the destination system to run a file through a specific program. Electronic mail in UUCP is simply requesting the destination machine to run "mail" on a data file. UUCP communicates by means of "protocols", the most common being "g", a method for transmission of data over telephone lines and ensuring that the data is not corrupted. There are several other protocols, none universally available, and most oriented towards communication media other than telephone voice lines (such as dialup X.25, PAD X.25, or LAN connects). UUCP operates over fixed system-to-system links, so sending mail from one system to another often has to traverse other intermediate systems. | If you like source, Taylor UUCP is an excellent full-featured | implementation of UUCP, with many enhancements to deal with higher | modem speeds. It is FreeWARE. | UUCP mail protocols (bang paths) are now being deprecated, because | DNS and MX etc., are making it wholly unnecessary. TCP/IP (Transmission Control Protocol/Internet Protocol): TCP/IP is a protocol that allows any system on a network to talk "directly" to any other, by passing packets of information back and forth. TCP/IP (and its later relative OSI) is usually used over networks built on top of Ethernet, Token-Ring, Starlan and other LANS. SMTP: Or, "Simple Mail Transfer Protocol", is the communications protocol used most commonly over TCP/IP links in UNIX environments for mail. SMTP usually operates directly between the source and destination machines, so intermediate machines don't get involved (except for gateways, see below). SMTP is usually part of the MTA. SLIP (Serial Line Internet Protocol): SLIP is an implementation of TCP/IP designed for use over RS232 serial lines (ie: modems). The other difference is that some SLIP implementations have the ability to "dial the phone" to make a connection for a specific transfer, whereas LAN TCP/IP is physically continuously connected. You'd also need TCP/IP to run a SMTP mail connection. PPP (Point-to-Point Protocol): A successor to SLIP. X.25/X.29: X.25 is a packet switched data network which is usually half-duplex. In this context, it's really an alternative to dialup over voice telephone lines with modems. X.25 is available in several "flavours", either direct X.25 trunk connects over leased lines, through "PAD" interfaces, or by ordinary dialup modem access to X.25 "ports". To be useable in the context of mail transfers, you also have to use a file transfer protocol/mechanism of some sort on top of X.25. The most common being UUCP "f" protocol (through PADS or dialup), or "x" with direct X.25 connects. Whether you use X.25 or phones plus modems depends on a number of factors - usually the determining factor is cost. In North America, high speed modems (eg: 9600 baud and above) over telephone lines tends to be less expensive. However, Europe's really wierd phone system structure usually makes X.25 more cost-effective, and therefore, X.25 use in UNIX mail systems is much more common in Europe than North America. X.29 is the command set used to configure and establish X.25 connections when you're using asynchronous connections to a PAD. Networks: Internet: An "internet" is a network comprised of computers that talk to each other using TCP/IP, and usually SMTP for mail. The "Internet" is a vast network of hundreds of thousands of machines using SMTP protocol mail, communicating with each other over relatively high speed lines. But not all "internets" are connected to *the* Internet. The Internet grew out of a US government funded project in inter-computer communications that grew into an enormous network of systems. | One of the principal characteristics of this network is that machines are addressed by domain names which identify the destination, rather than addresses that are constructed out of the route from machine-to-machine-to-machine. UUCP Network: The UUCP network is that set of machines that talk to each other via UUCP. Sending mail through this network requires that the sender know the network topology of UUCP links, and specify a path from one machine to the next. (There are, of course, ways around this. See the section on "do's and don'ts".) Mail addresses: Addresses: An email address is a method of specifying a given person on a specific machine. There are scads of conventions, usually determined by the presence of "@"'s, "!"'s and other special characters in the name. An address usually consists of two parts: a userid/name and a machine specification. A Domain address usually looks like: userid@domain-address Whereas a UUCP address usually looks like: siteA!siteB!siteC!userid Domain Addresses: Domains are a way of uniquely specifying a destination. Much like a postal address, a domain specifies a set of progressively more restrictive "domains" of the potential address space. It would perhaps be illustrative to give an example: clewis@ferret.marketing.fooinc.com You read these things right to left: "com" means the commercial domain. "fooinc" is the name of an organization within the commercial domain. "Marketing" is the name of a suborganization within fooinc, and ferret gives the name of a machine (usually). Domains can have any number of levels. The top level domain (com in the above example) has many possible values. In the United States, "com", "mil", "edu", and "gov" are fairly standard. Elsewhere, the top level domain tends to be a country code, the second level tends to be a province or state, OR a classification like "edu" or "ac" for academic (such as ac.jp, go.jp, ac.uk, edu.au, etc) and the third an organization. But, for example, there are many .com and .edu sites in Canada and other countries. FQDN A fully-qualified-domain-name (FQDN) has a entry for each level of the domain, from individual machine to top-level domain. In many cases, an organization has implemented an organizational "gateway" at a higher level of domain, so that people from outside don't have to specify FQDN's to get to a specific person. In the above example, for instance, "fooinc.com" may be sufficient to get to anyone inside fooinc, and "ferret.marketing" may not be necessary. On the other hand, people sometimes leave out the higher levels of the address, as in "ferret.marketing". This is a bad idea - because if the mail is cc'd out of the organization, chances are the external recipient cannot reply, because "ferret.marketing" is incomplete. So use addresses that are specified sufficiently for external users to use. (fooinc.com if a organizational gateway is used, the whole ferret.marketing.fooinc.com if not) NIC Internet TOP-LEVEL domains (edu, com, gov, mil) are controlled by a single organization, the NIC (internic.net). An organization "gets a piece" of the namespace by registering with the NIC, and then they are free to administer their own namespace (everything under fooinc.com) as they choose. The same is true for foreign countries; Once they have their top-level domain (usually the two-letter ISO country code) registered with the NIC, they do the rest, and divide it as they see fit. In contrast, on UUCPnet, all machine names everywhere share a single flat namespace. So it is important to choose a name that has not been used before. (See do's and don'ts). This is why FQDN's help. We can tell the difference between ferret.fooinc.com and ferret.blah.edu by their full names. (Instead of UUCP paths which may turn out to be wrong, and autorouting will probably send the mail to the wrong machine) MX record: A non-SMTP/Internet site that wishes to register on the Internet will need to get a "nearby" Internet site to set up a MX record for them. An MX record is essentially a domain-server database record that (effectively) registers your domain name on the Internet, and indicates that the Internet site knows how to forward mail to you. Usually via some non-SMTP/Internet route, such as UUCP. You can get an MX record for one site, or a "wildcard" MX record so that you can have your own subdomains. Bang-Paths: With UUCP mail, the MTA has to specify a route to get from one machine to another. "A!B!C!userid" means go to machine A, then B, then C, then user "userid" on C. You should strive, however, for a MUA that allows you to use domain addressing, and let the MTA figure out the bang routing as appropriate. Miscellaneous: Gateways: There are several meanings of this term, only three are relevant here. The first is a mechanism for getting from one network to another network that uses different protocols. The second is a mechanism for getting from one logical (often organizational) network to another using the same protocol. Often for example, there will be a LAN in one department of an organization, and one machine in the LAN has the connection to another LAN in another department. This means that mail from one LAN to the other has to pass thru the gateway machine. Another form, which we'll mention later is that of mail to news gatewaying. Routers: There are several definitions, but the most important is that part of the TA that figures out how to send a message to a given machine. This often uses a database that provides routes from one machine to the other machines on the network. Smarthost: In many cases, your machine won't know how to get to a specific destination. You can usually set up your mail system to send mail, that it doesn't know how to deliver, to a machine that is more likely to. RFC's: A set of documents that include formal descriptions of mail formats used on the Internet, and are adhered to by many non-Internet systems. More specifically, in the "worldnet" of Usenet, Internet and UUCP, the RFC's set the standards for mail exchange. RFC822, 1123 and 976 are the most important for Internet/UUCP mail. It should be pointed out, however, that there are some regions where the RFC's are not entirely respected. For example, the British academic email networks (JANET) uses domains, but they're specified backwards (they drive on the wrong side of the road too ;-). MIME: Mime is the official proposed standard format for multimedia Internet mail encapsulated inside standard Internet RFC 822 messages. Facilities include sending multiple objects in a single message, character sets other than US-Ascii, multi-font text messages, non-textual material such as images and audio fragments, and other extensions. For an overview of Mime, see ftp.uu.net:networking/mail/metamail/MIME-overview.txt.Z. The defining document is Internet RFC 1341: N Borenstein & N Freed, ``Mime (Multipurpose Internet Mail Extensions) mechanisms for specifying and describing the format of Internet message bodies'' (June 1992). Also see RFC 1344: N Borenstein, ``Implications of Mime for Internet mail gateways'' (June 1992). RFC1341 and 1342 have since been superceded by RFC 1521 and 1522. Mime covers only message bodies, not message headers; to see how to represent non-Ascii characters in message headers, see Internet RFC 1342: K Moore, ``Representation of non-Ascii text in Internet message headers'' (June 1992). X.400: A CCITT standard for email formats, more or less an alternative to RFC 822/976/1123. This format will probably start taking over from RFC 822/976/1123 mail. It is likely to (already has?) become an ISO/IEEE standard along with OSI etc. "The Maps": A set of files describing machine-to-machine links distributed over Usenet in the group comp.mail.maps. These are usually posted on a monthly schedule, and can be automatically received and transformed into a routing database that describes the "optimal" route to each machine. These are operated by the "UUCP Mapping Project". See the README posted along with the maps for more details. Aliases: Aliases are a mechanism by which you can specify the destination for mail on your machine. Through the use of aliases you can redirect mail to "virtual userids". For example, you should have a mail destination on your machine called "postmaster", which is aliased to send the mail to the System Administrator (ie: you probably). Aliasing often also permits you to send mail to groups of users (not necessarily on the same machine as you) pipelines of commands or to specific files. Mailing lists: Are similar to Usenet newsgroups. They are usually aliases pointing to groups of users, and allow mail to be sent to the whole group at once. Mailing lists are set up to carry certain subjects. The difference between a mailing list and a Usenet newsgroup is that the messages are sent by mail, probably as a copy to each recipient, rather than broadcast. ------------------------------ Subject: Do's and Don'ts: 1) Register a domain name. Even on UUCP, where <machine>.UUCP is often used as a kludge, it is MUCH preferred that you obtain a real domain address. If you are directly connecting to the Internet, you will get one as part of your registration with the NIC. If you aren't connecting directly to the Internet, obtaining a registration will usually require you finding a nearby friendly Internet site willing to act as a mail forwarder to you from the Internet - the site that will set up a "MX record" for you. Many sites will do this for you for free, and several of the commercial email services (eg: uunet) will do it for you for a nominal charge (without requiring you buy the rest of their services). There are occasions where you can join what is called a "domain park". These are most often small regional groups of systems that have gotten one of their number properly registered as a domain, and provides forwarding services out to other systems. For example, in my address "ferret.ocunix.on.ca", "ocunix.on.ca" is a domain park made up of the Ottawa-Carleton UNIX User's Group, one of the other machines in the group provides a gateway between our systems and the Internet. 2) If your machine is going to "speak" UUCP to the outside world, choose a unique UUCP name. You can find out whether a name you want is taken by consulting the UUCP maps. Or by asking someone else who's using them. 3) Register your machine with the UUCP Mapping Project if you're going to use UUCP. Information on how to do this is included in the monthly maps postings in the file "README". This is usually only required when your machine talks UUCP to the outside world, or when other machines have to address you by your UUCP name. If you don't do this, somone else may choose the same name, and gross confusion will arise when smart routers won't be able to tell whether to send a piece of mail to you, or your doppelganger[s]. If you register with the UUCP Mapping Project, you have prior use, and people who choose the same name afterwards will be told to get a new one. If you're "behind" an organizational gateway, don't do this. (Your organizational gateway is the thing that needs to be registered) If you do fill in a map, please take the time to fill it in carefully, giving contact people and phone numbers. Just in case your machine goes crazy and starts doing something nasty. Note expecially the latitude and longitude. Get it right, or omit it. Brian Reid gets really annoyed with sites that are half a world away from where they really are. 4) If you're going to be setting up multiple machines, have only one or two connections to the outside world. 5) Install a mail system that understands domain addressing, even if you aren't registered. (In fact, all of the suggested configurations in this FAQ do) 6) *Never* use UUCP bang-routing with the MUA if you can possibly avoid it - each of the suggested mail configurations provide mechanisms where you, the user, do not have to specify routes to the MUA - you can specify domains, and the TA will do the routing (possibly bang-routing) for you. Important: many mailers that understand UUCP attempt to be pedantically "UUCPish" in the construction of headers, such as generating "bang routes" in From:/To: etc. lines. Which, given that the whole "mail network" is generally converging on more Internet-like standards, and that even UUCP sites are using fully domain-capable mailers, is a big mistake. RFC976 attempts to codify a "meta standard" that allows the coexistance of RFC822 (Internet mail) with UUCP-based networks. What this means is, essentially, that headers are formed in the SMTP form, even if the transport will be via UUCP. Unfortunately, however, many mailers insist on "UUCP-izing" perfectly useable Internet/domain headers. "Fixing" them to prevent this is sometimes difficult. Sendmail is almost always a problem in this regard. 7) Find a friendly neighboring SA to help. A SA who has already operating mail in your area will help smooth over the regional "gotchas" that are bound to crop-up. And advise you on the right software to use, where to obtain it, and how to install it. 8) Do NOT use "any old" Map unpacking program. Most available map unpacking programs automatically run the shell (or shar) to unpack map articles. Since it is trivially easy to forge map articles, using this type of unpacking program can easily let very destructive trojan horse or virus programs into your machine. The two specific map unpackers described in this FAQ are known to be secure from such attacks. Do not run any other unpacker unless you are aware of the issues and can inspect the code for such vulnerabilities. [If you know of other "secure" map unpackers that are generally available, please let me know] 9) If the people on your site, or small network, receive mailing lists, it's often a good idea to gateway them to news: Netnews often performs many of the same services as email. The primary difference is that messages are centrally stored, rather than delivered to individual's mailboxes, and that distribution looks more like a broadcast then a set of point-to-point communications. This means usually means that news can handle more volume, more efficiently, then email can. Because of the differences (and also the similarities) people often want to tie news and mail together. This is known as "gatewaying." For example, a small software development site might subscribe to the X Windows mailing list. Rather than have (say) eight copies of each mail message sent to their host, they would rather have it stored as a local newsgroup that everyone in the company can read, and which can be centrally archived. This is a typical use of a "mail to news" gateway. When a user makes a posting to this local group the article should be sent back out to the mailing list; this is a typical use of a "news to mail" gateway. On a larger scale, the "inet" groups are bi-directional gateways of Internet mailing lists. Within mainstream Usenet, many popular groups such as comp.windows.x, comp.protocols.tcp-ip, comp.unix.wizards, and so on, are gatewayed to mailing lists and back. Many subtle issues often come up when gatewaying mail and news, so unless you are experienced you should use one of the already-available packages for your local organization. For example, you probably do not want to write a brand-new Perl script and create a new "inet" newsgroup. The C News distribution includes some basic gateway tools in the contrib/nntpmail directory. Many people use Rich $alz's "newsgate" package that appeared in comp.sources.unix Volume 24; it includes discussion of some of the more subtle issues that come up. Before starting a mailing list gateway, apart from the technical aspect of the job you should also be aware of one important point: mailing-lists are considered private, whereas newsgroups are public. One can know who gets a list, but not who reads the group. It is always wise to get the authorization of the mailing-list manager and of the readers before creating a mail/news gateway. 10) If you're connecting to the Internet, or are setting up a large local internet, you really should get a copy of the DNS and BIND book mentioned in the bibliography.