From List-Managers-Owner Tue Feb 23 16:01:27 1993 Return-Path: Received: by mycroft.GreatCircle.COM (4.1/SMI-4.1/Brent-921015) id AA14263; Tue, 23 Feb 93 16:01:27 GMT Received: from relay1.UU.NET by mycroft.GreatCircle.COM (4.1/SMI-4.1/Brent-921015) id AA13811; Tue, 23 Feb 93 04:59:54 PST Received: from uunet.uu.net (via LOCALHOST.UU.NET) by relay1.UU.NET with SMTP (5.61/UUNET-internet-primary) id AA19436; Tue, 23 Feb 93 05:51:55 -0500 Received: from eiffel.UUCP by uunet.uu.net with UUCP/RMAIL (queueing-rmail) id 055040.13844; Tue, 23 Feb 1993 05:50:40 EST Received: from lyon.eiffel.com by glasgow.eiffel.com (4.1/SMI-4.0) id AA28803; Mon, 22 Feb 93 17:31:12 PST Received: by lyon.eiffel.com (5.61/1.34) id AA19008; Mon, 22 Feb 93 17:33:52 -0800 To: list-managers@GreatCircle.COM Subject: Mailing list software review X-Mailer: MH [version 6.7.2] Organization: Interactive Software Engineering, Santa Barbara CA Date: Mon, 22 Feb 93 17:33:51 PST Message-Id: <19006.730431231@lyon.eiffel.com> From: Raphael Manfredi Sender: List-Managers-Owner@GreatCircle.COM INTRODUCTION This article presents a comparison of three mailing-list software, majordomo, procmail and listserver. Only those parts of the software related to mailing list management and their satellites (archival, digests, list maintenance, etc...). We'll start with a quick introduction to those three packages, then we'll draw a feature comparison table for quick reference, and we'll conclude with a discussion, which will deal with specific aspects like bounce control or possibilities of extension. I have tried to keep this presentation as technical as possible, and I made all the possible efforts to ensure its accuracy. However, those packages are maintained and their code evolves. Some new features might be added in the near future, so this comparison is only a snapshot of the current releases. I apologize in advance for any mistake which could have slipped through and I take full responsibility for them. PACKAGES OVERVIEW --------------------------------------------------------------------------- Name: Majordomo Version: 1.46 Author: Brent Chapman Retrieved-From: ftp.greatcircle.com:/pub/majordomo.tar.Z * Overview Majordomo is a set of perl scripts implementing a mailing list server along with some administrative tools. Majordomo itself is the recipient and handler of all the *-request mail, and can deal with typical subscribe/unsubscribe requests, as well as basic interrogations on the lists (who is on the list, which lists one belongs, etc...). One Majordomo site can manage multiple mailing lists. Actually what happens is that *-request mail is automatically replied to with a canned answer explaining how to use majordomo. The messages have to be sent explicitly to the majordomo server. (Note that majordomo could be made smart enough to derive the missing info from the address in the To:, so that single requests like 'unsubscribe' would take the list argument from the part before the *-request in the To: field) Majordomo understands the concept of "closed" lists (lists where approval is necessary to get subscribed) and "private" lists (where only member may query the list database about others). There are also "secret" lists (lists that do not show up in the list of lists managed), and "auto" lists (the complete inverse from "closed"; all requests are automatically approved). * Configuration A file Majordomo.cf specifies basic configuration parameters, using plain perl syntax (the file is eval'ed directly by the perl scripts). There are a few parameters, setting the machine name, the owner of the list, the location of the logfile, the mailer to be used along with its arguments, etc... One directory 'listdir' is used to store the files for all the mailing lists managed by the site, and the name of the list is prepended to all the various auxiliary files. For instance, list-name.passwd contains the "approve" password for the list-name list. It's possible to run multiple majordomo servers answering different aliases, since each majordomo has its own config file. * Installation To install majordomo, you need to create a new account (say 'list'), set up a majordomo.cf file (in /etc for instance) and create 'homedir' and 'listdir' directories (as specified by majordomo.cf). This is done once and for all, no matter how many lists you want to manage. To create a new list "list-name", you need to set up a few files in the 'listdir' directory (by hand, since there is no tool to help you do that), and edit sendmail's aliases file to record the list aliases. Here is an example of the aliases you will need: list-name: "|/majordomo/wrapper resend -p bulk -l list-name -s list-name-out" list-name-out: :include:/majordomo/lists/list-name list-name-approval: list-name-owner list-name-request: "|/majordomo/wrapper request-answer list-name" And of course, you need the basic majordomo alias and majordomo-owner which will redirect administrative mail to the correct account/program. * Notes The documentation is not completely accurate here. Apparently, some recent changes (like request-answer was formerly called request-recording) have not been completely propagated throughout the supporting files. The author recommends the usage of a C wrapper to reset uid/gid so that perl does not complain (taint checks). Also, scripts make usage of perl's -U option -- which allows "unsafe" operations to be performed. Administration of the majordomo server may be done remotely via e-mail request, the list owner does not need to have an account on the server machine, and could even belong to another organization. * Sources The package is completely written in perl, with the small exception of the C wrapper (a little more than one hundred lines). Perl seems to be well suited for that task, and the complete majordomo system "weighs" only around 2200 lines. The code is typical perl code: not too cryptic but not completely obvious either, and sometimes lacks some comments. However, should a problem occur at a user site, someone with a fair knowledge of perl would be able to easily track down the bug, and eventually fix it. * Compilation Irrelevant (perl scripts). * Documentation The documentation included in the package is minimal, but I believe it covers everything which should be known. The Majordomo paper, available from ftp.greatcircle.com:/pub/majordomo.ps.Z gives a general introduction to Majordomo's functionalities, but does not really document the package itself. ------------------------- Name: procmail Version: 2.81 Author: Stephen R. van den Berg Retrieved-From: comp.sources.misc Patches-From: Author * Overview Procmail is mainly a mail filter, coming along with a set of programs for mail reformatting and mailing list management. The procmail program analyzes some "recipe" file and determines where a message should be delivered based (mainly) on the information contained in its header. The mailing list subsystem can manage multiple lists, answers automatically to *-request mail, subscribing/unsubscribing users, listing the current subscribers, etc... The mailing list maintainer can remotely "administrate" his list via X-Command command headers. Each list is automatically archived, the amount of backlog being configurable. The archive can be accessed via e-mail. (Note that since procmail is a mail filter, the mailing list management is only ONE example of activities that can be "programmed" via rules and scripts or programs, based on mail header/body parsing. You could use procmail to program a mail archive, an archie tool, or anything you want which involves e-mail.) * Configuration Relatively easy. All you need to do is edit the rc.init file and set up about a dozen parameters. Some of those parameters are really obscure, for instance: match_threshold = 30730 # for close matches to the list kept is mystery all the way round. What does THIS mean? I could not find any pointer in the documentation. Actually, the release comes with a set of defaults so you do not need to go through the hassle of configuring each list -- simply accepting the defaults should be fine 80% of the time. * Installation Really easy. All is done from a 'list' account or under a special 'list' directory in your own account, initialized by a script 'install.sh'. To create a new list, you say createlist testing maintainer@somewhere.edu and to remove an existing list, you say removelist testing The 'createlist' script does not actually modify the sendmail aliases file, but echoes the lines which should be included verbatim into /etc/aliases (or /usr/lib/aliases). * Notes Although the initialization and customization of the list is somehow straightforward, the processing of the mail sent to the list requires the launch of many processes: 'procmail' first, which in turn will spawn 'formail' to munge headers and 'sendmail' to actually dispatch the mail. It should be noted that those processes tend to be small and do not use much CPU resource. Recipe files (the heart of procmail) are *really* cryptic. Those files conduct the filtering process by specifying rules and actions to be undertaken when those rules succeed, but it looks like this: :3wc !^Subject:(.*[^a-z])?Re: $!^(X-(Loop: $listaddr|Diagnostic:)|$X_COMMAND) ^FROM_DAEMON | sed -e $cutoff_bounce',$ d' >tmp.request :Ahfw | procbounce Not that intuitive isn't it? Once you know the format, you understand how easy it is to parse, but I would prefer more parsing effort from procmail and an enhanced readability instead of this obscure specification. That criticism has to be soften somehow by considering that the mailing list package is an "advanced" application for the procmail filter. I am not reviewing procmail as a filter here. The good news is: the author supplies you with all the necessary rule files and you should not have to dive into them in order to use the mailinglist management capabilities. * Sources The main programs are written in C (procmail, formail, multigram), the remaining is half shell scripts and half recipe files. The overall package size is medium (around 5400 lines of C, which means 15000 when formatted the "usual" way). The C code is really obscure, with little comments. Formatting is original, but that cannot be an issue. Shell scripts are trivial and I already said what I thought of the recipe files -- they are well commented though. * Compilation Impossible on my MIPS. The author says my include files are damaged beyond repair. Compiles without trouble on a SUN SPARC. The package is auto-configured, but does not uses any public domain auto configuration package. The author devised his own (impressive sometimes) autoconf scheme. When compilation fails, it's hard to know what to do and where to look at, since it's easy to get lost throughout the multiple and recursive calls to make and other shell scripts. I have to say that my machine is a really weird one, having both BSD and USG environments at the same time. * Documentation Good. It surely makes a strong point explaining why the unique combination of procmail and sendmail makes it easier to deal with a mailing list: much filtering can be done by procmail to avoid loops, maintain the archive backlog and answer remote administrative requests, or redirecting subscribe requests to the *-request address instead of dispatching them to the whole list. After reading all the various README, INTRO and FEATURES files, I felt the author had made a wonderful documenting job. ------------------------- Name: UNIX listserver Version: 6.0 beta 4 Author: Anastasios Kotsikonas Retrieved-From: tasos@cs.bu.edu * Overview Listserver is a complete mailing list server, whose goal is to automate as much as possible that part of the mailing list management process which can be automated. It is possible to establish a network connection to the serverd daemon for issuing requests and or maintaining the list, but a batch processing of e-mail commands is also possible (typical subscribe or approve commands). The listserver may manage multiple lists, moderated or not, public or private and/or closed. Lists can be archived and the program provides an interface to retrieve messages from the archive. Users may also upload (and archive) files if the owner(s) allow so. Each user may temporarily suspend its participation and still remain on the list (so he may still "post" but will not receive anything), may switch between digest and reflector mode, may ask to receive a copy of his own contributions, etc... Gateways to news or to other lists may be installed, and the listserver supports the interesting notion of "peer" and "remote" lists, a list which is known locally but handled by a peer listserver or another compatible server. * Configuration Adding a new list seems relatively easy. An alias has to be added to the /etc/aliases, looking like this: list_alias: "|HOMEDIR/catmail -L LIST_ALIAS -f" The list server itself runs under its own uid and has a dedicated 'server' account. Another entry: listserver: "|HOMEDIR/catmail -r -f" is needed for the correct forwarding of administrative requests. * Installation I did not go that far (the system would not compile on my machine...). (The author mentions it has been successfully ported on the following hosts: SUN IBM SGI DEC HP Convex Stardent NeXT SCO Apollo Sequent DG) However, it appears the following steps are necessary: - A dedicated user account must be chosen to run the package, and a listserver alias has to be set to redirect all the mail to that user account. - If TCP connection is wanted, the /etc/services file has to be edited to add a port entry for serverd. - A global 'config' file must then be edited to set up basic configuration parameters like the name of the local organization, the lists we wish to serve, etc... Then of course, each mailing list has its private files held in a separate directory, one per list. Those may be made public for direct ftp access, depending what the specification is. * Notes I have been really impressed by the quality of the documentation and the impressive list of features. The whole package is consistent and seems well thought out. There has been lots of contribution (code and/or suggestions) from various "active" managers so the code is "alive". Unfortunately, since I was not able to compile the system and use it, my whole judgment is only based on what I read in the documentation and what I could understand by having a close look to the sources. One of the advantages of the direct connection is that any requests for "remote" or "peer" lists are handled by chain-internet-connecting to the remote servers, if they go "live" too. Therefore, a user may no longer have to know where a list is located around the world -- there are currently 222 such registered remote lists. (This last figure according to the author) * Sources The whole package is written in C. The formatting used makes the code a little compact, but it's very very well commented and clear. I had absolutely no trouble glancing at the various parts of the code and understanding what their purpose was. * Compilation Failed, unfortunately, on my MIPS. Despite my willing, I was unable to find enough free time to complete the port: too many problems. The author does not uses any auto configuration script, so the initial parameters setting and Makefile editing is boring. When something goes wrong, it's hard to make a general change, so you end-up adding some #ifdef for your own platform and the code becomes messy. Given the overall quality of the product, not being able to compile it is really frustrating. I recommend the use of the metaconfig tool from the dist 3.0 release (dist 2.0 was originally written by Larry Wall, but dist 3.0 has been developed independently) or the GNU autoconf package, which both achieve the same goal using completely different techniques. * Documentation Excellent. I was able to fully appreciate its quality given my impossibility to test the package "for real". I guess this is a rather personal appreciation, since other people have called it "sparse". Truth probably lies somewhere between those two extremes. --------------------------------------------------------------------------- FEATURE TABLE Legend: yes: feature is implemented no: feature is not implemented n/a: not available as is, but some tools might be connected to the program (or on top of the program) to implement them. o/s: out of scope; such a feature may or may not be externally provided, but does not directly fall under the program's jurisdiction, so to speak. (12): please refer to note #12 for further comments. Feature Majordomo Procmail Listserver Basic administrative requests - subscribe, unsubscribe yes yes yes - free format for (un)subscribe no yes no Advanced user requests: - temporarily stop reception no no yes - change between digest/reflector no no yes - who is on the list yes yes yes - lists served yes no yes - which lists user belongs to yes no yes - getting list charter/info yes yes yes - ask for no copy to sender n/a no yes Administrative requests: - changing list password yes no yes - command approval by owner yes no yes - command on behalf of user no no yes Remote administration: - via e-mail yes yes yes - via direct connection no no yes Types of lists (may combine): - public (no prior approval) yes yes yes - closed (subscription approval) yes yes yes - private (only member may query) yes no yes - secret (not advertised) yes no yes (6) - auto (all requests approved) yes no no Types of distribution: - moderated o/s no yes - posting restricted to members no yes yes - reflector mode yes yes yes - automatic digest mode o/s no yes Archive: - provides message archival no yes yes - archive retrieval by e-mail no yes yes - interactive access to server no no yes - faxing files no no yes (3) List administration: - multiple owners yes yes yes - select copy of user's requests no no yes List configuration: - maximum message size o/s yes yes - maximum messages per day o/s no yes - peer lists o/s no yes - remote lists no no yes (4) - news gateway o/s no yes - digest specification o/s no yes - batch periods o/s no yes - archive limits o/s yes yes - shared customization n/a (1) yes yes - bounce history limits n/a yes yes - bounce autoremoval threshold n/a yes yes Miscellaneous: - request log yes no yes - statistics n/a (2) n/a (5) yes - mailer configuration yes yes yes - unattended handling of bounces no yes yes NOTES #1: Shared customization is the ability to specify a common set of attributes for various mailing lists, which can usually be achieved through links of configuration files. Majordomo cannot easily do that since it tends to stuff all the config parameters in one file. However, you may install more than one majordomo server and group lists under one or another. #2: By parsing the log file in perl, it would be easy to build statistics since majordomo traces everything it does. #3: It does so by calling a 'fax' program -- if you do not have it and have the appropriate modem connection, you will not be able to use that feature anyway. #4: Remote lists are served by a remote server which is not necessarily a listserver but has a compatible syntax so administrative requests may be forwarded to its own server. #5: Since procmail is rule driven, it is easy to add hooks to provide some sort of statistics collection. Adding a user interface to retrieve them is another matter, of course. #6: Supported only post beta4 (which is the version I got). DISCUSSION When reviewing those three packages, I have felt victim of my first intuition, which I usually trust anyway. Now that I have completed my task and understand how each software works and what is the philosophy behind, I feel somehow relieved in seeing that my first impression was right. In a word, procmail is the most practical if you want to hack the system, listserver is the most sophisticated with well thought features and a "standard" interface, and majordomo is the most ambitious bet. Let's start with majordomo: it's the toolbox approach. Written in a language well suited for the task (perl), it implements in a few thousand lines of code what listserv or procmail achieve at higher cost, it's simple to understand, easy to install and certainly easier to manage than its two "competitors". However it's a crude tool, designed only to answer *-request type mails and has to rely on other pieces of software to send digests or to implement archival of mailing lists. Majordomo is not meant to manage the traffic on the list, nor issue periodical digests. Listserver looks like the Rolls Royce of mailing list management software. It certainly benefits from a lot of experience and feedback of many list owners, and implements nearly everything which needs to be implemented. There are however a lot of parameters involved and it seems equally easy (some may say uneasy) to install listserver than it is to install UUCP (Taylor version, let's be nice :-)... Once installed however, it should be the most reliable. And it provides some kind of traffic control over the list: it will limit the size of messages, handle bounces automatically, allow users to switch between digest and non-digest mode, etc... This requests far more sophistication than majordomo, but to be fair, majordomo does not really aspire to such a control level. Procmail has the best approach I think, since it uses filtering abilities to perform most of its tasks. Granted, the filtering language is rather cryptic, but at least things are not hardwired and can be customized more freely by a knowledgeable user than listserver for instance. I don't thing customizing majordomo would require more efforts, but you need to change the source code, whereas here the customization is done externally. Due to its ability to handle the mail to different programs, procmail also has a toolbox approach. However, it lacks one great thing which majordomo has for it: Perl. (More on this later.) If we make abstraction of the limits majordomo imposes on itself, and mentally add all the external utilities provided externally by the author, then the three packages offer sensibly the same features, and have comparable interfaces. [Although I *hate* the subscribe interface listserver imposes. Who cares about the real name, since it is already contained in the header? Providing the e-mail address would make more sense, for those of us with a broken mail gateway which cannot be fixed for political reasons -- for instance if you know more about configuring sendmail than the local postmaster but he's in the Computer Center and you're not! :-) -- majordmo's choice makes more sense here, since you may optionally provide your e-mail address] It is my impression that Majordomo, being written in Perl has the biggest potential. I do not want to start another pro/against perl discussion here, but perl is an interpreted language and therefore is subject to many techniques which only interpreted language can easily provide. Although procmail is also based on shell scripts, which is another interpreted language, the shell alone with the standard sed/awk tools is far less efficient than perl, if only by the number of processes involved, and perl is more modular than standard Bourne Shell (it has functions and many many builtins which the shell lacks). Anyway, an interpreted language is its natural extension language, provided you give the user some well defined hooks and/or interface functions. Everything does not have to reside inside the process, since perl can defer compilation/loading until needed, whereas a C program has to contain everything which might be needed, therefore ending up being bigger. [That makes majordomo a candidate for integration in my own filtering program, (mailagent, written in perl), which would bring it some of the flexibility procmail achieves as well as a reliable delivery frame. More importantly, the two applications being written in the same language, a real symbiotic merging can be achieved here :-). Majordomo would simply need to be made a little more modular (more functional entry points) and the mailagent dynamic plugging abilities would do the rest.] CONCLUSION I have presented and discussed three different mailing list software, majordomo (the perl way), procmail (the filter approach) and listserv (the Rolls Royce). Although very different in their implementation and philosophy, they are very similar when looking at the services they provide -- with the notable exception of majordomo which needs external software to reach the same level, but it does not claim to be as ambitious as the others :-). I would like to thank the three authors: Brent Chapman Stephen R. van den Berg Anastasios Kotsikonas for providing me with their software, answering my questions, reviewing the overview presentation of their package and the feature comparison table. Each of you has been doing a great job, even though for personal reasons I like/dislike parts of your software. I simply hope my criticism has been constructive and that my feedback will be helpful to the whole community. -- Raphael Manfredi Interactive Software Engineering Inc. 270 Storke Road, Suite #7 / Tel +1 (805) 685-1006 \ Goleta, California 93117, USA \ Fax +1 (805) 685-6869 /