From majordomo-docs-owner@greatcircle.com Thu Jun 19 15:10:45 2003 Received: from daniel.liston.nu (24-116-58-25.cpe.cableone.net [24.116.58.25]) by mycroft.greatcircle.com (Postfix) with ESMTP id AC93D195A03; Thu, 19 Jun 2003 15:10:42 -0700 (PDT) Received: from sonny.org (50.sub-199-74-155.myvzw.com [199.74.155.50]) by daniel.liston.nu (iPlanet Messaging Server 5.2 Patch 1 (built Aug 19 2002)) with ESMTPA id <0HGR00G0J1LTZ9@daniel.liston.nu>; Thu, 19 Jun 2003 17:10:42 -0500 (CDT) Date: Thu, 19 Jun 2003 17:10:09 -0500 From: Daniel Liston Subject: Archive/Digest HOW-TO (revision 2) In-reply-to: <3BB1B813.6080202@pandasoftware.es> To: majordomo-users@GreatCircle.Com, majordomo-docs@greatcircle.com Message-id: <3EF234C1.6040306@sonny.org> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii; format=flowed Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 References: <3BB1B813.6080202@pandasoftware.es> X-Archive-Number: 200306/1 X-Sequence-Number: 10 Disclaimer: I am not a professional writer or programmer. Please report any/all discrepencies in this document to me via the majordomo-users or majordomo-docs mailing lists at greatcircle.com. Majordomo list owners often have trouble setting up a digest version of their list. I am writing this in hopes of clearing up some of this confusion. Successfully archiving your lists could be a side effect of implementing the knowledge gained from the examples included. Assumptions: Majordomo version 1.94.5 Sendmail version 8.8 (or higher) You have root permission on the server to create users, modify the aliases file, and rebuild the aliases database. You have created a majordomo user and majordomo group. The majordomo $HOME directory is /usr/local/majordomo and this is owned by user majordomo and group majordomo. The smrsh directory (if used) is /etc/smrsh and holds a link owned by root to /usr/local/majordomo/wrapper.mytld There are no group or world "writable" directories in the path to /usr/local/majordomo, including / (root). Sendmail "trusts" majordomo. You have a Tmajordomo line in sendmail.cf and submit.cf or these files use the /etc/mail/trusted-users file. Sendmail only recognizes one "/etc/aliases" file. Others are allowed but outside the scope of this note. Sendmail uses the "always_add_domain" feature. The majordomo config file is /etc/majordomo.cf The majordomo.cf variables are defined statically. No $whereami = "`/bin/dnsdomainname`"; it should be $whereami = "myhost.mydomain.tld"; You already have a working list that utilizes resend. eg. wrapper resend -l -outgoing Your aliases file contains nobody: /dev/null Preface: The directories above may be different, but in that case, you will modify anything stated further to match your installation. To set up a digest, you must also have a handle on archiving. Digest has some dependencies, or I should say, makes some assumptions. One of these is that you have an archive directory configured to hold the digestified message before delivery, and another is that you will call your digest delivery address -digest-outgoing. Your digest and archive directories are defined in the majordomo.cf file as $digest_work_dir and $filedir. $filedir_suffix is also used for defining the location of your archives. For the purpose of this recipe, I will keep these in the same directory tree as my other majordomo data files and directories. The sample.cf gives an example of /usr/local/mail/digest when lists are in /usr/local/lists and archive files are in /usr/local/mail/files. Here are my suggested settings based on the assumptions above; $homedir = "/usr/local/majordomo"; $digest_work_dir = "$homedir/digests"; $listdir = "$homedir/lists"; $filedir = "$homedir/archives"; $filedir_suffix = ""; What is not explained very well, is that you will need to create these directories (if you haven't already) as the majordomo user, or change ownership to the majordomo user if you created them as root. Make sure these directories are not group or world writable, to prevent sendmail from complaining. You will also have to reference these directories in your /etc/aliases so mail is passed to these scripts that will be doing the actual work. In some (if not all) cases, there will also be subdirectories involved that will need the same ownerships and permissions as their parents. A "test" mailing list that does archives AND digests will have aliases that look like this; owner-test: test-owner test: "|/usr/local/majordomo/wrapper resend -l test test-outgoing,nobody" test-owner: someone@your.domain, test-request: "|/usr/local/majordomo/wrapper majordomo -l test" test-approval: moderator@your.domain ### could also be pointed to test-owner test-outgoing: :include:/usr/local/majordomo/lists/test,test-archiver,test-digestify test-archiver: "|/usr/local/majordomo/wrapper archive2.pl -f /usr/local/majordomo/archives/test/test -a -M" test-digestify: "|/usr/local/majordomo/wrapper digest -r -C -l test-digest test-digest-outgoing" owner-test-digest: test-owner test-digest: test, test-digest-owner: test-owner test-digest-request: "|/usr/local/majordomo/wrapper majordomo -l test-digest" test-digest-approval: test-approval test-digest-outgoing: :include:/usr/local/majordomo/lists/test-digest,nobody" The -approval addresses are not required, and will default to owner-(-digest), and the -request addresses could just be pointed to the owner/moderator alias depending on your specific needs or requirements. Some of the "owner" aliases are redundant, but safe. Let me also explain that you do not need to enable archiving to use a digest version of your list, but you WILL need the majordomo.cf and archives directory properly configured for digestification to happen. The digest script also "assumes" your outgoing address for the digest is called -digest-outgoing. In the case above, test-digest-outgoing. To continue with our example, we now have a dependency on subdirectories for archiver and the digestify. We need to create these before we run "newaliases" or "sendmail -bi". mkdir /usr/local/majordomo/archives/test mkdir /usr/local/majordomo/archives/test-digest mkdir /usr/local/majordomo/digests/test-digest The above need to be owner:group majordomo:majordomo and chmod 755. Loose security could be tightened up, but that is a different HOW-TO. :) The test-archiver: alias will stamp the archive with a 4 digit year and 2 digit month. If you notice your archive showing up as 1901xx instead of 2001xx, please apply the folllowing patch to archive2.pl =========================================== --- archive2.pl.orig 2000/01/07 11:00:49 +++ archive2.pl.new 2000/05/15 00:55:00mkdir /usr/local/majordomo/archives/test @@ -152,7 +152,11 @@ local($junk, $addr, $dow, $moy, $dom, $time, $year, @rest); ($junk, $addr, $dow, $moy, $dom, $time, $year, @rest) = split(/\s+/,$from); - &open_archive($FH, $year % 100, $MoY{$moy}, $dom); + if ($year =~ /\d{4}/) { + &open_archive($FH, $year -1900, $MoY{$moy}, $dom); + } else { + &open_archive($FH, $year % 100, $MoY{$moy}, $dom); + } } sub open_archive { =========================================== NOTE: To do digesting without the standard archive, there are two changes to the aliases. test-outgoing: :include:/usr/local/majordomo/test,test-digestify and you do not "need" the test-archiver: alias. If this is the case, you also do not need to create the /usr/local/majordomo/archives/test directory. Leaving them in, even though they are never used, does not really hurt either. It is all a matter of calling or not calling the test-archiver: alias via the test-outgoing: alias. The final steps in making your digest happen daily/nightly, is to set your test-digest.config file variables high enough that digestification does not try to happen automatically, and to create a crontab entry that will make majordomo execute the digest script. In your test-digest.config file, depending on how busy you expect your list to be, adjust these variables appropriately high. I have not tried it, but maybe they can all be left blank. digest_maxdays = 30 digest_maxlines = 9999 maxlength = 500000 The maxlength is in bytes, not Kbytes. The other two are self explanatory. You only have to set one value, but you will have to experiment to get it right. A good starting point, might be to set digest_maxdays to 7, and leave the other two blank. Otherwise, you will probably have to tinker with the maxlength in hundreds of thousands or millions of bytes. As long as we are in the test-digest.config file, don't forget about the other digest_* variables either. digest_archive = digest_issue = 1 digest_name = test-digest digest_rm_footer = test digest_rm_fronter = test digest_volume = 1 digest_work_dir = To keep your digests clean, it is also a good idea to use purge_received = yes on your regular list (test.config) as well as the digest. However, this does make tracking problems more difficult. Then you can get a little fancy and set your fronters and footers, message_fronter << END - Include in this issue: _SUBJECTS_ END message_footer << END unsubscribe instructions, copyright message etc. END If your security uses cron.allow or cron.deny, install the appropriate setting for . Your crontab entry should not be owned by root or majordomo to avoid "Majordomo ABORT" messages. I typically fill in with the uid of the list-owner. Now to access the crontab... crontab -u -e SHELL=/bin/sh MAILTO="" 30 4 * * 5 echo mkdigest test-digest | mail majordomo #| | | | | ^here to % or EOL is field 6 for commands #| | | | +-- 3 letter day of week or 0-7 0=Sun #| | | +-- 3 letter month or 1-12 #| | +-- day 1-31 #| +-- hours 0-23 #+-- minutes 0-59 The 5th line above sends majordomo an email forcing it to make a digest and send it to the default test-digest-outgoing alias every Friday at 4:30AM. If your delivery address is not test-digest-outgoing, that line can be changed to something like 30 4 * * 5 echo mkdigest test-digest test-digest-whatever | mail majordomo If you want the list to go out daily, change the "5" above, to a "*". Having a password stored in the crontab is probably not a good idea either, but it does get the job done. Dan Liston PS. Please feel free to add comments/criticisms about this document. It is after all my first revision at explaining the steps necessary to archive/digest a mailing list. It is possible that I may have still omitted something useful.