Thomas Leavitt writes:
> One of our users, who maintains a relatively large mailing list, has
> had his subscriber lists truncated by Majordomo at least twice. Needless to
> say, this is quite distressing.
> I have been told by another list-owner at Netcom that this has happened
> to her a number of times as well... from this, I suspect this phenomenon,
> is likely to occur when the load on the server the list is running
> on peaks, since Netcom machines are cronically overloaded, and the
> truncations on our site have occured when we've be dealing with rogue processes
> spamming the system.
> We are working on a custom solution to this problem, but I thought
> I'd raise the issue here, and see if anyone else had experienced it,
> or has identified the cause (we added a number of error checks after
> the first occurance, but these seem to have been ignored when the
> phenomenon repeated itself).
This problem also happened to us after transferring two large and
one enormous mailing list from InterNIC Information Services to
our host on Information Directory and Database Services. The
enormous mailing list (scout-report) got spammed at least 4 times
in one week. We also took a tremendous performance hit since the
other two large lists are very active.
I've been working on solutions to this problem and have developed
the following facilities that appear to working somewhat okay:
1.) Queue all mail coming in to majordomo and the mailing lists
and process the queue in order to single thread all requests.
This was a hack on the queuer we have for our email server
applications. Instead of sending the mail directly to majordomo
or to resend, it's saved in a queue file with a flag telling
which service it should be forwarded to. Every half-hour, cron
kicks off the job to process the queue in order, waiting until
requested service completes before sending off the next request.
2.) For large mailing lists (> 2500 subscribers), batch the
subscription requests and process them as a group once/day.
This involved a new parameter in the list config files
(batch_subs = yes|no). If turned on, the subscribe/unsubscribe
requests are saved in <listname>.sub. Notification is sent
to the requester that the request has been queued. Once a day,
cron kicks off a job to process a list's .sub file. The
requesters are notified when the subscribe/unsubscribe completes,
and the list-owner gets a list of the subscribed/unsubscribed
Since I've implemented these two facilities, the lists have not
been spammed. Yes, it slows down getting mail posted to the lists,
but it has also pretty much eliminated the battles for the lock files
causing failed majordomo requests and posting email to the lists.
I'm still working out some minor glitches in the batch subscription
processor (changing from a grep to a hash seach to verify if an
id is in the list). If you're interested, I'll share what I've got.
InterNIC Directory and Database Services Support
AT&T Bell Laboratories