Great Circle Associates Majordomo-Workers
(August 1996)
 

Indexed By Date: [Previous] [Next] Indexed By Thread: [Previous] [Next]

Subject: Re: New! - Fast delivery for large lists
From: pdc @ virtualvin . com (Paul Close)
Date: Thu, 8 Aug 1996 10:52:23 -0700 (PDT)
To: stas @ isf . ru
Cc: majordomo-workers @ GreatCircle . COM
In-reply-to: <3209A0AB.2A11@isf.ru> from "Stanislav Sinyagin" at Aug 08, 1996 12:09:15 PM

> I created a perl program which dramatically speeds up the 
> list delivery: 350-users list is being delivered in 25 minutes, 
> while canonical Majordomo sends it for about 7 hours.
> What we usually do, is
>         test-outgoing: :include:/usr/local/Lists/test
> Thus, Sendmail creates one queue job for all the list members, 
> and delivers the mail sequentually.
> Now we do
>         test-outgoing: "|/home/majordomo/wrapper splitlist -l test"
> 
> Splitlist opens a SMTP connection with localhost, and for each N (N=10) 
> users sends a separate message. So, sendmail gets several litle jobs, 
> N users each. Sure, then it forks so much as it needs to process these
> queues 
> parallelly. But don't be much afraid, it won't break your machine down:
> my 486dx/33 with 16mb memory sends 350-users list in 25 minutes.

This is very interesting -- perhaps you could post it here (if it's not too
large!).

A few observations:

    - I hope N is settable, as many people have lists 10x your 350 list.
      35 sendmail processes isn't that scary, but 350 is!

    - You should really sort your mailing list by host.  Otherwise sendmail
      can't collapse hosts as well as it would like.

    - Along those lines, you should not use 'N' arbitrarily -- it's always
      better to batch all mail for one domain into a single batch no matter
      how large, since you are typically talking to only one host.  Just
      batch up what's left....

-- 
Paul Close		http://reality.sgi.com/pdc (moving soon)
pdc@netcontents.com


Follow-Ups:
References:
Indexed By Date Previous: New! - Fast delivery for large lists
From: Stanislav Sinyagin <stas@isf.ru>
Next: Re: New! - Fast delivery for large lists
From: Chris Shenton <cshenton@it.hq.nasa.gov>
Indexed By Thread Previous: New! - Fast delivery for large lists
From: Stanislav Sinyagin <stas@isf.ru>
Next: Re: New! - Fast delivery for large lists
From: Chris Shenton <cshenton@it.hq.nasa.gov>

Google
 
Search Internet Search www.greatcircle.com