Great Circle Associates Majordomo-Users
(September 2005)
 

Indexed By Date: [Previous] [Next] Indexed By Thread: [Previous] [Next]

Subject: Suggestions requested for throttling input
From: Stewart Dean <sdean @ bard . edu>
Date: Mon, 12 Sep 2005 11:37:23 -0400
To: majordomo-users @ greatcircle . com
User-agent: Mozilla Thunderbird 1.0.6 (Windows/20050716)

Majordomo is working as designed, but the mailing list server and 
mailserver are getting somewhat overwhelmed by having enormous clots of 
list thrown at them at the same time on Mondays and Fridays.

I am wondering if anyone has implemented or knows of anyone implementing 
a throttling/priority dispatch.  I have two features in mind:
1) to throttle output
2) to attach priority to mailings.  I wonder what I'd do if I had a 
queue full of mailings and an emergency came up that I had to get out a 
mailing immediately

I /am/ already using a wrapper for bulkmailer (because without it, all 
the bm submission fail on Draining Input) and I suppose I could use the 
wrapper do something like check the load, and if excessive, insert a 
-sendmail -0dq flag in the bulkwrapper invocation.  Anyone with better 
ideas?

Background =======================================================

My machines are running AIX V5.2 and sendmail V8.11.  When things get 
bad, we'll see 125-175 mailings of about 100-400 recipients in each 
mailing dropped in the hopper within the span of 30-45 minutes.  This is 
a rather free-thinking college, which means we have very little ability 
to control the users.

Before we (recently) put in bulk_mailer in the mailing list server, its 
throughput limitations served to throttle what was coming at the mail 
server...it would run for 2-3 hours to chew its way through the mailings 
log jam and it didn't much impact the college mailserver.  But mailings 
were getting through in a timely fashion and recently I realized that 
some mailings weren't getting through during the peak load periods... 
getting lost due to the drop that happens when you see the dreaded 
"Timeout waiting for input from local during Draining Input"

Now, with bulk_mailer, the mailing list server chews through in 
something like 30-45 minutes, there are no Draining Input messages, 
though I do see, during peak loads, a bulkmailer failure syslog message:
Sep  9 16:18:37 shell sendmail[1556506]: j89GHfY245932: 
to="|/usr/local/bin/bulk_wrapper owner-stu07@bard.edu 
/usr/local/majordomo/lists/stu07", ctladdr=stu07-doit (1/0), 
delay=00:00:56, xdelay=00:00:48, mailer=prog, pri=290194, dsn=4.0.0,
stat=Operating system error
Even so, this does get queued, so it does get sent a bit later.

OTOH, this greater throughput makes the load on the mail server a lot 
heavier.  Before it wasn't really noticeable, but now the load of a mass 
mailing hits the mail the mail server hard.  Used to be, its CPU rarely 
got over 20%.  Now the increased throughput means the mail server's Cpu 
  can go to 70 %.....

Any ideas?

-- 
====
Stewart Dean, Unix System Admin, Henderson Computer Resources
Center of Bard College, Annandale-on-Hudson, New York  12504
sdean@bard.edu  voice: 845-758-7475, fax: 845-758-7035

Indexed By Date Previous: Re: Confused - again.
From: "Jim Hermann" <wizard@jimhermann.com>
Next: Error emails
From: "Robert Isaac" <robert.isaac@volvoclub.org.uk>
Indexed By Thread Previous: Re: Confused - again.
From: "Jim Hermann" <wizard@jimhermann.com>
Next: Error emails
From: "Robert Isaac" <robert.isaac@volvoclub.org.uk>

Google
 
Search Internet Search www.greatcircle.com