Great Circle Associates Majordomo-Workers
(June 2000)
 

Indexed By Date: [Previous] [Next] Indexed By Thread: [Previous] [Next]

Subject: Re: bounce processing in mj2 ...
From: Jason L Tibbitts III <tibbs @ math . uh . edu>
Date: 07 Jun 2000 17:05:57 -0500
To: The Hermit Hacker <scrappy @ hub . org>
Cc: majordomo-workers @ GreatCircle . COM, mj2-dev @ csf . colorado . edu
In-reply-to: The Hermit Hacker's message of "Wed, 7 Jun 2000 18:21:48 -0300 (ADT)"
References: <Pine.BSF.4.21.0006071820490.465-100000@thelab.hub.org>
User-agent: Gnus/5.0803 (Gnus v5.8.3) Emacs/20.4

>>>>> "THH" == The Hermit Hacker <scrappy@hub.org> writes:

THH> right now, it sends me a message that states that a 'user unknown' was
THH> detected ... any way of just getting it to unregister that user and
THH> save me a several hundred mail messages to wade through? :)

Not yet.  The first phase is detection of actual bounces and parsing of
those that we can parse (relying on our VERP implementation for the rest).
I'm working on this steadily as I get bounces that the software doesn't
parse.

Next is statistics gathering.  If you wanted to always remove a user the
first time you see a bounce from them, we can do that.  But that's pretty
harsh, so we need to gather data so we can detect bounces.

Finally comes the decision making: when does an address get removed?

For statistics gathering, I'm just planning to store a list of bounced
message numbers and times in the subscriber database.  If we keep info on
the past several bounces then when we can get a bounce we can say:

This is the only bounce from the user in the last N messages.
The user has bounced the past M consecutive messages.
The user has bounced M (X%) of the past N messages.
The user has bounced M messages in the past day (week, month).

You get odd effects from delayed bounces because the bounces don't happen
in step with the messages being sent but I don't envision it being a
problem.

You just have to decide when to remove stuff.  There are lots of ways to do
this, but then I was thinking the other day that we already have a pretty
good method: access_rules.  Just keep the same basic parser and hook it up
to a bounce_rules variable:

remove
$days_since_subscribe < 2

keep
$bounce_consecutive < 5

remove
$bounce_percent > 60 || $bounce_consecutive >= 10

would remove any address that bounces at all within the first two days
after the user signed up, then keeps addresses which are bouncing which
haven't bounced five in a row, then bounces any address that has bounced
either 60% of the last N messages or the last ten messages in a row.

Not too difficult to implement, and all available straight from some simple
gathered statistics.

THH> Or ... how about for 'known bounces', create a daily summary and
THH> discarding the bounces?

That is another useful step.  These should be logged, and we can use the
inform variable to designate whether the owner is notified for each
identified bounce, if they are notified in a periodic report or if they
aren't told at all.

Unfortunately, the report generation stuff has yet to be written.  This
would be a great thing for someone not intimately affiliated with the
project to work on; we have a global master log and a number of message
logs per list; all are in text format and should be easy to parse.  I even
have some prototype code.  I've always been excited about writing this but
never had the time.

 - J<


References:
Indexed By Date Previous: Re: Majordomo2 nightly snapshots ...
From: Ajay Sharma <ssharma@odc.net>
Next: Re: Majordomo2 nightly snapshots ...
From: Jason L Tibbitts III <tibbs@math.uh.edu>
Indexed By Thread Previous: bounce processing in mj2 ...
From: The Hermit Hacker <scrappy@hub.org>
Next: Re: bounce processing in mj2 ...
From: Jason L Tibbitts III <tibbs@math.uh.edu>

Google
 
Search Internet Search www.greatcircle.com