Great Circle Associates Majordomo-Workers
(April 1995)
 

Indexed By Date: [Previous] [Next] Indexed By Thread: [Previous] [Next]

Subject: Re: large volumes?
From: dwolfe @ risc . sps . mot . com (Dave Wolfe)
Date: Thu, 20 Apr 1995 15:57:18 -0500 (CDT)
To: majordomo-users @ GreatCircle . COM
Cc: majordomo-workers @ GreatCircle . COM
In-reply-to: <199504190920.MAA14401@cc.joensuu.fi> from "Marko Toivanen" at Apr 19, 95 12:19:59 pm
Reply-to: David Wolfe <david_wolfe @ risc . sps . mot . com>

[ Marko Toivanen writes: ]
> 
> Responding to a previous article by Deborah A Hamilton:
> * Also - sorting the large mailing lists by host.domain.name made
> * our mailer more efficient.
> 
> So, anybody got a sort program that would sort beginning from the end of
> each line where the most significat domain lies, and would also lock the
> list file to prevent majordomo using while sorting?

How 'bout the following. Since there's been some discussion about
unsubscribing taking too long (it has to rewrite the entire list for
each unsubscribe request), I realize that this change could also be too
time and memory consuming to sort the list for every subscription, so
decide for yourself. It works for me but hasn't had a lot of testing
time. Diff is against 1.93. Note that it also includes a fix of mine for
mj's annoying habit of being rather arbitrary about permissions and
ownership.

Index: majordomo
===================================================================
RCS file: /cvs/tools/PD/majordomo/majordomo,v
retrieving revision 1.1.1.1
diff -c -r1.1.1.1 majordomo
*** 1.1.1.1	1995/03/01 14:43:23
--- majordomo	1995/04/20 20:36:52
***************
*** 223,229 ****
  	    #    already on the list, and if not, add them to the list.
  	    # Lock and open the list first, even though &is_list_member()
  	    #	 will reopen it read-only, to prevent a race condition
! 	    &lopen(LIST, ">>", "$listdir/$clean_list")
  		|| &abort("Can't append to $listdir/$clean_list: $!");
  	    if (&is_list_member($subscriber, $listdir, $clean_list)) {
  		print REPLY "**** Address already subscribed to $clean_list\n";
--- 219,225 ----
  	    #    already on the list, and if not, add them to the list.
  	    # Lock and open the list first, even though &is_list_member()
  	    #	 will reopen it read-only, to prevent a race condition
! 	    &lopen(LIST, "+>>", "$listdir/$clean_list")
  		|| &abort("Can't append to $listdir/$clean_list: $!");
  	    if (&is_list_member($subscriber, $listdir, $clean_list)) {
  		print REPLY "**** Address already subscribed to $clean_list\n";
***************
*** 234,239 ****
--- 230,267 ----
  		} else {
  		    print LIST $subscriber, "\n";
  		}
+ 
+ 		# sort list by domain (last field is most significant)
+ 		local($entry, $user, @nodes, %canonical, $mode, $uid, $gid);
+ 		seek(LIST, 0, 0)
+ 		    || &abort("Can't rewind $listdir/$clean_list: $!");
+ 		while (<LIST>) {
+ 		    $entry = $_;
+ 		    tr/A-Z/a-z/;
+ 		    ($_) = &ParseAddrs($_);
+ 		    ($user, @nodes) = split(/[.\@]/);
+ 		    @nodes = $#nodes ? ((reverse @nodes), ('') x (3 - $#nodes))
+ 				     : (('') x 3, @nodes); # just host
+ 		    $canonical{join("\0", @nodes, $user)} = $entry;
+ 		}
+ 		(($mode, $uid, $gid) = (stat(LIST))[2,4,5]) ||
+ 		    &abort("Can't stat listdir/$clean_list: $!");
+ 		open(NEW, ">$listdir/$clean_list.new") ||
+ 		    &abort("Can't open $listdir/$clean_list.new: $!");
+ 		chmod($mode, "$listdir/$clean_list.new") ||
+ 		    &abort("chmod($mode, \"$listdir/$clean_list.new\"): $!");
+ 		chown($uid, $gid, "$listdir/$clean_list.new") ||
+ 		  &abort("chown($uid, $gid, \"$listdir/$clean_list.new\"): $!");
+ 		for (sort keys %canonical) {
+ 		    print NEW $canonical{$_};
+ 		}
+ 		close(NEW)
+ 		    || &abort("Error closing listdir/$clean_list.new: $!");
+ 		rename("$listdir/$clean_list", "$listdir/$clean_list.old") ||
+    &abort("rename(\"$listdir/$clean_list\", \"$listdir/$clean_list.old\"): $!");
+ 		rename("$listdir/$clean_list.new", "$listdir/$clean_list") ||
+    &abort("rename(\"$listdir/$clean_list.new\", \"$listdir/$clean_list\"): $!");
+                 unlink("$listdir/$clean_list.old");
  		print REPLY "Succeeded.\n";
  		&log("subscribe $clean_list $subscriber");
  		# Send the new subscriber a welcoming message, and 

-- 
 Dave Wolfe    *Not a spokesman for Motorola*  (512) 891-3246
 Motorola MMTG  6501 Wm. Cannon Dr. W. OE112  Austin  TX  78735-8598

Indexed By Date Previous: unsubscribe *
From: Paul-Joseph de Werk <paul%av2.vrx.vhi.com@vhipub.vhi.com>
Next: Re: large volumes?
From: dwolfe@risc.sps.mot.com (Dave Wolfe)
Indexed By Thread Previous: unsubscribe *
From: Paul-Joseph de Werk <paul%av2.vrx.vhi.com@vhipub.vhi.com>
Next: Re: large volumes?
From: dwolfe@risc.sps.mot.com (Dave Wolfe)

Google
 
Search Internet Search www.greatcircle.com