Great Circle Associates Majordomo-Users
(September 1993)
 

Indexed By Date: [Previous] [Next] Indexed By Thread: [Previous] [Next]

Subject: Re: Obtaining Lists
From: "John P. Rouillard" <rouilj @ cs . umb . edu>
Date: Tue, 14 Sep 1993 11:34:00 -0400
To: majordomo-users @ GreatCircle . COM
In-reply-to: Your message of "Tue, 14 Sep 1993 00:44:13 -0800." <m0ocV3f-0002wDC@hock.bolis.sf-bay.org>


In message <m0ocV3f-0002wDC@hock.bolis.sf-bay.org>, Alan Millar
> > #writes: Is there any way to use the Majordomo listserv to obtain
> > #a list of all the lists on all the Majordomo servers as the $LIST
> > #GLOBAL command does with Eric Thomas's Revised LISTSERV?
> 
[...]
> I have an idea of how to do it.
>
> For the first item (how to extend the lists command) we could allow the
> command to take optional parameters.  If a parameter is specified,
> it would be used as the target of a substring search, just like the
> "which" command.  

I like the idea of the optional search parameter, and I think it would
be worth having in the stock do_lists command.

> If the word "global" is the first word of the parameter, then an
> administrator-defined set of files is also searched for any line
> that matches the parameter.  This means the lists command could
> return information on local lists, lists on other Majordomo systems
> (assuming we had a list of all Majordomo-run lists available in a
> text file- more on that next), and even lists from BITNET (by
> keeping a recent copy of a Revised Listserv LIST GLOBAL command in a
> text file) or other mailing list managers!

But the global command and its extentions should force a replacement
of the standard do_lists command from a module, don't build this into
majordomo proper, so those people who aren't interested in it don't
have to deal with it as far as setup or anything.

> Related to this, it would be nice to extend the current built-in
> lists command to display a one-line description for each list.
> This could be in another list.something file, or we could take the 
> first line of the list.info file, or we could just wait for the new
> configuration system to be finished and have a "description" keyword.

Try sending a lists command to majordomo-test@cs.umb.edu. That address
runs the config file version of majordomo. As with Prego spagetti
sauce, its in there. Also for anybody who wants to retrieve the
configuration file, send a message "config test getconfig" to
majordomo-test@cs.umb.edu. This may or may not be the pretty file
depending on my current test state. majordomo@cs.umb.edu runs a stock
1.56 server, try running lists against it and compare.

> > # If not, is there any way to obtain a complete list of networked servers
> > # and the lists they carry?
> > 
> > Not really.  I've been half-heartedly compiling a list of the
> > Majordomo servers that I know about, just for my own amusement,
> > but I don't really have time to do it on any kind of formal basis.
> 
> Here's my idea for the master Majordomo list:  We set up a master
> mailing list of all Majordomo servers that wish to participate.  We
> include this in the doc files, so new installations know how to join.

Use majordomo to handle registration of majordomo. Are you sure you
aren't tapping my brainwaves?

> We set up a mechanism for a site to run which sends a "lists" command
> to all the Majordomo servers on the list (say, once a month).

hmm, how about:

  /usr/lib/sendmail -f poll_prog@cs.umb.edu  majordomo-servers@cs.umb.edu << EOM
  Approved: <approved>

  lists
  EOM

We need the approved header because the list will have to be moderated.

> The results are returned to a program which combines them into a
> consolidated file.  This consolidated file would be used as one of
> the inputs to the extended "lists" command I described above.

I like it. I had exactly the same idea.

> This polling/consolidating process could be done from a single central
> site and then a consolidated file would be distributed to anyone who 
> wants it.  

I would suggest not distributing it, but have the remote sites poll
the majordomo-master using "lists global".

> Or each site that wishes to use it could run the programs 
> directly themselves with a cron job and a few new entries in their
> system's aliases file.  This may be a little more work to set up,
> but would scale better.

Well along the same lines, why not have a list of master-majordomo
list servers (sort of like the root caches in DNS)?  If somebody
wanted to serve as a master majordomo server they post a "subscribe
majordomo-masters" to one of the majordomo master list-servers. This
list server then propigates the add request to all other
majordomo-masters.  That way a subscribe request to any one of the
majordomo-masters would get propigated. This gets around the problem
of hinging everything on one machine. If that one machine moves, or
goes down it would toast the entire indexing mechanism. With this
distributed scheme each majordomo-master could poll the
majordomo-servers list seperately. That does bring up a problem of
keeping all of the lists synchronized though. Maybe something along
the lines of NTP would work. There is one master of masters, if it can
be reached, the list of lists from there is used.  If the
master-master can't be reached by a master-majordomo, then the master
majordomo goes out and polls. (Multiple majordomo's at multiple levels
are a possibility (to take ntp heirarchy even further,) but that seems
a bit much).

A subscribe/unsubscribe request to the majordomo-servers list would
get sent to all the servers on the majordomo-masters list, and would
be added to the majordomo-servers list on all master servers.  The who
command (kept private for both lists) could be used to do additional
synchronization as well.

With a full pre/post/do syntax this is easy to implement as a loadable
module, and doesn't have to be added to majordomo proper.

> Comments?  What does everyone think about the usefulness of a consolidated
> list?  How about this scheme to implement it?

I think it will work fine, but I DO NOT want to see it in majordomo
proper.  Put it into an entirely seperate file that can be included
from majordmo.cf, or via a module load command.
(&module_load(module_dir) requires all files in that directory).

Besides there are multiple ways of structuring the heirarchy, DNS
like, NTP like etc. We may have many different implementations, and
the user can choose which is best for them.

> P.S.  Yes, I'll volunteer to run the master list at my site and develop 
> the code, but I want discussion/feedback on the features.

Count me in too.

The things we might need are:

 majordomo command:

  poll-version

     if not recognzed, we don't look at lists of lists at all otherwise it
     returns:

     the type of polling we are doing
	  we get lists of list from a master server (say which one)

          E.G.     slave majordomo@cs.umb.edu

          we are a master server, and maintain a master-server mailing list

          E.G. master 2
          E.G. master 1 (a master master server)

     the version number of the polling software we are using
          So we can identify old copies of the polling module.

Also I think the majordomo-master lists should be closed, and possibly
  the majordomo-server list as well so that mortals can't subscribe to
  the list accidentally. 

  Actually the lists should not be listed in the lists of lists (or
  indeed in a lists command) since humans can't get on them. This part
  is easy to do the code is already in the config file verion (note,
  you don't see bblisa-calendar in the lists command anywhere, but it
  is a majordomo list.)

  Since the list is closed, if we want automatic propagation of
  subscribe requests to work, we may need a mechanism for putting out
  approved passwords with the subscribe requests, alternatively some
  is_member(majordomo-master) check that alows unapproved subscribes
  to the list could be used.


				-- John
John Rouillard

Special Projects Volunteer	University of Massachusetts at Boston
rouilj@cs.umb.edu (preferred)	Boston, MA, (617) 287-6480

Consulting Systems Programmer	Bose
rouilj@bose.com			Framingham, MA (508) 879-1916 x6483
===============================================================================
My employers don't acknowledge my existence much less my opinions.



Follow-Ups:
References:
Indexed By Date Previous: A near majordomo trick
From: Tom Limoncelli <tom_limoncelli@Warren.MENTORG.COM>
Next: Re: Majordomo addin modules
From: "John P. Rouillard" <rouilj@cs.umb.edu>
Indexed By Thread Previous: Re: Obtaining Lists
From: Alan Millar <amillar@bolis.sf-bay.org>
Next: Re: Obtaining Lists, addin modules
From: troyer@heffalump.ucsf.edu

Google
 
Search Internet Search www.greatcircle.com