From Robin's Wiki

RobinsStuff: ClusteredMailingList

Clustered Mailing List

Typically, mailing lists are run from a single mail server (there are exceptions, e.g. the cypherpunks list, but that uses a method different from the one I'm proposing here). This is fine 99% of the time, but if that server dies, then the list as a whole is gone, possibly taking subscription records with it. This page is a plan for a mailing list setup with automatic failover, so that if any number of servers go down, the list will still run without anybody having to do anything. This is a problem in two parts:

  1. Mail to be sent to the list must be routed to a mail server that is running, that then distributes the message out to all subscribers.
  2. User settings must be able to be changed at any one of these servers and automatically propagate to the other servers.

The Mail Side

This is the easy part, so will be dealt with first. We need to ensure that mail sent to list@lists.example.com will get to a working server in order to get out to the subscribers. This situation is neatly handled by the SMTP system as it stands.

All the servers that are handling the lists will have their mail server configured to accept mail from the domain lists.example.com and pass that on to Mailman. Mailman will then resend it to all list subscribers.

The DNS for the list domain will be set so that every server participating in the system is referenced in an MX entry, so that if one server is down, mail servers will automatically try another one.

The Settings Side

This is significantly more complex, and isn't addressed by any software as far as I'm aware. The problem is that if a user changes a setting, subscribes, or unsubscribes at one location, that change is rapidly (within about 10 minutes) sent out to the other servers. Ideally, even servers that are down should eventually get this updated information.

This information would best be transmitted by sending it out in an email. One way of doing this is for each server to have a special address for capturing these messages and processing them. When a user changes their settings on a server, the server sends a message out to all the other servers detailing the changes.

On the server, this would entail two types of hooks into mailman:

  1. One that detects changed settings, subscriptions, and unsubscriptions and sends an email out to the other servers with the changed information
  2. One that receives these emails and updates the local database with the new information.

Aside from this, a method of adding and removing servers is needed, along with a method of preventing malicious users from forging emails and sending fake updates. This can be done simply by having an administrator send a control message out to all the servers telling them of the addition or removal of a server. The security aspect can be dealt with here too. Each server will have a public key associated with it, and when sending a message will sign it with the appropriate private key. All servers receiving a control message will verify that it is signed by a key matching one on the list of servers, if the signature fails or is unknown, the message is ignored (although an admin should be notified)

Retrieved from http://www.kallisti.net.nz/RobinsStuff/ClusteredMailingList
Page last modified on October 10, 2005, at 05:58 AM