HA Sendmail

Currently I have 4 Linux/sendmail boxes behind a (hardware)
LoadBalancer. Each host has it's own local disk and single sendmail
queue directory. I currently process about 380k emails (as determined
by grep -ci 'stat=sent' by date in the shared log)... with about
1.8million log lines are written per day.

Q: is it feasible to have shared disk for multiple sendmail hosts, and
if yes, how ... NFS doesn't seem like a good idea due to the high IO
required. If I do SAN, will this even work with sendmail, or is there
a better solution?

Thanks in advance! phil

Re: HA Sendmail

In article <1191373149.673309.315080@22g2000hsm.googlegroups.com>,
Phil wrote:

> Currently I have 4 Linux/sendmail boxes behind a (hardware)
> LoadBalancer. Each host has it's own local disk and single sendmail
> queue directory. I currently process about 380k emails (as determined
> by grep -ci 'stat=sent' by date in the shared log)... with about
> 1.8million log lines are written per day.
>
> Q: is it feasible to have shared disk for multiple sendmail hosts, and
> if yes, how ... NFS doesn't seem like a good idea due to the high IO
> required. If I do SAN, will this even work with sendmail, or is there
> a better solution?

It generally isn't safe to share the queue between Sendmail instances on
different machines, and you would even in principle buy very little from
sharing the queue even if you worked out the details of how to avoid
queue ID collisions. It is possible to use customized clustering
approaches and SAN disk to have Sendmail queues from failed hosts taken
over by other nodes for clearance, but that's a pretty small benefit for
the cost.

Depending on your specific flavor of your sendmail use and other
functions of those machines (such as mailstore/POP3/IMAP) there might be
advantages from sharing delivery space.

--
Now where did I hide that website...

Re: HA Sendmail

On 10/2/2007 11:32 PM, Bill Cole wrote:
> It generally isn't safe to share the queue between Sendmail instances
> on different machines, and you would even in principle buy very
> little from sharing the queue even if you worked out the details of
> how to avoid queue ID collisions. It is possible to use customized
> clustering approaches and SAN disk to have Sendmail queues from
> failed hosts taken over by other nodes for clearance, but that's a
> pretty small benefit for the cost.

Just thinking out loud here...

Would it be possible to have multiple sendmail systems queue messages in
their own respective queue on a shared queue file system. I.e. box A
uses queue A and box B uses queue B and box C uses queue C on a central
SAN. Configure each box to only queue messages to its own queue and
walk it every N minutes / hours while also configuring each box to walk
another systems queue every 2 * N minutes / hours (or longer) and only
process messages that are at least 2 * N minutes / hours (or longer)
old. The idea being that if any given box stops processing messages for
some reason, another system will pick up and process the failed systems
queue (long) *AFTER* the failed system would have processed it. In
essence having one system process the queue first and then a different
system process the queue subsequently? The idea being that messages
that were queued would be moved through the queue by the first system
that is processing them, thus the system that queued them. However if a
given system stops processing its queue another will pick up and process
it for it later on down the line. I know that this fails to take in to
account messages that are almost persistent in the queue, but in this
case you may want to hand such messages off to a different system with a
longer term queue life cycle so that queues for the main systems stay
small and turn over faster.

Again, just thinking out loud.

> Depending on your specific flavor of your sendmail use and other
> functions of those machines (such as mailstore/POP3/IMAP) there might
> be advantages from sharing delivery space.

*nod*



Grant. . . .

Re: HA Sendmail

On Wed, 3 Oct 2007, Grant Taylor wrote:

> Just thinking out loud here...
>
> uses queue A and box B uses queue B and box C uses queue C on a central
> SAN. Configure each box to only queue messages to its own queue and

I've always been against SAN's, and prefere NAS's, simply because SAN
still AFAIK has the limitation of only one access per filesystem at the
exact same time, where NAS does not have this limitation.


--

Cheers
Res

Slackware -V- sloooUbuntoooou
http://lxer.com/module/newswire/view/93393/

Re: HA Sendmail

On 10/3/2007 7:52 PM, Res wrote:
> I've always been against SAN's, and prefere NAS's, simply because SAN
> still AFAIK has the limitation of only one access per filesystem at
> the exact same time, where NAS does not have this limitation.

Really? I'm not really versed in SANs any more than recreational
playing with first generation fiber channel dot com bust stuff so I
can't say for sure, but this surprises me. Are you meaning to tell me
that you can not have more than one system access different blocks of a
cluster aware file system at the exact same time and that the SAN will
block the access to the second while the first is accessing it? Forgive
my naivety in that I feel this to be wrong. Now if you are trying to
say that more than one system can not access a non cluster aware file
system at the same time as in only one can have it mounted at a time,
I'll buy that.



Grant. . . .

Re: HA Sendmail

In article ,
Res wrote:

> On Wed, 3 Oct 2007, Grant Taylor wrote:
>
> > Just thinking out loud here...
> >
> > uses queue A and box B uses queue B and box C uses queue C on a central
> > SAN. Configure each box to only queue messages to its own queue and
>
> I've always been against SAN's, and prefere NAS's, simply because SAN
> still AFAIK has the limitation of only one access per filesystem at the
> exact same time, where NAS does not have this limitation.

Unless I am misunderstanding you, I don't believe that GFS, SFCFS,
OCFS2, or GPFS have such a limitation.

I'm unconvinced that the complexity that comes with such clustered
filesystems is worthwhile for most mail environments. Having the queues
of other Sendmail instances mounted and ready to be run is not likely to
be worth the trouble. If one is going to need to watchdog the host that
owns a particular queue, it is simpler to just have the SAN allow all of
the nodes to see the disks for all nodes' queues and have the watchdog
mount the queues of a dead node elsewhere as needed for clearing. Online
shared filesystems historically have had per-node performance penalties
relative to traditional filesystems (as does NFS) and if the goal is to
clear a many-thousand-message queue left by a dead Sendmail it is
probably more valuable to have it perform fast in that clearance than to
save the 10 seconds it might take to mount and fsck the dropped SAN
disks on another node.

--
Now where did I hide that website...