mqueue too large!

Hi, I'm trying to send a lot of mail. Messages get queued up in
/var/spool/mqueue. It makes the server go down because theres like over
15,000 files in the mqueue directory.

How can I help this? Can I split up the df and qf files somehow in my
sendmail.mc file? Like /var/spool/mqueue/df/* and /var/spool/mqueue/qf/*?

Are there and MTAs that can handle alot of messages? Like is postfix
better and handling large amounts of queued messages?

Any help would be great.

I'm using linux 2.4 with sendmail 8.14.1 (whatever the latest is). I'm
using ext3 under linux, and it seems at about 15,000 files and the
server frezzes up.

Re: mqueue too large! [multiple queue directories]

Ken Williams writes:

> Hi, I'm trying to send a lot of mail. Messages get queued up in
> /var/spool/mqueue. It makes the server go down because theres like
> over 15,000 files in the mqueue directory.
>
> How can I help this? Can I split up the df and qf files somehow in my
> sendmail.mc file? Like /var/spool/mqueue/df/* and
> /var/spool/mqueue/qf/*?
>
> Are there and MTAs that can handle alot of messages? Like is postfix
> better and handling large amounts of queued messages?
>
> Any help would be great.
>
> I'm using linux 2.4 with sendmail 8.14.1 (whatever the latest is).
> I'm using ext3 under linux, and it seems at about 15,000 files and the
> server frezzes up.

[ should work for sendmail-8.10+ ]
1) In *.mc file add
define(`QUEUE_DIR, `/var/spool/mqueue*')
[current queue directory with * added at the end]
2) create new sendmail.cf
3) create additional queue directories matching the pattern in "1" e.g.
/var/spool/mqueue.1 /var/spool/mqueue.2 /var/spool/mqueue.3
[ initially create extra directories ]
4) restart sendmail daemon

*New* messages will be distributed among all queue directories.

Creating separate Queue Groups for top destinations may give even better
results.

--
[pl>en: Andrew] Andrzej Adam Filip : anfi@priv.onet.pl : anfi@xl.wp.pl
Open-Sendmail: http://open-sendmail.sourceforge.net/
The smallest worm will turn being trodden on.
-- William Shakespeare, "Henry VI"
----
http://groups.google.com/groups?selm=87odbahkqi@tara.fsf.hob by-site.com

Re: mqueue too large! [multiple queue directories]

Thanks, but how does sendmail know how many files/items will go into
mqueue.1 or mqueue.2 or mqueue.3? Is there like a fixed limit? I'm
just wondering what the logic is for "messages will be distributed among
all queue directories".


Andrzej Adam Filip wrote:
> Ken Williams writes:
>
> [ should work for sendmail-8.10+ ]
> 1) In *.mc file add
> define(`QUEUE_DIR, `/var/spool/mqueue*')
> [current queue directory with * added at the end]
> 2) create new sendmail.cf
> 3) create additional queue directories matching the pattern in "1" e.g.
> /var/spool/mqueue.1 /var/spool/mqueue.2 /var/spool/mqueue.3
> [ initially create extra directories ]
> 4) restart sendmail daemon
>
> *New* messages will be distributed among all queue directories.
>
> Creating separate Queue Groups for top destinations may give even better
> results.
>

Re: mqueue too large! [multiple queue directories]

Post removed (X-No-Archive: yes)

Re: mqueue too large!

In article ,
Ken Williams wrote:

> Hi, I'm trying to send a lot of mail. Messages get queued up in
> /var/spool/mqueue. It makes the server go down because theres like over
> 15,000 files in the mqueue directory.

That indicates a problem deeper than Sendmail. There's something wrong
with that system if it can't work its way out of that situation. It's
not an ideal circumstance obviously, but it should not be fatal.

> How can I help this? Can I split up the df and qf files somehow in my
> sendmail.mc file? Like /var/spool/mqueue/df/* and /var/spool/mqueue/qf/*?

That's not the way it works, but yes, you can split the queue. The usual
approach is to split the queue groups based on recipient address.

This is something you really need to read the docs on and analyze your
specific needs. A reasonable starting point on queue groups is the the
Operations Guide (in the Sendmail distribution at doc/op/{me,ps,txt})
but that won't give you a cookbook recipe. Google can probably find you
someone's cookbook recipe, but using anything like that needs to be done
very carefully.

> Are there and MTAs that can handle alot of messages? Like is postfix
> better and handling large amounts of queued messages?

Postfix may be slightly better as a high-volume sending platform, but
not significantly. Nearly all truly high-volume broadcast senders use
tools specifically designed for high-volume broadcast sending. Sendmail
can do it on suitable hardware with suitable configuration, but you
really need to get immersed in the details of Sendmail configuration and
specific analysis of your system and the sorts of list(s) you are
mailing to get it right. If you are not willing to at least learn to
read and tweak sendmail.cf rules, Sendmail is the wrong platform for you
to base a high-volume sending system on.

> Any help would be great.
>
> I'm using linux 2.4 with sendmail 8.14.1 (whatever the latest is). I'm
> using ext3 under linux, and it seems at about 15,000 files and the
> server frezzes up.

Again, something is very wrong there. I don't run any Linux mail servers
that handle significant volume, but given what I've seen Sendmail manage
with bloated single-directory queues on AdvFS and VxFS, I have to
believe that you've got something deeper than can be fixed with Sendmail
config. Whether it's more memory, filesystem tuning, kernel tuning, or
better storage hardware, I can't dream of saying because you don't
mention anything about those aspects and I'm not terribly familiar with
high-performance Linux config. In general, you should NOT just accept
the behavior you seem to be describing in the server as a whole as
normal or beyond analysis. Something specific is happening to freeze it
up, such as acute complete memory starvation, and you might find relief
of a sort simply by addressing that specific immediate cause.

--
Now where did I hide that website...

Re: mqueue too large! [multiple queue directories]

In article ,
Ken Williams wrote in an annoyingly top-posted
message:

> Thanks, but how does sendmail know how many files/items will go into
> mqueue.1 or mqueue.2 or mqueue.3? Is there like a fixed limit? I'm
> just wondering what the logic is for "messages will be distributed among
> all queue directories".

It's random.

A simple division of the queue directory is not always the best
approach, particularly for lists that include many addresses in domains
that have very poor acceptance speed. The 8.12+ queue group feature
provides more configurability so that you can do things like give Yahoo
its own queue group and reduce the impact of their incompetence on
non-Yahoo addresses.

Have you considered the advantages you could gain by reading some
documentation? The Operation Guide and cf/README are free, and the
O'Reilly Sendmail "Bat" book and Nick Christenson's Sendmail
Performance Tuning are each cheaper than a moderately good dinner out.


> Andrzej Adam Filip wrote:
> > Ken Williams writes:
> >
> > [ should work for sendmail-8.10+ ]
> > 1) In *.mc file add
> > define(`QUEUE_DIR, `/var/spool/mqueue*')
> > [current queue directory with * added at the end]
> > 2) create new sendmail.cf
> > 3) create additional queue directories matching the pattern in "1" e.g.
> > /var/spool/mqueue.1 /var/spool/mqueue.2 /var/spool/mqueue.3
> > [ initially create extra directories ]
> > 4) restart sendmail daemon
> >
> > *New* messages will be distributed among all queue directories.
> >
> > Creating separate Queue Groups for top destinations may give even better
> > results.
> >

--
Now where did I hide that website...

Re: mqueue too large!

In article ,
Bill Cole wrote:

> In article ,
> Ken Williams wrote:
>
> > Hi, I'm trying to send a lot of mail. Messages get queued up in
> > /var/spool/mqueue. It makes the server go down because theres like over
> > 15,000 files in the mqueue directory.
>
> That indicates a problem deeper than Sendmail. There's something wrong
> with that system if it can't work its way out of that situation. It's
> not an ideal circumstance obviously, but it should not be fatal.
>
> > How can I help this? Can I split up the df and qf files somehow in my
> > sendmail.mc file? Like /var/spool/mqueue/df/* and /var/spool/mqueue/qf/*?
>
> That's not the way it works, but yes, you can split the queue. The usual
> approach is to split the queue groups based on recipient address.

Correction: You CAN have Sendmail split storage of qf/df/xf files into
subdirectories out of a queue directory simply by creating the
subdirectories. It isn't likely to be a high-yield solution unless you
spread those across different filesystems with different paths to
different disks.

--
Now where did I hide that website...