Sendmail perfomance

Greetings.

A server sendmail (clamav, spamassassin) on Linux. A mail stream in
day: rejected + viruses + spam =3D 2 million messages, send + received =3D
100 thousand messages. Peak threads of sendmail nearby 700,
simultaneously smtp connections 600 (plus about 1500 local sockets for
clamav-milter and spamass-milter). Mail queue is average 100 letters.
The processor, memory, network and a disk subsystem - low utilization.

The swap is completely free, anything in it is not unloaded. Memory on
30 % is occupied by system and sendmail's threads, the rest is
occupied by cache (50 %) and buffers (10 %).

At peak load (five hours in day) response time sendmail reaches to 50
seconds (between the beginning of tcp connection and output of smtp
banner to the remote side).

Something does not allow sendmail'=D5 to fork more threads, here just?

Operating system at base level is optimised.

sendmail.cf
O MaxDaemonChildren=3D5000
O ConnectionRateThrottle=3D150

Re: Sendmail perfomance

Post removed (X-No-Archive: yes)

Re: Sendmail perfomance

j-k-l@mail.ru wrote:
> A server sendmail (clamav, spamassassin) on Linux. A mail stream in
> day: rejected + viruses + spam = 2 million messages, send + received =
> 100 thousand messages. Peak threads of sendmail nearby 700,
> simultaneously smtp connections 600 (plus about 1500 local sockets for
> clamav-milter and spamass-milter). Mail queue is average 100 letters.
> The processor, memory, network and a disk subsystem - low utilization.
>
> The swap is completely free, anything in it is not unloaded. Memory on
> 30 % is occupied by system and sendmail's threads, the rest is
> occupied by cache (50 %) and buffers (10 %).
>
> At peak load (five hours in day) response time sendmail reaches to 50
> seconds (between the beginning of tcp connection and output of smtp
> banner to the remote side).
>
> Something does not allow sendmail'у to fork more threads, here just?

Sendmail-8 uses processes not threads.

> Operating system at base level is optimised.
>
> sendmail.cf
> O MaxDaemonChildren=5000
> O ConnectionRateThrottle=150

What is the response time in off peak hours?
[ Are you *SURE* the delays are (mostly) load related? ]

URL(s):
http://www.sendmail.org/documentation/books
[...]
sendmail Performance Tuning
Authored by Nick Christenson
http://www.jetcafe.org/~npc/book/sendmail/
Date: 2003 (September, 2002) Pages: 256

--
[pl>en: Andrew] Andrzej Adam Filip anfi@xl.wp.pl
Open-Sendmail: http://open-sendmail.sourceforge.net/
No one becomes depraved in a moment.
-- Decimus Junius Juvenalis

Re: Sendmail perfomance

On Apr 9, 12:52=A0pm, Res wrote:

> define(`confTO_IDENT', `0s')

#grep ident sendmail.cf
#O Timeout.ident=3D0


>make sure your Greetpause is not set to high, a value of 5000 is ample

i will try it


> What does 'w' report in this peak time

# w
16:28:51 up 5 days, 3:52, 2 users, load average: 0.61, 0.60, 0.61
and one user (me) logged in



On Apr 9, 1:53=A0pm, Andrzej Adam Filip wrote:

> What is the response time in off peak hours?

At night, for example, 0.01-0.65 sec. average, i am assured of it,
because use monitoring system

Re: Sendmail performance

j-k-l@mail.ru wrote:

> On Apr 9, 12:52 pm, Res wrote:
>
>> define(`confTO_IDENT', `0s')
>
> #grep ident sendmail.cf
> #O Timeout.ident=0
>
>
>>make sure your Greetpause is not set to high, a value of 5000 is ample
>
> i will try it
>
>
>> What does 'w' report in this peak time
>
> # w
> 16:28:51 up 5 days, 3:52, 2 users, load average: 0.61, 0.60, 0.61
> and one user (me) logged in
>
>
>
> On Apr 9, 1:53 pm, Andrzej Adam Filip wrote:
>
>> What is the response time in off peak hours?
>
> At night, for example, 0.01-0.65 sec. average, i am assured of it,
> because use monitoring system

Sorry for asking but ~30% of questions here contain "misleading" hints :-)

0) Do you use local (caching) DNS server on the same host as sendmail?
[ DNS response times may be load depended - yet another *stupid* check]
1) Some DNSBL limit number of DNS queries services - maybe you get
"no response" during peak load periods and DNS timeouts
2) Have you used "top" during peak hours?
AFAIR *Linux* does not count "I/O waiting" processes into system load,
"top" command on Linuxes shows some statistics about "I/O waiting" processes
3) Could you post which Linux distribution you use?
[ There is a few percent chance that the problem is distribution depended]

--
[pl>en: Andrew] Andrzej Adam Filip anfi@xl.wp.pl
Open-Sendmail: http://open-sendmail.sourceforge.net/
"Those who believe in astrology are living in houses with foundations of
Silly Putty."
-- Dennis Rawlins, astronomer

Re: Sendmail performance

On Apr 9, 8:38=A0pm, Andrzej Adam Filip wrote:

> Sorry for asking but ~30% of questions here contain "misleading" hints :-)=

i have used almost all tips&hints from all sources accessible to me
and i am ready to answer any questions

0) yes, i use local caching nameserver and i will look at its
productivity

1) it can be a bottleneck as on the average 1500 msgs/min rejectes by
DNSBL, i will check it

2) i thought of it and i have statistics for last two weeks:
user 3.96%
nice 0.23%
system 1.80%
iowait 0.53%
irq 0.01%
soft 0.05%
steal 0.00%
idle 93.07%

3) fedora 7, 2.6.22 :(

Re: Sendmail performance

j-k-l@mail.ru wrote:

> On Apr 9, 8:38 pm, Andrzej Adam Filip wrote:
>
>> Sorry for asking but ~30% of questions here contain "misleading" hints :-)
> i have used almost all tips&hints from all sources accessible to me
> and i am ready to answer any questions
>
> 0) yes, i use local caching nameserver and i will look at its
> productivity
>
> 1) it can be a bottleneck as on the average 1500 msgs/min rejectes by
> DNSBL, i will check it
>
> 2) i thought of it and i have statistics for last two weeks:
> user 3.96%
> nice 0.23%
> system 1.80%
> iowait 0.53%
> irq 0.01%
> soft 0.05%
> steal 0.00%
> idle 93.07%
>
> 3) fedora 7, 2.6.22 :(

Fedora specific:
AFAIR redhat and fedora compile sendmail with libwrap (tcpwrappers) library.
The library does its own separate access control e.g. it does *separate*
ident lookups. Newer librap version allow to setup ident lookup timeout.

General:
Could you post *exact* timings of "connect to greeting" delays and some
hints how it grows with load?
*Exact* timing and "jumps" in delay timing may give some hints about
what causes the problem.

P.S. I have run out from obvious hints :-)

--
[pl>en: Andrew] Andrzej Adam Filip anfi@xl.wp.pl
Open-Sendmail: http://open-sendmail.sourceforge.net/
....Unix, MS-DOS, and Windows NT
(also known as the Good, the Bad, and the Ugly).
-- Matt Welsh

Re: Sendmail performance

thanks for the help...

here statistic - http://img374.imageshack.us/img374/9758/statvk7.jpg

big peak of mail queue on thursday - i have disabled dnsbl checking

Re: Sendmail performance

> here statistic -http://img374.imageshack.us/img374/9758/statvk7.jpg

and for a day
http://img391.imageshack.us/img391/8149/statuu5.jpg