Problems with Apache SSL under load

I have Apache 1.3.27 compiled with mod SSL using openssl 0.9.6.g
OS=AIX 5.1.

The SSL site stops executing CGI scripts when load gets a little
high. I checked the process list and found 106 httpd servers running.
System loads at the UNIX level were nominal (< 0.8).

I get tons of the following error in my error logs:

[Thu Dec 11 06:00:00 2003] [error] [client ] (11)Resource temporarily unavailable: couldn't spawn child process: /usr/local/apache/sslcgi/navbar1
[Thu Dec 11 06:00:00 2003] [error] [client ] (11)Resource temporarily unavailable: couldn't spawn child process: /usr/local/apache/sslcgi/navbar2
[Thu Dec 11 06:00:00 2003] [error] [client ] (11)Resource temporarily unavailable: couldn't spawn child process: /usr/local/apache/sslcgi/register.cgi

HTML page responses are still very fast even with the errors.

Problem does not occur when number of Apache servers < 70.

This is not a great deal of load. The hardware is capable of handling
a lot more than that.

Can someone point me in the right direction? Help is greatly appreciated.
Server configs availble on request. Don't want to send large stuff over
the list.

Thanks.

------------------------------------------------------------ ---------

Dale Weaver deweaver@waketech.edu
UNIX Systems Administrator (919) 662-3508
Wake Technical Community College fax (919) 662-3504

____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org

Re: Problems with Apache SSL under load

On Thursday 11 December 2003 17:16, Dale Weaver wrote:
> The SSL site stops executing CGI scripts when load gets a little
> high. I checked the process list and found 106 httpd servers running.
> System loads at the UNIX level were nominal (< 0.8).
>
> I get tons of the following error in my error logs:
>
> [Thu Dec 11 06:00:00 2003] [error] [client ] (11)Resource temporarily
> unavailable: couldn't spawn child process: /usr/local/apache/sslcgi/navbar1
> [Thu Dec 11 06:00:00 2003] [error] [client ] (11)Resource temporarily
> unavailable: couldn't spawn child process: /usr/local/apache/sslcgi/navbar2
> [Thu Dec 11 06:00:00 2003] [error] [client ] (11)Resource temporarily
> unavailable: couldn't spawn child process:
> /usr/local/apache/sslcgi/register.cgi
>

for executing a cgi apache has to fork a new child process. But forking seems
to fail. Maybe because of an RLIMIT_NPROC you have on your apache or because
the server has reached a totel process limit.

>
> Thanks.
>
> ------------------------------------------------------------ ---------
>
> Dale Weaver deweaver@waketech.edu
> UNIX Systems Administrator (919) 662-3508
> Wake Technical Community College fax (919) 662-3504
>
> ____________________________________________________________ __________
> Apache Interface to OpenSSL (mod_ssl) www.modssl.org
> User Support Mailing List modssl-users@modssl.org
> Automated List Manager majordomo@modssl.org

--
e-admin internet gmbh
Andreas Gietl tel +49 941 3810884
Ludwig-Thoma-Strasse 35
93051 Regensburg mobil +49 171 6070008

PGP/GPG-Key unter http://www.e-admin.de/gpg.html




____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org

RE: Problems with Apache SSL under load

> -----Original Message-----
> From: Dale Weaver [mailto:dale@spiff.wake.tec.nc.us]
>=20
> I have Apache 1.3.27 compiled with mod SSL using openssl 0.9.6.g
> OS=3DAIX 5.1.
>=20
> The SSL site stops executing CGI scripts when load gets a little=20
> high. I checked the process list and found 106 httpd servers running.
> System loads at the UNIX level were nominal (< 0.8).
>=20
> I get tons of the following error in my error logs:
>=20
> [Thu Dec 11 06:00:00 2003] [error] [client ] (11)Resource=20
> temporarily unavailable: couldn't spawn child process:=20
> /usr/local/apache/sslcgi/navbar1
> [Thu Dec 11 06:00:00 2003] [error] [client ] (11)Resource=20
> temporarily unavailable: couldn't spawn child process:=20
> /usr/local/apache/sslcgi/navbar2
> [Thu Dec 11 06:00:00 2003] [error] [client ] (11)Resource=20
> temporarily unavailable: couldn't spawn child process:=20
> /usr/local/apache/sslcgi/register.cgi

Might be to do with system resources like file descriptors or
semaphores. I'm afraid I don't know where to check these on AIX...

Rgds,
Owen Boyle
Disclaimer: Any disclaimer attached to this message may be ignored.=20

>=20
> HTML page responses are still very fast even with the errors.
>=20
> Problem does not occur when number of Apache servers < 70.
>=20
> This is not a great deal of load. The hardware is capable of handling
> a lot more than that.
>=20
> Can someone point me in the right direction? Help is greatly=20
> appreciated.
> Server configs availble on request. Don't want to send large=20
> stuff over
> the list.
>=20
> Thanks.
>=20
> ------------------------------------------------------------ ---------
>=20
> Dale Weaver deweaver@waketech.edu
> UNIX Systems Administrator (919) 662-3508=09
> Wake Technical Community College fax (919) 662-3504
>=20
> ____________________________________________________________ __________
> Apache Interface to OpenSSL (mod_ssl) www.modssl.org
> User Support Mailing List modssl-users@modssl.org
> Automated List Manager majordomo@modssl.org
>=20
Diese E-mail ist eine private und persönliche Kommunikation. Sie hat
keinen Bezug zur Börsen- bzw. Geschäftstätigkeit der SWX Gruppe. =
This
e-mail is of a private and personal nature. It is not related to the
exchange or business activities of the SWX Group. Le pr=E9sent e-mail =
est
un message priv=E9 et personnel, sans rapport avec l'activit=E9 =
boursi=E8re du
Groupe SWX.

This message is for the named person's use only. It may contain
confidential, proprietary or legally privileged information. No
confidentiality or privilege is waived or lost by any mistransmission.
If you receive this message in error, please notify the sender urgently
and then immediately delete the message and any copies of it from your
system. Please also immediately destroy any hardcopies of the message.
You must not, directly or indirectly, use, disclose, distribute, print,
or copy any part of this message if you are not the intended recipient.
The sender's company reserves the right to monitor all e-mail
communications through their networks. Any views expressed in this
message are those of the individual sender, except where the message
states otherwise and the sender is authorised to state them to be the
views of the sender's company.=20


____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org

Re: Problems with Apache SSL under load

On Thu, 11 Dec 2003, Andreas Gietl wrote:

> > [Thu Dec 11 06:00:00 2003] [error] [client ] (11)Resource temporarily
> > unavailable: couldn't spawn child process: /usr/local/apache/sslcgi/navbar1
> > [Thu Dec 11 06:00:00 2003] [error] [client ] (11)Resource temporarily
> > unavailable: couldn't spawn child process: /usr/local/apache/sslcgi/navbar2
> > [Thu Dec 11 06:00:00 2003] [error] [client ] (11)Resource temporarily
> > unavailable: couldn't spawn child process:
> > /usr/local/apache/sslcgi/register.cgi
>
> for executing a cgi apache has to fork a new child process. But forking seems
> to fail. Maybe because of an RLIMIT_NPROC you have on your apache or because
> the server has reached a totel process limit.

Yes, I concur, this sounds like the most likely cause.

--Cliff
____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org

RE: Problems with Apache SSL under load

changing max proc per user might help, say to 1000

chdev -l sys0 -a maxuproc=3D'1000'

for AIX 4.3.3.0

HTH
jorge

--- Boyle Owen escribi=F3: > >
-----Original Message-----
> > From: Dale Weaver
> [mailto:dale@spiff.wake.tec.nc.us]
> >=20
> > I have Apache 1.3.27 compiled with mod SSL using
> openssl 0.9.6.g
> > OS=3DAIX 5.1.
> >=20
> > The SSL site stops executing CGI scripts when load
> gets a little=20
> > high. I checked the process list and found 106
> httpd servers running.
> > System loads at the UNIX level were nominal (<
> 0.8).
> >=20
> > I get tons of the following error in my error
> logs:
> >=20
> > [Thu Dec 11 06:00:00 2003] [error] [client ]
> (11)Resource=20
> > temporarily unavailable: couldn't spawn child
> process:=20
> > /usr/local/apache/sslcgi/navbar1
> > [Thu Dec 11 06:00:00 2003] [error] [client ]
> (11)Resource=20
> > temporarily unavailable: couldn't spawn child
> process:=20
> > /usr/local/apache/sslcgi/navbar2
> > [Thu Dec 11 06:00:00 2003] [error] [client ]
> (11)Resource=20
> > temporarily unavailable: couldn't spawn child
> process:=20
> > /usr/local/apache/sslcgi/register.cgi
>=20
> Might be to do with system resources like file
> descriptors or
> semaphores. I'm afraid I don't know where to check
> these on AIX...
>=20
> Rgds,
> Owen Boyle
> Disclaimer: Any disclaimer attached to this message
> may be ignored.=20
>=20
> >=20
> > HTML page responses are still very fast even with
> the errors.
> >=20
> > Problem does not occur when number of Apache
> servers < 70.
> >=20
> > This is not a great deal of load. The hardware is
> capable of handling
> > a lot more than that.
> >=20
> > Can someone point me in the right direction? Help
> is greatly=20
> > appreciated.
> > Server configs availble on request. Don't want to
> send large=20
> > stuff over
> > the list.
> >=20
> > Thanks.
> >=20
> >
>
------------------------------------------------------------ ---------
> >=20
> > Dale Weaver =20
> deweaver@waketech.edu
> > UNIX Systems Administrator (919)
> 662-3508=09
> > Wake Technical Community College fax
> (919) 662-3504
> >=20
> >
>
____________________________________________________________ __________
> > Apache Interface to OpenSSL (mod_ssl) =20
> www.modssl.org
> > User Support Mailing List =20
> modssl-users@modssl.org
> > Automated List Manager =20
> majordomo@modssl.org
> >=20
> Diese E-mail ist eine private und persönliche
> Kommunikation. Sie hat
> keinen Bezug zur Börsen- bzw. Geschäftstätigkeit der
> SWX Gruppe. This
> e-mail is of a private and personal nature. It is
> not related to the
> exchange or business activities of the SWX Group. Le
> pr=E9sent e-mail est
> un message priv=E9 et personnel, sans rapport avec
> l'activit=E9 boursi=E8re du
> Groupe SWX.
>=20
> This message is for the named person's use only. It
> may contain
> confidential, proprietary or legally privileged
> information. No
> confidentiality or privilege is waived or lost by
> any mistransmission.
> If you receive this message in error, please notify
> the sender urgently
> and then immediately delete the message and any
> copies of it from your
> system. Please also immediately destroy any
> hardcopies of the message.
> You must not, directly or indirectly, use, disclose,
> distribute, print,
> or copy any part of this message if you are not the
> intended recipient.
> The sender's company reserves the right to monitor
> all e-mail
> communications through their networks. Any views
> expressed in this
> message are those of the individual sender, except
> where the message
> states otherwise and the sender is authorised to
> state them to be the
> views of the sender's company.=20
>=20
>=20
>
____________________________________________________________ __________
> Apache Interface to OpenSSL (mod_ssl) =20
> www.modssl.org
> User Support Mailing List =20
> modssl-users@modssl.org
> Automated List Manager =20
majordomo@modssl.org=20

------------
Los mejores usados y las m=E1s tentadoras=20
ofertas de 0km est=E1n en Yahoo! Autos.
Compr=E1 o vend=E9 tu auto en
http://autos.yahoo.com.ar
____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org

RE: Problems with Apache SSL under load

THANK YOU!! I just missed it! It was still set to the default (450). =20

Should work much better now. =20

Thanks again to all who responded. I think this is the solution.
Won't know for sure until the next wave hits.

I guess I should be nominated for a bonehead award. ;)

------------------------------------------------------------ ---------

Dale Weaver deweaver@waketech.edu
UNIX Systems Administrator (919) 662-3508=09
Wake Technical Community College fax (919) 662-3504

On Fri, 12 Dec 2003, [iso-8859-1] Jorge Carrizo wrote:

> changing max proc per user might help, say to 1000
>=20
> chdev -l sys0 -a maxuproc=3D'1000'
>=20
> for AIX 4.3.3.0
>=20
> HTH
> jorge
>=20
> --- Boyle Owen escribi=F3: > >
> -----Original Message-----
> > > From: Dale Weaver
> > [mailto:dale@spiff.wake.tec.nc.us]
> > >=20
> > > I have Apache 1.3.27 compiled with mod SSL using
> > openssl 0.9.6.g
> > > OS=3DAIX 5.1.
> > >=20
> > > The SSL site stops executing CGI scripts when load
> > gets a little=20
> > > high. I checked the process list and found 106
> > httpd servers running.
> > > System loads at the UNIX level were nominal (<
> > 0.8).
> > >=20
> > > I get tons of the following error in my error
> > logs:
> > >=20
> > > [Thu Dec 11 06:00:00 2003] [error] [client ]
> > (11)Resource=20
> > > temporarily unavailable: couldn't spawn child
> > process:=20
> > > /usr/local/apache/sslcgi/navbar1
> > > [Thu Dec 11 06:00:00 2003] [error] [client ]
> > (11)Resource=20
> > > temporarily unavailable: couldn't spawn child
> > process:=20
> > > /usr/local/apache/sslcgi/navbar2
> > > [Thu Dec 11 06:00:00 2003] [error] [client ]
> > (11)Resource=20
> > > temporarily unavailable: couldn't spawn child
> > process:=20
> > > /usr/local/apache/sslcgi/register.cgi
> >=20
> > Might be to do with system resources like file
> > descriptors or
> > semaphores. I'm afraid I don't know where to check
> > these on AIX...
> >=20
> > Rgds,
> > Owen Boyle
> > Disclaimer: Any disclaimer attached to this message
> > may be ignored.=20
> >=20
> > >=20
> > > HTML page responses are still very fast even with
> > the errors.
> > >=20
> > > Problem does not occur when number of Apache
> > servers < 70.
> > >=20
> > > This is not a great deal of load. The hardware is
> > capable of handling
> > > a lot more than that.
> > >=20
> > > Can someone point me in the right direction? Help
> > is greatly=20
> > > appreciated.
> > > Server configs availble on request. Don't want to
> > send large=20
> > > stuff over
> > > the list.
> > >=20
> > > Thanks.
> > >=20
> > >
> >
> ------------------------------------------------------------ ---------
> > >=20
> > > Dale Weaver =20
> > deweaver@waketech.edu
> > > UNIX Systems Administrator (919)
> > 662-3508=09
> > > Wake Technical Community College fax
> > (919) 662-3504
> > >=20
> > >
> >
> ____________________________________________________________ __________
> > > Apache Interface to OpenSSL (mod_ssl) =20
> > www.modssl.org
> > > User Support Mailing List =20
> > modssl-users@modssl.org
> > > Automated List Manager =20
> > majordomo@modssl.org
> > >=20
> > Diese E-mail ist eine private und persönliche
> > Kommunikation. Sie hat
> > keinen Bezug zur Börsen- bzw. Geschäftstätigkeit der
> > SWX Gruppe. This
> > e-mail is of a private and personal nature. It is
> > not related to the
> > exchange or business activities of the SWX Group. Le
> > pr=E9sent e-mail est
> > un message priv=E9 et personnel, sans rapport avec
> > l'activit=E9 boursi=E8re du
> > Groupe SWX.
> >=20
> > This message is for the named person's use only. It
> > may contain
> > confidential, proprietary or legally privileged
> > information. No
> > confidentiality or privilege is waived or lost by
> > any mistransmission.
> > If you receive this message in error, please notify
> > the sender urgently
> > and then immediately delete the message and any
> > copies of it from your
> > system. Please also immediately destroy any
> > hardcopies of the message.
> > You must not, directly or indirectly, use, disclose,
> > distribute, print,
> > or copy any part of this message if you are not the
> > intended recipient.
> > The sender's company reserves the right to monitor
> > all e-mail
> > communications through their networks. Any views
> > expressed in this
> > message are those of the individual sender, except
> > where the message
> > states otherwise and the sender is authorised to
> > state them to be the
> > views of the sender's company.=20
> >=20
> >=20
> >
> ____________________________________________________________ __________
> > Apache Interface to OpenSSL (mod_ssl) =20
> > www.modssl.org
> > User Support Mailing List =20
> > modssl-users@modssl.org
> > Automated List Manager =20
> majordomo@modssl.org=20
>=20
> ------------
> Los mejores usados y las m=E1s tentadoras=20
> ofertas de 0km est=E1n en Yahoo! Autos.
> Compr=E1 o vend=E9 tu auto en
> http://autos.yahoo.com.ar
> ____________________________________________________________ __________
> Apache Interface to OpenSSL (mod_ssl) www.modssl.org
> User Support Mailing List modssl-users@modssl.org
> Automated List Manager majordomo@modssl.org
>=20

____________________________________________________________ __________
Apache Interface to OpenSSL (mod_ssl) www.modssl.org
User Support Mailing List modssl-users@modssl.org
Automated List Manager majordomo@modssl.org