Stability problems with Apache on OS X 10.6 Server

--_000_C7553C401381Athscccimagazinecom_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi,

I'm having stability problems with apache on OS X 10.6.2 Server. I would li=
ke to know if anybody else is having the same experience. And if there is a=
workaround.

The apache server is Apache/2.2.8 (Unix) mod_ssl/2.2.8 OpenSSL/0.9.8k DAV/2=
with mod_ldap build in. The server is build against OpenLDAP v 2.4.19 wi=
th DB backupend. The apache server queries the LDAP on user login.

After 5000+ SSL requests, users begin to be denied log-in to the web site. =
In the Apache error_log the following is written:

[Mon Dec 21 09:14:23 2009] [info] Initial (No.1) HTTPS request received for=
child 6 (server 172.25.2.99:443)
could not lookup DNS configuration info service: (ipc/send) invalid destina=
tion port
could not lookup DNS configuration info service: (ipc/send) invalid destina=
tion port
.... 50 more ...
could not lookup DNS configuration info service: (ipc/send) invalid destina=
tion port
could not lookup DNS configuration info service: (ipc/send) invalid destina=
tion port
could not lookup[Mon Dec 21 09:14:23 2009] [warn] [client x.x.x.x] [95611] =
auth_ldap authenticate: user y authentication failed; URI /images/redesign/=
info.gif [LDAP: ldap_simple_bind_s() failed][Can't contact LDAP server], re=
ferer: https://.....

After a restart the server runs flawless untill the error occurs again afte=
r another 5000+ requests. While the errors are appearing in the log file, =
I can use the LDAP service without any problems in a terminal using ldap* p=
rograms.

On the same server I have a ProFTPD running which also does log-in verifica=
tion against the LDAP server and retrieves varios information about the use=
r. The program runs into the same problems, ie. the "could not lookup ..."=
begins to appear in the it=F8s log file, and users are refused access to t=
he FTP server.

The messages "could not lookup .." does not begin to appear at the same tim=
e in the logfiles. The apache may be running fine, while the FTP server is =
rejecting users, and wise versa.

I googled "(ipc/send) invalid destination port" and found references to peo=
ple who experienced problems in OS X 10.0, but have not found anything for =
OS X 10.6.

But from my findings I suspect the problem might arise from OS X 10.6 and n=
ot from Apache / ProFTPD. But it just a qualified quess.

Does anybody have the same experience or something similar ? And if so, how=
did you solve it ?

Any help is appreciated !

Regards,

Thomas Schneider


--_000_C7553C401381Athscccimagazinecom_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable






11pt'>Hi,



I’m having stability problems with apache on OS X 10.6.2 Server. I wo=
uld like to know if anybody else is having the same experience. And if ther=
e is a workaround.



The apache server is Apache/2.2.8 (Unix) mod_ssl/2.2.8 OpenSSL/0.9.8k DAV/2=
 with mod_ldap build in.  The server is build against OpenLDAP v=
2.4.19 with DB backupend. The apache server queries the LDAP on user login=
..



After 5000+ SSL requests, users begin to be denied log-in to the web site. =
In the Apache error_log the following is written:

  

[Mon Dec 21 09:14:23 2009] [info] Initial (No.1) HTTPS request received for=
child 6 (server 172.25.2.99:443)

could not lookup DNS configuration info service: (ipc/send) invalid destina=
tion port

could not lookup DNS configuration info service: (ipc/send) invalid destina=
tion port

.... 50 more ...

could not lookup DNS configuration info service: (ipc/send) invalid destina=
tion port

could not lookup DNS configuration info service: (ipc/send) invalid destina=
tion port

could not lookup[Mon Dec 21 09:14:23 2009] [warn] [client x.x.x.x] [95611] =
auth_ldap authenticate: user y authentication failed; URI /images/redesign/=
info.gif [LDAP: ldap_simple_bind_s() failed][Can't contact LDAP server], re=
ferer: .



After a restart the server runs flawless untill the error occurs again afte=
r another  5000+ requests. While the errors are appearing in the log f=
ile, I can use the LDAP service without any problems in a terminal using ld=
ap* programs.



On the same server I have a ProFTPD running which also does log-in verifica=
tion against the LDAP server and retrieves varios information about the use=
r. The program runs into the same problems, ie. the “could not lookup=
 ...” begins to appear in the itøs log file, and users a=
re refused access to the FTP server.



The messages “could not lookup ..” does not begin to appear at =
the same time in the logfiles. The apache may be running fine, while the FT=
P server is rejecting users, and wise versa.



I googled “(ipc/send) invalid destination port” and found refer=
ences to people who experienced problems in OS X 10.0, but have not found a=
nything for OS X 10.6.



But from my findings I suspect the problem might arise from OS X 10.6 and n=
ot from Apache / ProFTPD. But it just a qualified quess.



Does anybody have the same experience or something similar ? And if so, how=
did you solve it ?



Any help is appreciated !



Regards,



 Thomas Schneider








--_000_C7553C401381Athscccimagazinecom_--

Re: Stability problems with Apache on OS X 10.6 Server

In users Digest Issue 3741 (21 Dec 2009 15:54:32 -0000), Thomas Scheider wro=
te:

> [problems with LDAP authentication on 10.6.2]
> After 5000+ SSL requests, users begin to be denied log-in to the web
> site. In the Apache error_log the following is written:
>
> [Mon Dec 21 09:14:23 2009] [info] Initial (No.1) HTTPS request received=
for child 6 (server 172.25.2.99:443)
> could not lookup DNS configuration info service: (ipc/send) invalid=
destination port
> [...]
> On the same server I have a ProFTPD running which also does log-in
> verification against the LDAP server and retrieves varios information
> about the user. The program runs into the same problems, ie. the "could
> not lookup ..." begins to appear in the it=F8s log file, and users are
> refused access to the FTP server.
> [...]

This would imply that the problem lies not with Apache, but with either
the LDAP server or OS 10.6.2 (which is, I assume, the OS on which the
Apache and ProFTPd applications are running).

> The messages "could not lookup .." does not begin to appear at the same
> time in the logfiles. The apache may be running fine, while the FTP server=

> is rejecting users, and wise versa.

I'd suspect that the connections to the LDAP server are not being "cleaned
up", and once the application reaches it's per-process file descriptor limit
it is being denied its request to open another network connection (i.e.,
allocate another file descriptor). Check the output of:
lsof -nPi | grep ":389"
(NOTE: You must execute this as "root" in order to see _all_ the connections=
)
This should show you all the current connections to the LDAP server, which
application/process is "controlling" that connection, and the current
connection state.

You might also check the system.log; there may be entries in there if the
LDAP connection requests ARE being rejected due to the filedescriptor limit.=


What to do next depends on:
a) Whether my theory is correct; and,
b) What state the connections are "hung" in.

Regards,

Michael A. Pasek

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Stability problems with Apache on OS X 10.6 Server

--_000_C7871F5013C89thscccimagazinecom_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi Michael,

Thank you for the pointers. I have used you suggestions to try a out (quite=
) a few things. Not that I have solved my problems.
I'll try to summarize my findings.

When I start apache, it launched a few worker threads. As I start using the=
server, it launched new threads. The first time a thread gets a request, i=
t connects to the LDAP server. Once connected the thread keeps the connecti=
on to the LDAP server open / alive until the thread gets shutdown. This I c=
an see while watching the what port connections are on the machine between =
the LDAP Server and the Apache server (Thanks for the command tip).

This runs fine for while. Then apache server starts to fail. Does not fail =
completly, it is more random thing. Some requests are servered correct, whi=
le other get a "Internal Server Error". If I do a reload on the failed req=
uest, it may process correctly. So some of the apache threads are working, =
while others failing.

Here is a snapshot of the ports and runnning processes at a time when the s=
erver is partially failing. There are no warnings in any of the system logf=
iles about file handles or ports limits being reached.

losf:

httpd 2947 ctp 12u IPv6 0xffffff802288f2f0 0t0 T=
CP [::1]:55452->[::1]:389 (ESTABLISHED)
httpd 2947 ctp 13u IPv6 0xffffff800f60ae80 0t0 T=
CP [::1]:56411->[::1]:389 (ESTABLISHED)
httpd 2949 ctp 12u IPv6 0xffffff80228914a0 0t0 T=
CP [::1]:55451->[::1]:389 (ESTABLISHED)
httpd 2949 ctp 13u IPv6 0xffffff800f608fe0 0t0 T=
CP [::1]:55776->[::1]:389 (ESTABLISHED)
slapd 2971 root 6u IPv6 0xffffff800f609f30 0t0 T=
CP *:389 (LISTEN)
slapd 2971 root 7u IPv4 0xffffff8022936d68 0t0 T=
CP *:389 (LISTEN)
slapd 2971 root 12u IPv6 0xffffff802288efe0 0t0 T=
CP [::1]:389->[::1]:55451 (ESTABLISHED)
slapd 2971 root 15u IPv6 0xffffff802288fc20 0t0 T=
CP [::1]:389->[::1]:55452 (ESTABLISHED)
slapd 2971 root 16u IPv6 0xffffff8022890b70 0t0 T=
CP [::1]:389->[::1]:56411 (ESTABLISHED)
slapd 2971 root 17u IPv6 0xffffff800f609910 0t0 T=
CP [::1]:389->[::1]:55465 (ESTABLISHED)
slapd 2971 root 18u IPv6 0xffffff802288ff30 0t0 T=
CP [::1]:389->[::1]:55466 (ESTABLISHED)
slapd 2971 root 19u IPv6 0xffffff802288e9c0 0t0 T=
CP [::1]:389->[::1]:57959 (ESTABLISHED)
slapd 2971 root 20u IPv6 0xffffff800f60a860 0t0 T=
CP [::1]:389->[::1]:55621 (ESTABLISHED)
slapd 2971 root 22u IPv6 0xffffff800f60b7b0 0t0 T=
CP [::1]:389->[::1]:56296 (ESTABLISHED)
slapd 2971 root 23u IPv6 0xffffff800f607d80 0t0 T=
CP [::1]:389->[::1]:55666 (ESTABLISHED)
slapd 2971 root 24u IPv6 0xffffff802288e3a0 0t0 T=
CP [::1]:389->[::1]:55679 (ESTABLISHED)
slapd 2971 root 25u IPv6 0xffffff802288f910 0t0 T=
CP [::1]:389->[::1]:55691 (ESTABLISHED)
slapd 2971 root 26u IPv6 0xffffff802288ecd0 0t0 T=
CP [::1]:389->[::1]:55776 (ESTABLISHED)
httpd 2976 ctp 12u IPv6 0xffffff802288e6b0 0t0 T=
CP [::1]:55465->[::1]:389 (ESTABLISHED)
httpd 2977 ctp 12u IPv6 0xffffff800f6083a0 0t0 T=
CP [::1]:55466->[::1]:389 (ESTABLISHED)
httpd 3055 ctp 12u IPv6 0xffffff800f609c20 0t0 T=
CP [::1]:55621->[::1]:389 (ESTABLISHED)
httpd 3185 ctp 12u IPv6 0xffffff800f609600 0t0 T=
CP [::1]:55666->[::1]:389 (ESTABLISHED)
httpd 3198 ctp 12u IPv6 0xffffff8022890550 0t0 T=
CP [::1]:55691->[::1]:389 (ESTABLISHED)
httpd 3199 ctp 12u IPv6 0xffffff8022890e80 0t0 T=
CP [::1]:55679->[::1]:389 (ESTABLISHED)
httpd 3199 ctp 13u IPv6 0xffffff80228917b0 0t0 T=
CP [::1]:57959->[::1]:389 (ESTABLISHED)
httpd 28410 ctp 12u IPv6 0xffffff800f607a70 0t0 T=
CP [::1]:56296->[::1]:389 (ESTABLISHED)

ps ax | grep httpd
2946 ?? Ss 0:00.97 /usr/local/apache2/bin/httpd -k start
2947 ?? S 0:00.32 /usr/local/apache2/bin/httpd -k start
2949 ?? S 0:00.35 /usr/local/apache2/bin/httpd -k start
2976 ?? S 0:00.31 /usr/local/apache2/bin/httpd -k start
2977 ?? S 0:00.30 /usr/local/apache2/bin/httpd -k start
3055 ?? S 0:00.25 /usr/local/apache2/bin/httpd -k start
3185 ?? S 0:00.24 /usr/local/apache2/bin/httpd -k start
3198 ?? S 0:00.36 /usr/local/apache2/bin/httpd -k start
3199 ?? S 0:00.24 /usr/local/apache2/bin/httpd -k start
28410 ?? S 0:00.08 /usr/local/apache2/bin/httpd -k start
28752 ?? S 0:00.02 /usr/local/apache2/bin/httpd -k start

Combining the two you can see, that some of the threads have a connection w=
hile others do not. And in the the apache error log I see:

could not lookup DNS configuration info service: (ipc/send) inval[Thu Jan 2=
8 10:04:02 2010] [warn] [client 10.2.3.164] [28752] auth_ldap authenticate:=
user thsc authentication failed; URI xxx [LDAP: ldap_simple_bind_s() faile=
d][Can't contact LDAP server], referer: https://xxx/xxxx

So the thread not serving request is the one which has no connection to the=
LDAP server and can't get it either. Other programs, proftd, ldapsearch, a=
ccessing the LDAP server are not experiencing any problems while this is ha=
pping.

Now if I shutdown the LDAP server and start it again, the apache server com=
es back into the same "state", ie randomly failing. If I that a new snapsho=
t of the ports open. all threads are able to reconnect to LDAP server, exce=
pt the thread 28752 , which puts out the same log message again.

To see if it might have something to do with a specific version of the Open=
LDAP server, I have also done this with different versions of the OpenLDAP =
server., ie. Shutting down one version and starting up an older version of =
the LDAP server. The result is the same.

Last time I wrote that is seams to depend on the number of requests to the =
server. After my testing I more inclined to think it is a time issue. It se=
ems to be of less importance how much I stress the server. If I stress it a=
lot, it can run 1/2 to 1 hour before starting to fail. If I only make occa=
tional request I get from 1 hour up to 3 hours. If I start the server, make=
a few requests and leave it for the night, it will be in the partially fai=
led state in the morning.

I have the (nearly) same setup running on an OS X 10.4 server without any p=
roblems. I write nearly, because on the 10.4 machine it is an apache 2.0.49=
server, while I have an 2.2.14 apache server on the OS X 10.6 server. The =
ProFTPD server I use on the two setups is of the same version and it runs i=
nto the exact same problem on OX 10.6 as the Apache server does, with the s=
ame error message in it's log files.

Does this make any sense ? After this I'm still inclined to suspect OS X 10=
..6 ...

Regards,

Thomas Schneider


On 21/12/09 17:31, "Michael A. Pasek" wrote:

In users Digest Issue 3741 (21 Dec 2009 15:54:32 -0000), Thomas Scheider wr=
ote:

> [problems with LDAP authentication on 10.6.2]
> After 5000+ SSL requests, users begin to be denied log-in to the web
> site. In the Apache error_log the following is written:
>
> [Mon Dec 21 09:14:23 2009] [info] Initial (No.1) HTTPS request received f=
or child 6 (server 172.25.2.99:443)
> could not lookup DNS configuration info service: (ipc/send) invalid desti=
nation port
> [...]
> On the same server I have a ProFTPD running which also does log-in
> verification against the LDAP server and retrieves varios information
> about the user. The program runs into the same problems, ie. the "could
> not lookup ..." begins to appear in the it=F8s log file, and users are
> refused access to the FTP server.
> [...]

This would imply that the problem lies not with Apache, but with either
the LDAP server or OS 10.6.2 (which is, I assume, the OS on which the
Apache and ProFTPd applications are running).

> The messages "could not lookup .." does not begin to appear at the same
> time in the logfiles. The apache may be running fine, while the FTP serve=
r
> is rejecting users, and wise versa.

I'd suspect that the connections to the LDAP server are not being "cleaned
up", and once the application reaches it's per-process file descriptor limi=
t
it is being denied its request to open another network connection (i.e.,
allocate another file descriptor). Check the output of:
lsof -nPi | grep ":389"
(NOTE: You must execute this as "root" in order to see _all_ the connection=
s)
This should show you all the current connections to the LDAP server, which
application/process is "controlling" that connection, and the current
connection state.

You might also check the system.log; there may be entries in there if the
LDAP connection requests ARE being rejected due to the filedescriptor limit=
..

What to do next depends on:
a) Whether my theory is correct; and,
b) What state the connections are "hung" in.

Regards,

Michael A. Pasek


--_000_C7871F5013C89thscccimagazinecom_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable






11pt'>Hi Michael,



Thank you for the pointers. I have used you suggestions to try a out (quite=
) a few things. Not that I have solved my problems.

I’ll try to summarize my findings.



When I start apache, it launched a few worker threads. As I start using the=
server, it launched new threads. The first time a thread gets a request, i=
t connects to the LDAP server. Once connected the thread keeps the connecti=
on to the LDAP server open / alive until the thread gets shutdown. This I c=
an see while watching the what port connections are on the machine between =
the LDAP Server and the Apache server (Thanks for the command tip).



This runs fine for while. Then apache server starts to fail. Does not fail =
completly, it is more random thing. Some requests are servered correct, whi=
le other get a  “Internal Server Error”. If I do a =
reload on the failed request, it may process correctly. So some of the apac=
he threads are working, while others failing.



Here is a snapshot of the ports and runnning processes at a time when the s=
erver is partially failing. There are no warnings in any of the system logf=
iles about file handles or ports limits being reached.



losf:



httpd      2947      &nbs=
p;     ctp   12u  IPv6 0xffffff8022=
88f2f0      0t0    TCP [::1]:55452-=
>[::1]:389 (ESTABLISHED)

httpd      2947      &nbs=
p;     ctp   13u  IPv6 0xffffff800f=
60ae80      0t0    TCP [::1]:56411-=
>[::1]:389 (ESTABLISHED)

httpd      2949      &nbs=
p;     ctp   12u  IPv6 0xffffff8022=
8914a0      0t0    TCP [::1]:55451-=
>[::1]:389 (ESTABLISHED)

httpd      2949      &nbs=
p;     ctp   13u  IPv6 0xffffff800f=
608fe0      0t0    TCP [::1]:55776-=
>[::1]:389 (ESTABLISHED)

slapd      2971      &nbs=
p;    root    6u  IPv6 0xffffff800f=
609f30      0t0    TCP *:389 (LISTE=
N)

slapd      2971      &nbs=
p;    root    7u  IPv4 0xffffff8022=
936d68      0t0    TCP *:389 (LISTE=
N)

slapd      2971      &nbs=
p;    root   12u  IPv6 0xffffff802288efe=
0      0t0    TCP [::1]:389->[::=
1]:55451 (ESTABLISHED)

slapd      2971      &nbs=
p;    root   15u  IPv6 0xffffff802288fc2=
0      0t0    TCP [::1]:389->[::=
1]:55452 (ESTABLISHED)

slapd      2971      &nbs=
p;    root   16u  IPv6 0xffffff8022890b7=
0      0t0    TCP [::1]:389->[::=
1]:56411 (ESTABLISHED)

slapd      2971      &nbs=
p;    root   17u  IPv6 0xffffff800f60991=
0      0t0    TCP [::1]:389->[::=
1]:55465 (ESTABLISHED)

slapd      2971      &nbs=
p;    root   18u  IPv6 0xffffff802288ff3=
0      0t0    TCP [::1]:389->[::=
1]:55466 (ESTABLISHED)

slapd      2971      &nbs=
p;    root   19u  IPv6 0xffffff802288e9c=
0      0t0    TCP [::1]:389->[::=
1]:57959 (ESTABLISHED)

slapd      2971      &nbs=
p;    root   20u  IPv6 0xffffff800f60a86=
0      0t0    TCP [::1]:389->[::=
1]:55621 (ESTABLISHED)

slapd      2971      &nbs=
p;    root   22u  IPv6 0xffffff800f60b7b=
0      0t0    TCP [::1]:389->[::=
1]:56296 (ESTABLISHED)

slapd      2971      &nbs=
p;    root   23u  IPv6 0xffffff800f607d8=
0      0t0    TCP [::1]:389->[::=
1]:55666 (ESTABLISHED)

slapd      2971      &nbs=
p;    root   24u  IPv6 0xffffff802288e3a=
0      0t0    TCP [::1]:389->[::=
1]:55679 (ESTABLISHED)

slapd      2971      &nbs=
p;    root   25u  IPv6 0xffffff802288f91=
0      0t0    TCP [::1]:389->[::=
1]:55691 (ESTABLISHED)

slapd      2971      &nbs=
p;    root   26u  IPv6 0xffffff802288ecd=
0      0t0    TCP [::1]:389->[::=
1]:55776 (ESTABLISHED)

httpd      2976      &nbs=
p;     ctp   12u  IPv6 0xffffff8022=
88e6b0      0t0    TCP [::1]:55465-=
>[::1]:389 (ESTABLISHED)

httpd      2977      &nbs=
p;     ctp   12u  IPv6 0xffffff800f=
6083a0      0t0    TCP [::1]:55466-=
>[::1]:389 (ESTABLISHED)

httpd      3055      &nbs=
p;     ctp   12u  IPv6 0xffffff800f=
609c20      0t0    TCP [::1]:55621-=
>[::1]:389 (ESTABLISHED)

httpd      3185      &nbs=
p;     ctp   12u  IPv6 0xffffff800f=
609600      0t0    TCP [::1]:55666-=
>[::1]:389 (ESTABLISHED)

httpd      3198      &nbs=
p;     ctp   12u  IPv6 0xffffff8022=
890550      0t0    TCP [::1]:55691-=
>[::1]:389 (ESTABLISHED)

httpd      3199      &nbs=
p;     ctp   12u  IPv6 0xffffff8022=
890e80      0t0    TCP [::1]:55679-=
>[::1]:389 (ESTABLISHED)

httpd      3199      &nbs=
p;     ctp   13u  IPv6 0xffffff8022=
8917b0      0t0    TCP [::1]:57959-=
>[::1]:389 (ESTABLISHED)

httpd     28410       &nb=
sp;    ctp   12u  IPv6 0xffffff800f607a7=
0      0t0    TCP [::1]:56296->[=
::1]:389 (ESTABLISHED)



ps ax | grep httpd

2946   ?? =
 Ss     0:00.97 /usr/local/apache2/bin/httpd -k st=
art

 2947   ??  S      0:00.32 /us=
r/local/apache2/bin/httpd -k start

 2949   ??  S      0:00.35 /us=
r/local/apache2/bin/httpd -k start

 2976   ??  S      0:00.31 /us=
r/local/apache2/bin/httpd -k start

 2977   ??  S      0:00.30 /us=
r/local/apache2/bin/httpd -k start

 3055   ??  S      0:00.25 /us=
r/local/apache2/bin/httpd -k start

 3185   ??  S      0:00.24 /us=
r/local/apache2/bin/httpd -k start

 3198   ??  S      0:00.36 /us=
r/local/apache2/bin/httpd -k start

 3199   ??  S      0:00.24 /us=
r/local/apache2/bin/httpd -k start

28410   ??  S      0:00.08 /usr/loc=
al/apache2/bin/httpd -k start

28752   ??  S      0:00.02 /usr/loc=
al/apache2/bin/httpd -k start



Combining the two you can see,=
that some of the threads have a connection while others do not. And in the=
the apache error log I see:



could not lookup DNS =
configuration info service: (ipc/send) inval[Thu Jan 28 10:04:02 2010] [war=
n] [client 10.2.3.164] [28752] auth_ldap authenticate: user thsc authentica=
tion failed; URI xxx [LDAP: ldap_simple_bind_s() failed][Can't contact LDAP=
server], referer:



So the thread not serving requ=
est is the one which has no connection to the LDAP server and can’t g=
et it either. Other programs, proftd, ldapsearch, accessing the LDAP server=
are not experiencing any problems while this is happing.



Now if I shutdown the LDAP server and start it again, the apache server com=
es back into the same “state”, ie randomly failing. If I that a=
new snapshot of the ports open. all threads are able to reconnect to LDAP =
server, except the thread 28752 , which puts out the same log message again=
..



To see if it might have something to do with a specific version of the Open=
LDAP server, I have also done this with different versions of the OpenLDAP =
server., ie. Shutting down one version and starting up an older version of =
the LDAP server. The result is the same.



Last time I wrote that is seams to depend on the number of requests to the =
server. After my testing I more inclined to think it is a time issue. It se=
ems to be of less importance how much I stress the server. If I stress it a=
lot, it can run 1/2 to 1 hour before starting to fail. If I only make occa=
tional request I get from 1 hour up to 3 hours. If I start the server, make=
a few requests and leave it for the night, it will be in the partially fai=
led state in the morning.



I have the (nearly) same setup running on an OS X 10.4 server without any p=
roblems. I write nearly, because on the 10.4 machine it is an apache 2.0.49=
server, while I have an 2.2.14 apache server on the OS X 10.6 server. The =
ProFTPD server I use on the two setups is of the same version and it runs i=
nto the exact same problem on OX 10.6 as the Apache server does, with the s=
ame error message in it’s log files.



Does this make any sense ? After this I’m still inclined to suspect O=
S X 10.6 ...



Regards,



 Thomas Schneider





On 21/12/09 17:31, "Michael A. Pasek" < ael-pasek.com">mike001@michael-pasek.com> wrote:

>In users Digest Issue 3741 (21 Dec 2009 15:=
54:32 -0000), Thomas Scheider wrote:



> [problems with LDAP authentication on 10.6.2]

> After 5000+ SSL requests, users begin to be denied log-in to the web R>
> site. In the Apache error_log the following is written:

>

> [Mon Dec 21 09:14:23 2009] [info] Initial (No.1) HTTPS request receive=
d for child 6 (server 172.25.2.99:443)

> could not lookup DNS configuration info service: (ipc/send) invalid de=
stination port

> [...]

> On the same server I have a ProFTPD running which also does log-in

> verification against the LDAP server and retrieves varios information<=
BR>
> about the user. The program runs into the same problems, ie. the "=
;could

> not lookup  ..." begins to appear in the itøs log fil=
e, and users are

> refused access to the FTP server.

> [...]



This would imply that the problem lies not with Apache, but with either

the LDAP server or OS 10.6.2 (which is, I assume, the OS on which the

Apache and ProFTPd applications are running).



> The messages "could not lookup .." does not begin to appear =
at the same

> time in the logfiles. The apache may be running fine, while the FTP se=
rver

> is rejecting users, and wise versa.



I'd suspect that the connections to the LDAP server are not being "cle=
aned

up", and once the application reaches it's per-process file descriptor=
limit

it is being denied its request to open another network connection (i.e., >
allocate another file descriptor).  Check the output of:

  lsof -nPi | grep ":389"

(NOTE: You must execute this as "root" in order to see _all_ the =
connections)

This should show you all the current connections to the LDAP server, which<=
BR>
application/process is "controlling" that connection, and the cur=
rent

connection state.



You might also check the system.log; there may be entries in there if the R>
LDAP connection requests ARE being rejected due to the filedescriptor limit=
..



What to do next depends on:

  a) Whether my theory is correct; and,

  b) What state the connections are "hung" in.



Regards,



Michael A. Pasek