Apache Hangs.. Server-Status shows all Reading

Apache Hangs.. Server-Status shows all Reading

am 07.11.2007 18:31:58 von altendew

Hi this keeps happening a lot where my server will be unresponsive... it just
hangs forever.. so I checked the apache server-status and there was 131
requests that looked like this..

39-16 2177 0/67/114 R 0.95 47 562 0.0 0.48 0.66 ? ? ..reading..
40-16 29189 0/220/220 R 3.40 47 135 0.0 0.67 0.67 ? ? ..reading..
41-16 3959 0/7/111 R 0.21 48 81 0.0 0.01 0.42 ? ? ..reading..

They were all just in the "reading" state and i couldnt get an open slot nor
anyone else who was viewing our websites..

I restarted apache and all was fine.. but then 20 minutes later they went
back all into a reading state.. it appears as if slowly each processes goes
into the reading state?? I dont understand what the problem is.
--
View this message in context: http://www.nabble.com/Apache-Hangs..-Server-Status-shows-all -Reading-tf4766110.html#a13631744
Sent from the Apache HTTP Server - Users mailing list archive at Nabble.com.


------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Apache Hangs.. Server-Status shows all Reading

am 08.11.2007 08:30:46 von Christian Folini

Hey Andrew,

You have to try and isolate the problem.
It's a start to remove modules and make the issue
go away in a lab setup and thus identify the component
that is causing the problem. Try to nail down the
individual requests that cause a server process/thread
to hang.

Ideally mod_forensic should tell you about this
requests as the forensic log will tell you about incoming
requests before they are handled. But I am not sure
they are in the forensic log as your status suggests
they are still being read.

tcpdump outside of your apache could help here (start
with "tcpdump -A -s 0 port xxx" here. Unless it is
https traffic, then it would not tell you much.

regs,

Christian

On Wed, Nov 07, 2007 at 09:31:58AM -0800, Andrew Rosolino wrote:
>
> Hi this keeps happening a lot where my server will be unresponsive... it just
> hangs forever.. so I checked the apache server-status and there was 131
> requests that looked like this..
>
> 39-16 2177 0/67/114 R 0.95 47 562 0.0 0.48 0.66 ? ? ..reading..
> 40-16 29189 0/220/220 R 3.40 47 135 0.0 0.67 0.67 ? ? ..reading..
> 41-16 3959 0/7/111 R 0.21 48 81 0.0 0.01 0.42 ? ? ..reading..
>
> They were all just in the "reading" state and i couldnt get an open slot nor
> anyone else who was viewing our websites..
>
> I restarted apache and all was fine.. but then 20 minutes later they went
> back all into a reading state.. it appears as if slowly each processes goes
> into the reading state?? I dont understand what the problem is.
> --
> View this message in context: http://www.nabble.com/Apache-Hangs..-Server-Status-shows-all -Reading-tf4766110.html#a13631744
> Sent from the Apache HTTP Server - Users mailing list archive at Nabble.com.
>
>
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Apache Hangs.. Server-Status shows all Reading

am 08.11.2007 16:14:25 von altendew

Ok next time it happens ill try tcpdump -A -s 0 port 80 and let you know..
thank u.

Christian Folini-4 wrote:
>
> Hey Andrew,
>
> You have to try and isolate the problem.
> It's a start to remove modules and make the issue
> go away in a lab setup and thus identify the component
> that is causing the problem. Try to nail down the
> individual requests that cause a server process/thread
> to hang.
>
> Ideally mod_forensic should tell you about this
> requests as the forensic log will tell you about incoming
> requests before they are handled. But I am not sure
> they are in the forensic log as your status suggests
> they are still being read.
>
> tcpdump outside of your apache could help here (start
> with "tcpdump -A -s 0 port xxx" here. Unless it is
> https traffic, then it would not tell you much.
>
> regs,
>
> Christian
>
> On Wed, Nov 07, 2007 at 09:31:58AM -0800, Andrew Rosolino wrote:
>>
>> Hi this keeps happening a lot where my server will be unresponsive... it
>> just
>> hangs forever.. so I checked the apache server-status and there was 131
>> requests that looked like this..
>>
>> 39-16 2177 0/67/114 R 0.95 47 562 0.0 0.48 0.66 ? ? ..reading..
>> 40-16 29189 0/220/220 R 3.40 47 135 0.0 0.67 0.67 ? ? ..reading..
>> 41-16 3959 0/7/111 R 0.21 48 81 0.0 0.01 0.42 ? ? ..reading..
>>
>> They were all just in the "reading" state and i couldnt get an open slot
>> nor
>> anyone else who was viewing our websites..
>>
>> I restarted apache and all was fine.. but then 20 minutes later they went
>> back all into a reading state.. it appears as if slowly each processes
>> goes
>> into the reading state?? I dont understand what the problem is.
>> --
>> View this message in context:
>> http://www.nabble.com/Apache-Hangs..-Server-Status-shows-all -Reading-tf4766110.html#a13631744
>> Sent from the Apache HTTP Server - Users mailing list archive at
>> Nabble.com.
>>
>>
>> ------------------------------------------------------------ ---------
>> The official User-To-User support forum of the Apache HTTP Server
>> Project.
>> See for more info.
>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>> " from the digest: users-digest-unsubscribe@httpd.apache.org
>> For additional commands, e-mail: users-help@httpd.apache.org
>>
>
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>

--
View this message in context: http://www.nabble.com/Apache-Hangs..-Server-Status-shows-all -Reading-tf4766110.html#a13648781
Sent from the Apache HTTP Server - Users mailing list archive at Nabble.com.


------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Apache Hangs.. Server-Status shows all Reading

am 08.11.2007 19:27:59 von altendew

Ok I did it but how am I suppose to read this?

J5...3.P..w....8P...{.........
13:26:22.990526 IP alpha2.shiftcode.com.http > nat10.ekspres.net.pl.3891: .
ack 1 win 6432
E..(..@.@.G.J5..S..

.P.3...{..w.P.. \...
13:26:22.994371 IP nat10.ekspres.net.pl.3891 > alpha2.shiftcode.com.http: R
1:1(0) ack 1091 win 0
E..(0.@.r.~]S..

J5...3.P..w....zP...v.........
13:26:23.002050 IP n219077060129.netvigator.com.61317 >
alpha2.shiftcode.com.http: S 2990870443:2990870443(0) win 642
40

E..0.<@.v....M<.J5.....P.E......p...n...........
13:26:23.002083 IP alpha2.shiftcode.com.http >
n219077060129.netvigator.com.61317: S 3164683349:3164683349(0) ack 299
0870444 win 5840



Christian Folini-4 wrote:
>
> Hey Andrew,
>
> You have to try and isolate the problem.
> It's a start to remove modules and make the issue
> go away in a lab setup and thus identify the component
> that is causing the problem. Try to nail down the
> individual requests that cause a server process/thread
> to hang.
>
> Ideally mod_forensic should tell you about this
> requests as the forensic log will tell you about incoming
> requests before they are handled. But I am not sure
> they are in the forensic log as your status suggests
> they are still being read.
>
> tcpdump outside of your apache could help here (start
> with "tcpdump -A -s 0 port xxx" here. Unless it is
> https traffic, then it would not tell you much.
>
> regs,
>
> Christian
>
> On Wed, Nov 07, 2007 at 09:31:58AM -0800, Andrew Rosolino wrote:
>>
>> Hi this keeps happening a lot where my server will be unresponsive... it
>> just
>> hangs forever.. so I checked the apache server-status and there was 131
>> requests that looked like this..
>>
>> 39-16 2177 0/67/114 R 0.95 47 562 0.0 0.48 0.66 ? ? ..reading..
>> 40-16 29189 0/220/220 R 3.40 47 135 0.0 0.67 0.67 ? ? ..reading..
>> 41-16 3959 0/7/111 R 0.21 48 81 0.0 0.01 0.42 ? ? ..reading..
>>
>> They were all just in the "reading" state and i couldnt get an open slot
>> nor
>> anyone else who was viewing our websites..
>>
>> I restarted apache and all was fine.. but then 20 minutes later they went
>> back all into a reading state.. it appears as if slowly each processes
>> goes
>> into the reading state?? I dont understand what the problem is.
>> --
>> View this message in context:
>> http://www.nabble.com/Apache-Hangs..-Server-Status-shows-all -Reading-tf4766110.html#a13631744
>> Sent from the Apache HTTP Server - Users mailing list archive at
>> Nabble.com.
>>
>>
>> ------------------------------------------------------------ ---------
>> The official User-To-User support forum of the Apache HTTP Server
>> Project.
>> See for more info.
>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>> " from the digest: users-digest-unsubscribe@httpd.apache.org
>> For additional commands, e-mail: users-help@httpd.apache.org
>>
>
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>

--
View this message in context: http://www.nabble.com/Apache-Hangs..-Server-Status-shows-all -Reading-tf4766110.html#a13652832
Sent from the Apache HTTP Server - Users mailing list archive at Nabble.com.


------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Apache Hangs.. Server-Status shows all Reading

am 09.11.2007 07:48:28 von Christian Folini

On Thu, Nov 08, 2007 at 10:27:59AM -0800, Andrew Rosolino wrote:
>
> Ok I did it but how am I suppose to read this?

That's not the interesting part. There should be more data.

Below is an example of a simple GET request
in a local setup. You can clearly see the request.
You can also take this into wireshark/ethereal and
select "follow tcp stream" or so.

regs,

Christian

07:45:47.905305 IP wcwe003u.pnet.ch.58702 > w032y7.pnet.ch.www: S 318106195:318106195(0) win 5840
E..<..@.@.I.
...d
.....N.P...S...................

2
3.........
07:45:47.905345 IP w032y7.pnet.ch.www > wcwe003u.pnet.ch.58702: S 4288032982:4288032982(0) ack 318106196 win 5792
E..<..@.@.#.
....
...d.P.N..0....T...............

%`..2
3.....
07:45:47.905543 IP wcwe003u.pnet.ch.58702 > w032y7.pnet.ch.www: . ack 1 win 46
E..4..@.@.I.
...d
.....N.P...T..0............

2
3.%`..
07:45:47.905880 IP wcwe003u.pnet.ch.58702 > w032y7.pnet.ch.www: P 1:180(179) ack 1 win 46
E.....@.@.I.
...d
.....N.P...T..0.....P......
2
3.%`..GET /index.html HTTP/1.1
User-Agent: curl/7.13.2 (i386-pc-linux-gnu) libcurl/7.13.2 OpenSSL/0.9.7e zlib/1.2.2 libidn/0.5.13
Host: 10.226.0.220
Pragma: no-cache
Accept: */*



07:45:47.905888 IP w032y7.pnet.ch.www > wcwe003u.pnet.ch.58702: . ack 180 win 54
E..4..@.@...
....
...d.P.N..0........6.K.....

%`..2
3.
07:45:47.906504 IP w032y7.pnet.ch.www > wcwe003u.pnet.ch.58702: P 1:245(244) ack 180 win 54
E..(..@.@...
....
...d.P.N..0........6.......
%`..2
3.HTTP/1.1 200 OK
Date: Fri, 09 Nov 2007 06:45:47 GMT
Server: Apache
Last-Modified: Thu, 08 Nov 2007 11:29:36 GMT
ETag: "31101-d-2e4b0800"
Accept-Ranges: bytes
Content-Length: 13
Connection: close
Content-Type: text/plain

hello world!

07:45:47.906720 IP wcwe003u.pnet.ch.58702 > w032y7.pnet.ch.www: . ack 245 win 54
E..4..@.@.I.
...d
.....N.P......1....6
W.....

2
3.%`..
07:45:47.906797 IP w032y7.pnet.ch.www > wcwe003u.pnet.ch.58702: F 245:245(0) ack 180 win 54
E..4..@.@...
....
...d.P.N..1........6
V.....

%`..2
3.
07:45:47.908037 IP wcwe003u.pnet.ch.58702 > w032y7.pnet.ch.www: F 180:180(0) ack 246 win 54
E..4..@.@.I.
...d
.....N.P......1....6
U.....

2
3.%`..
07:45:47.908046 IP w032y7.pnet.ch.www > wcwe003u.pnet.ch.58702: . ack 181 win 54
E..4..@.@...
....
...d.P.N..1........6
T.....

%`. 2
3.




------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Apache Hangs.. Server-Status shows all Reading

am 09.11.2007 08:44:46 von altendew

I saved the file with the full output do you want to see that.. i only took a
section from it.


Christian Folini-4 wrote:
>
> On Thu, Nov 08, 2007 at 10:27:59AM -0800, Andrew Rosolino wrote:
>>
>> Ok I did it but how am I suppose to read this?
>
> That's not the interesting part. There should be more data.
>
> Below is an example of a simple GET request
> in a local setup. You can clearly see the request.
> You can also take this into wireshark/ethereal and
> select "follow tcp stream" or so.
>
> regs,
>
> Christian
>
> 07:45:47.905305 IP wcwe003u.pnet.ch.58702 > w032y7.pnet.ch.www: S
> 318106195:318106195(0) win 5840 > 0,nop,wscale 7>
> E..<..@.@.I.
> ..d
> ....N.P...S...................
>
> 2
> 3.........
> 07:45:47.905345 IP w032y7.pnet.ch.www > wcwe003u.pnet.ch.58702: S
> 4288032982:4288032982(0) ack 318106196 win 5792 > 627094047 839725985,nop,wscale 7>
> E..<..@.@.#.
> ...
> ..d.P.N..0....T...............
>
> %`..2
> 3.....
> 07:45:47.905543 IP wcwe003u.pnet.ch.58702 > w032y7.pnet.ch.www: . ack 1
> win 46
> E..4..@.@.I.
> ..d
> ....N.P...T..0............
>
> 2
> 3.%`..
> 07:45:47.905880 IP wcwe003u.pnet.ch.58702 > w032y7.pnet.ch.www: P
> 1:180(179) ack 1 win 46
> E.....@.@.I.
> ..d
> ....N.P...T..0.....P......
> 2
> 3.%`..GET /index.html HTTP/1.1
> User-Agent: curl/7.13.2 (i386-pc-linux-gnu) libcurl/7.13.2 OpenSSL/0.9.7e
> zlib/1.2.2 libidn/0.5.13
> Host: 10.226.0.220
> Pragma: no-cache
> Accept: */*
>
>
>
> 07:45:47.905888 IP w032y7.pnet.ch.www > wcwe003u.pnet.ch.58702: . ack 180
> win 54
> E..4..@.@...
> ...
> ..d.P.N..0........6.K.....
>
> %`..2
> 3.
> 07:45:47.906504 IP w032y7.pnet.ch.www > wcwe003u.pnet.ch.58702: P
> 1:245(244) ack 180 win 54
> E..(..@.@...
> ...
> ..d.P.N..0........6.......
> %`..2
> 3.HTTP/1.1 200 OK
> Date: Fri, 09 Nov 2007 06:45:47 GMT
> Server: Apache
> Last-Modified: Thu, 08 Nov 2007 11:29:36 GMT
> ETag: "31101-d-2e4b0800"
> Accept-Ranges: bytes
> Content-Length: 13
> Connection: close
> Content-Type: text/plain
>
> hello world!
>
> 07:45:47.906720 IP wcwe003u.pnet.ch.58702 > w032y7.pnet.ch.www: . ack 245
> win 54
> E..4..@.@.I.
> ..d
> ....N.P......1....6
> W.....
>
> 2
> 3.%`..
> 07:45:47.906797 IP w032y7.pnet.ch.www > wcwe003u.pnet.ch.58702: F
> 245:245(0) ack 180 win 54
> E..4..@.@...
> ...
> ..d.P.N..1........6
> V.....
>
> %`..2
> 3.
> 07:45:47.908037 IP wcwe003u.pnet.ch.58702 > w032y7.pnet.ch.www: F
> 180:180(0) ack 246 win 54
> E..4..@.@.I.
> ..d
> ....N.P......1....6
> U.....
>
> 2
> 3.%`..
> 07:45:47.908046 IP w032y7.pnet.ch.www > wcwe003u.pnet.ch.58702: . ack 181
> win 54
> E..4..@.@...
> ...
> ..d.P.N..1........6
> T.....
>
> %`. 2
> 3.
>
>
>
>
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>

--
View this message in context: http://www.nabble.com/Apache-Hangs..-Server-Status-shows-all -Reading-tf4766110.html#a13662733
Sent from the Apache HTTP Server - Users mailing list archive at Nabble.com.


------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Apache Hangs.. Server-Status shows all Reading

am 10.11.2007 04:36:51 von Sander Temme

On Nov 7, 2007, at 9:31 AM, Andrew Rosolino wrote:

> I restarted apache and all was fine.. but then 20 minutes later
> they went
> back all into a reading state.. it appears as if slowly each
> processes goes
> into the reading state?? I dont understand what the problem is.

If you have gdb on the box (and have some acumen using it), you can
attach to one of the processes and see if it is hanging in something
obvious.

If not, and your traffic is plaintext, I recommend the tcpdump
approach to see what is going on traffic-wise, and what triggers the
issue. I'd dump the traffic to disk (tcpdump -i ethX -s 0 -w /
wherever/traffic.dump port 80) and take it over to a box with
Ethereal/Wireshark for some cozy analysis.

If you're being DOS attacked by trickle requests, you could try
setting a very low timeout (default is 5 minutes which doesn't seem
to be working for you) and perhaps use mod_evasive or somesuch to
flag and firewall the bad clients.

S.

--
Sander Temme
sctemme@apache.org
PGP FP: 51B4 8727 466A 0BC3 69F4 B7B8 B2BE BC40 1529 24AF




------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Apache Hangs.. Server-Status shows all Reading

am 10.11.2007 14:20:53 von alf

Hi

This seems to be quite similar to what I have seen, and others have also
posted about the last couple of years.
I've done a lot of investigation, but have not found a real solution.
My problem is that the threads gets stuck in "W" / "Sending reply" state.

Are you using mod_php ? If so, it might be that one thread is doing
work, while the other threads are waiting to acquire a lock on the php
session file.
When PHP session files are used, the request are processed sequentially,
and not in parallel for a user.

I suggest you also use "strace" unix command to see what the different
threads are doing, that might be easier than using gdb.
When I use strace, I see that one thread is hanging on "poll" system
call, until the timeout (Timeout Apache directive - default 300 seconds)
occur. Then I see that the next thread hangs in "poll" and so on.
So if you set the "Timeout" lower, it might improve your situation.

You should also run "netstat", and see what the status of the sockets
are. If they are almost all in CLOSE_WAIT, I think you are seeing
something similar to me.

Here are some links to similar issues, I'm not sure if it will help you :
http://www.nabble.com/RE:-requests-time-out-under-load,-no-w arnings-in-logs-p13065495.html
http://mail-archives.apache.org/mod_mbox/httpd-users/200611. mbox/%3c20061114084333.GC10809@nexus.intra.coretrek.com%3e
( and the other posts to that thread)
http://www.linuxforums.org/forum/servers/27519-alright-im-dy ing-here-apache-help-mysql-php.html
http://osdir.com/ml/apache.mod-ssl.user/2004-10/msg00009.htm l

I have also seen other posts from people with very similar problems,
lots of thread stuck in a state.

What is your environment ? I.e. what version of Apache, what operating
system and version, are you generating dynamic pages, and if so, what
tools (for example php) are you using ?
Do you have a firewall or a load balancer in front of your web server ?

Are almost all the requests from the same IP adresse ? Are almost all
the request for the same URL ?

What my theory is at the moment, after throwing away some other
theories, are that somehow a client creates several hundred requests for
the same URL.
Since in my case this happens fairly often, several times a day on a
uncontroversial high traffic site, from different users, I do not think
it is someone doing a DoS attack.

I think it might be a problem with a firewall or a proxy or maybe the
browser (I have seen it happen for people using both Windows and Mac,
both Firefox and IE), that sits between the web server and the end user.
Somehow the closing of the socket/connection by the browser is not
picked up by the web server. And somehow the browser issues lots of
requests.
I have not completely ruled out that it might be a Apache web server bug
as well.
But since I have no way of reproducing this, apart from observing it
happening quite often, it is very hard to get any further.
So any help would be greatly appreciated.

Regards
Alf Hogemark



Sander Temme wrote:
>
> On Nov 7, 2007, at 9:31 AM, Andrew Rosolino wrote:
>
>> I restarted apache and all was fine.. but then 20 minutes later they
>> went
>> back all into a reading state.. it appears as if slowly each
>> processes goes
>> into the reading state?? I dont understand what the problem is.
>
> If you have gdb on the box (and have some acumen using it), you can
> attach to one of the processes and see if it is hanging in something
> obvious.
>
> If not, and your traffic is plaintext, I recommend the tcpdump
> approach to see what is going on traffic-wise, and what triggers the
> issue. I'd dump the traffic to disk (tcpdump -i ethX -s 0 -w
> /wherever/traffic.dump port 80) and take it over to a box with
> Ethereal/Wireshark for some cozy analysis.
>
> If you're being DOS attacked by trickle requests, you could try
> setting a very low timeout (default is 5 minutes which doesn't seem to
> be working for you) and perhaps use mod_evasive or somesuch to flag
> and firewall the bad clients.
>
> S.
>

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Apache Hangs.. Server-Status shows all Reading

am 11.11.2007 17:31:42 von altendew

Ok I used gdb to a process (that was stuck in reading state) and look at what
it told me..

#0 0x003307a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1 0x00408dbd in poll () from /lib/tls/libc.so.6
#2 0x00af595f in apr_poll (aprset=0xbfe55310, num=1, nsds=0xbfe5530c,
timeout=300000) at poll.c:130
#3 0x00af5d5f in apr_wait_for_io_or_timeout (f=0x0, s=0x959b4a8,
for_read=1)
at waitio.c:54
#4 0x00aec19c in apr_socket_recv (sock=0x959b4a8,
buf=0x9624688 "\n# AddHandler application/x-httpd-php .php
..sc\r\nIndexIgnore .htaccess */.??* *~ *# */HEADER* */README*
*/_vti*\r\nDirectoryIndex home.gc index.gc index.php\r\n\r\n POST>\r\n#The next line modified b"...,
len=0xbfe553e8) at sendrecv.c:87
#5 0x00fcc0dc in socket_bucket_read (a=0x95c0468, str=0xbfe553e4,
len=0xbfe553e8, block=APR_BLOCK_READ) at apr_buckets_socket.c:36
#6 0x00fcd06a in apr_brigade_split_line (bbOut=0x95d0f50, bbIn=0x959b9a0,
block=APR_BLOCK_READ, maxbytes=8192) at apr_brigade.c:289
#7 0x080b9a9b in core_input_filter (f=0x959b968, b=0x95d0f50,
mode=AP_MODE_GETLINE, block=APR_BLOCK_READ, readbytes=0) at core.c:3802
#8 0x0806e8db in logio_in_filter (f=0x959b928, bb=0x95d0f50,
mode=AP_MODE_GETLINE, block=APR_BLOCK_READ, readbytes=0) at
mod_logio.c:115
#9 0x080b3ae3 in ap_rgetline_core (s=0x95d02a0, n=8192, read=0xbfe55528,
r=0x95d0288, fold=0, bb=0x95d0f50) at protocol.c:230
#10 0x080b4577 in ap_read_request (conn=0x959b580) at protocol.c:586
#11 0x0808bc6e in ap_process_http_connection (c=0x959b580) at
http_core.c:271
---Type to continue, or q to quit---
#12 0x080b0a62 in ap_run_process_connection (c=0x959b580) at connection.c:43
#13 0x080a5ad1 in child_main (child_num_arg=Variable "child_num_arg" is not
available.
) at prefork.c:610
#14 0x080a5cfb in make_child (s=Variable "s" is not available.
) at prefork.c:704
#15 0x080a5d8c in startup_children (number_to_start=5) at prefork.c:722
#16 0x080a645f in ap_mpm_run (_pconf=0x8c2c0a8, plog=0x8c6c1a8, s=0x8c32ed0)
at prefork.c:941
#17 0x080ab803 in main (argc=4, argv=0xbfe558c4) at main.c:638


Sander Temme-2 wrote:
>
>
> On Nov 7, 2007, at 9:31 AM, Andrew Rosolino wrote:
>
>> I restarted apache and all was fine.. but then 20 minutes later
>> they went
>> back all into a reading state.. it appears as if slowly each
>> processes goes
>> into the reading state?? I dont understand what the problem is.
>
> If you have gdb on the box (and have some acumen using it), you can
> attach to one of the processes and see if it is hanging in something
> obvious.
>
> If not, and your traffic is plaintext, I recommend the tcpdump
> approach to see what is going on traffic-wise, and what triggers the
> issue. I'd dump the traffic to disk (tcpdump -i ethX -s 0 -w /
> wherever/traffic.dump port 80) and take it over to a box with
> Ethereal/Wireshark for some cozy analysis.
>
> If you're being DOS attacked by trickle requests, you could try
> setting a very low timeout (default is 5 minutes which doesn't seem
> to be working for you) and perhaps use mod_evasive or somesuch to
> flag and firewall the bad clients.
>
> S.
>
> --
> Sander Temme
> sctemme@apache.org
> PGP FP: 51B4 8727 466A 0BC3 69F4 B7B8 B2BE BC40 1529 24AF
>
>
>
>
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>

--
View this message in context: http://www.nabble.com/Apache-Hangs..-Server-Status-shows-all -Reading-tf4766110.html#a13693055
Sent from the Apache HTTP Server - Users mailing list archive at Nabble.com.


------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Apache Hangs.. Server-Status shows all Reading

am 11.11.2007 17:49:07 von altendew

Im using this for php..
LoadModule php4_module modules/libphp4.so

not sure if that is mod_php or what

I tried using strace but my server was going extremly slow when I started i=
t
probably because we get so much traffic so i couldnt debug it...

I did run the gdb on a process and got this..=20
#0 0x003307a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2=20
#1 0x00408dbd in poll () from /lib/tls/libc.so.6=20
#2 0x00af595f in apr_poll (aprset=3D0xbfe55310, num=3D1, nsds=3D0xbfe5530c=
,=20
timeout=3D300000) at poll.c:130=20
#3 0x00af5d5f in apr_wait_for_io_or_timeout (f=3D0x0, s=3D0x959b4a8,
for_read=3D1)=20
at waitio.c:54=20
#4 0x00aec19c in apr_socket_recv (sock=3D0x959b4a8,=20
buf=3D0x9624688 "\n# AddHandler application/x-httpd-php .php
..sc\r\nIndexIgnore .htaccess */.??* *~ *# */HEADER* */README*
*/_vti*\r\nDirectoryIndex home.gc index.gc index.php\r\n\r\n POST>\r\n#The next line modified b"...,=20
len=3D0xbfe553e8) at sendrecv.c:87=20
#5 0x00fcc0dc in socket_bucket_read (a=3D0x95c0468, str=3D0xbfe553e4,=20
len=3D0xbfe553e8, block=3DAPR_BLOCK_READ) at apr_buckets_socket.c:36=20
#6 0x00fcd06a in apr_brigade_split_line (bbOut=3D0x95d0f50, bbIn=3D0x959b9=
a0,=20
block=3DAPR_BLOCK_READ, maxbytes=3D8192) at apr_brigade.c:289=20
#7 0x080b9a9b in core_input_filter (f=3D0x959b968, b=3D0x95d0f50,=20
mode=3DAP_MODE_GETLINE, block=3DAPR_BLOCK_READ, readbytes=3D0) at core.=
c:3802=20
#8 0x0806e8db in logio_in_filter (f=3D0x959b928, bb=3D0x95d0f50,=20
mode=3DAP_MODE_GETLINE, block=3DAPR_BLOCK_READ, readbytes=3D0) at
mod_logio.c:115=20
#9 0x080b3ae3 in ap_rgetline_core (s=3D0x95d02a0, n=3D8192, read=3D0xbfe55=
528,=20
r=3D0x95d0288, fold=3D0, bb=3D0x95d0f50) at protocol.c:230=20
#10 0x080b4577 in ap_read_request (conn=3D0x959b580) at protocol.c:586=20
#11 0x0808bc6e in ap_process_http_connection (c=3D0x959b580) at
http_core.c:271=20
---Type to continue, or q to quit---=20
#12 0x080b0a62 in ap_run_process_connection (c=3D0x959b580) at connection.c=
:43=20
#13 0x080a5ad1 in child_main (child_num_arg=3DVariable "child_num_arg" is n=
ot
available.=20
) at prefork.c:610=20
#14 0x080a5cfb in make_child (s=3DVariable "s" is not available.=20
) at prefork.c:704=20
#15 0x080a5d8c in startup_children (number_to_start=3D5) at prefork.c:722=
=20
#16 0x080a645f in ap_mpm_run (_pconf=3D0x8c2c0a8, plog=3D0x8c6c1a8, s=3D0x8=
c32ed0)=20
at prefork.c:941=20
#17 0x080ab803 in main (argc=3D4, argv=3D0xbfe558c4) at main.c:638=20

havent tried netstat yet because the problem isnt occuring right now..
Here is some info on the Apache we are using:
Server Version: Apache/2.0.61 (Unix) mod_ssl/2.0.61 OpenSSL/0.9.7a
mod_auth_passthrough/2.1 FrontPage/5.0.2.2635 mod_bwlimited/1.4=20

We are running REDHAT Enterprise 4 i686 almost all our pages are dynamic
php.. and its running about 200 websites.. we dont have a load balancer..=
=20

All the requests are not from the same ip address very random.


Alf Høgemark wrote:
>=20
> Hi
>=20
> This seems to be quite similar to what I have seen, and others have also=
=20
> posted about the last couple of years.
> I've done a lot of investigation, but have not found a real solution.
> My problem is that the threads gets stuck in "W" / "Sending reply" state.
>=20
> Are you using mod_php ? If so, it might be that one thread is doing=20
> work, while the other threads are waiting to acquire a lock on the php=20
> session file.
> When PHP session files are used, the request are processed sequentially,=
=20
> and not in parallel for a user.
>=20
> I suggest you also use "strace" unix command to see what the different=20
> threads are doing, that might be easier than using gdb.
> When I use strace, I see that one thread is hanging on "poll" system=20
> call, until the timeout (Timeout Apache directive - default 300 seconds)=
=20
> occur. Then I see that the next thread hangs in "poll" and so on.
> So if you set the "Timeout" lower, it might improve your situation.
>=20
> You should also run "netstat", and see what the status of the sockets=20
> are. If they are almost all in CLOSE_WAIT, I think you are seeing=20
> something similar to me.
>=20
> Here are some links to similar issues, I'm not sure if it will help you :
> http://www.nabble.com/RE:-requests-time-out-under-load,-no-w arnings-in-lo=
gs-p13065495.html
> http://mail-archives.apache.org/mod_mbox/httpd-users/200611. mbox/%3c20061=
114084333.GC10809@nexus.intra.coretrek.com%3e=20
> ( and the other posts to that thread)
> http://www.linuxforums.org/forum/servers/27519-alright-im-dy ing-here-apac=
he-help-mysql-php.html
> http://osdir.com/ml/apache.mod-ssl.user/2004-10/msg00009.htm l
>=20
> I have also seen other posts from people with very similar problems,=20
> lots of thread stuck in a state.
>=20
> What is your environment ? I.e. what version of Apache, what operating=20
> system and version, are you generating dynamic pages, and if so, what=20
> tools (for example php) are you using ?
> Do you have a firewall or a load balancer in front of your web server ?
>=20
> Are almost all the requests from the same IP adresse ? Are almost all=20
> the request for the same URL ?
>=20
> What my theory is at the moment, after throwing away some other=20
> theories, are that somehow a client creates several hundred requests for=
=20
> the same URL.
> Since in my case this happens fairly often, several times a day on a=20
> uncontroversial high traffic site, from different users, I do not think=
=20
> it is someone doing a DoS attack.
>=20
> I think it might be a problem with a firewall or a proxy or maybe the=20
> browser (I have seen it happen for people using both Windows and Mac,=20
> both Firefox and IE), that sits between the web server and the end user.=
=20
> Somehow the closing of the socket/connection by the browser is not=20
> picked up by the web server. And somehow the browser issues lots of=20
> requests.
> I have not completely ruled out that it might be a Apache web server bug=
=20
> as well.
> But since I have no way of reproducing this, apart from observing it=20
> happening quite often, it is very hard to get any further.
> So any help would be greatly appreciated.
>=20
> Regards
> Alf Hogemark
>=20
>=20
>=20
> Sander Temme wrote:
>>
>> On Nov 7, 2007, at 9:31 AM, Andrew Rosolino wrote:
>>
>>> I restarted apache and all was fine.. but then 20 minutes later they=20
>>> went
>>> back all into a reading state.. it appears as if slowly each=20
>>> processes goes
>>> into the reading state?? I dont understand what the problem is.
>>
>> If you have gdb on the box (and have some acumen using it), you can=20
>> attach to one of the processes and see if it is hanging in something=20
>> obvious.
>>
>> If not, and your traffic is plaintext, I recommend the tcpdump=20
>> approach to see what is going on traffic-wise, and what triggers the=20
>> issue. I'd dump the traffic to disk (tcpdump -i ethX -s 0 -w=20
>> /wherever/traffic.dump port 80) and take it over to a box with=20
>> Ethereal/Wireshark for some cozy analysis.
>>
>> If you're being DOS attacked by trickle requests, you could try=20
>> setting a very low timeout (default is 5 minutes which doesn't seem to=
=20
>> be working for you) and perhaps use mod_evasive or somesuch to flag=20
>> and firewall the bad clients.
>>
>> S.
>>
>=20
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project=
..
> See for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>=20
>=20
>=20

--=20
View this message in context: http://www.nabble.com/Apache-Hangs..-Server-S=
tatus-shows-all-Reading-tf4766110.html#a13693182
Sent from the Apache HTTP Server - Users mailing list archive at Nabble.com=
..


------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Apache Hangs.. Server-Status shows all Reading

am 26.11.2007 16:49:25 von JEROME

SSd2ZSBnb3QgaGVyZSBleGFjdGx5IHRoZSBzYW1lIHByb2JsZW0sIGJ1dCB3 aXRob3V0IGFueSBw
aHAgbW9kdWxlcy4KTXkgc2VydmVyIHNlcnZlcyBvbmx5IHN0YXRpYyByZXNv dXJjZXMgYW5kIEl0
J3MgZmxvb2RlZCB3aXRoIHJlYWRpbmcKc3RhdGUgcmVxdWVzdCBzb21ldGlt ZS4KU29tZXRpbWUg
dGhlcmUncyBtb3JlIHRoYW4gODAgcmVxdWVzdCB3YWl0aW5nIGluIHJlYWRp bmcgc3RhdGUgLi4K
ClRoZSBvbmx5IHR1cm5hcm91bmQgSSBmb3VuZCBpcyB0byBsb3dlciB0aGUg c2VydmVyIHRpbWVv
dXQgKHRvIDEwcykKYW5kIGluY3JlYXNlIHRoZSBudW1iZXIgb2YgYXZhaWxh YmxlIHByb2Nlc3Nl
cy4gQXMgc29vbiBhcyBJJ20gc2VydmluZwpvbmx5IHN0YXRpYyBzdHVmZiwg aXQgc2VlbXMgdG8g
YmUgb2suCgoKT24gTm92IDExLCAyMDA3IDQ6NDkgUE0sIEFuZHJldyBSb3Nv bGlubyA8YW5kcmV3
QHNoaWZ0Y29kZS5jb20+IHdyb3RlOgo+Cj4gSW0gdXNpbmcgdGhpcyBmb3Ig cGhwLi4KPiBMb2Fk
TW9kdWxlIHBocDRfbW9kdWxlIG1vZHVsZXMvbGlicGhwNC5zbwo+Cj4gbm90 IHN1cmUgaWYgdGhh
dCBpcyBtb2RfcGhwIG9yIHdoYXQKPgo+IEkgdHJpZWQgdXNpbmcgc3RyYWNl IGJ1dCBteSBzZXJ2
ZXIgd2FzIGdvaW5nIGV4dHJlbWx5IHNsb3cgd2hlbiBJIHN0YXJ0ZWQgaXQK PiBwcm9iYWJseSBi
ZWNhdXNlIHdlIGdldCBzbyBtdWNoIHRyYWZmaWMgc28gaSBjb3VsZG50IGRl YnVnIGl0Li4uCj4K
PiBJIGRpZCBydW4gdGhlIGdkYiBvbiBhIHByb2Nlc3MgYW5kIGdvdCB0aGlz Li4KPgo+ICMwICAw
eDAwMzMwN2EyIGluIF9kbF9zeXNpbmZvX2ludDgwICgpIGZyb20gL2xpYi9s ZC1saW51eC5zby4y
Cj4gIzEgIDB4MDA0MDhkYmQgaW4gcG9sbCAoKSBmcm9tIC9saWIvdGxzL2xp YmMuc28uNgo+ICMy
ICAweDAwYWY1OTVmIGluIGFwcl9wb2xsIChhcHJzZXQ9MHhiZmU1NTMxMCwg bnVtPTEsIG5zZHM9
MHhiZmU1NTMwYywKPiAgICAgdGltZW91dD0zMDAwMDApIGF0IHBvbGwuYzox MzAKPiAjMyAgMHgw
MGFmNWQ1ZiBpbiBhcHJfd2FpdF9mb3JfaW9fb3JfdGltZW91dCAoZj0weDAs IHM9MHg5NTliNGE4
LAo+IGZvcl9yZWFkPTEpCj4gICAgIGF0IHdhaXRpby5jOjU0Cj4gIzQgIDB4 MDBhZWMxOWMgaW4g
YXByX3NvY2tldF9yZWN2IChzb2NrPTB4OTU5YjRhOCwKPiAgICAgYnVmPTB4 OTYyNDY4OCAiXG4j
IEFkZEhhbmRsZXIgYXBwbGljYXRpb24veC1odHRwZC1waHAgLnBocAo+IC5z Y1xyXG5JbmRleEln
bm9yZSAuaHRhY2Nlc3MgKi8uPz8qICp+ICojICovSEVBREVSKiAqL1JFQURN RSoKPiAqL192dGkq
XHJcbkRpcmVjdG9yeUluZGV4IGhvbWUuZ2MgaW5kZXguZ2MgaW5kZXgucGhw XHJcblxyXG48TGlt
aXQgR0VUCj4gUE9TVD5cclxuI1RoZSBuZXh0IGxpbmUgbW9kaWZpZWQgYiIu Li4sCj4gICAgIGxl
bj0weGJmZTU1M2U4KSBhdCBzZW5kcmVjdi5jOjg3Cj4gIzUgIDB4MDBmY2Mw ZGMgaW4gc29ja2V0
X2J1Y2tldF9yZWFkIChhPTB4OTVjMDQ2OCwgc3RyPTB4YmZlNTUzZTQsCj4g ICAgIGxlbj0weGJm
ZTU1M2U4LCBibG9jaz1BUFJfQkxPQ0tfUkVBRCkgYXQgYXByX2J1Y2tldHNf c29ja2V0LmM6MzYK
PiAjNiAgMHgwMGZjZDA2YSBpbiBhcHJfYnJpZ2FkZV9zcGxpdF9saW5lIChi Yk91dD0weDk1ZDBm
NTAsIGJiSW49MHg5NTliOWEwLAo+ICAgICBibG9jaz1BUFJfQkxPQ0tfUkVB RCwgbWF4Ynl0ZXM9
ODE5MikgYXQgYXByX2JyaWdhZGUuYzoyODkKPiAjNyAgMHgwODBiOWE5YiBp biBjb3JlX2lucHV0
X2ZpbHRlciAoZj0weDk1OWI5NjgsIGI9MHg5NWQwZjUwLAo+ICAgICBtb2Rl PUFQX01PREVfR0VU
TElORSwgYmxvY2s9QVBSX0JMT0NLX1JFQUQsIHJlYWRieXRlcz0wKSBhdCBj b3JlLmM6MzgwMgo+
ICM4ICAweDA4MDZlOGRiIGluIGxvZ2lvX2luX2ZpbHRlciAoZj0weDk1OWI5 MjgsIGJiPTB4OTVk
MGY1MCwKPiAgICAgbW9kZT1BUF9NT0RFX0dFVExJTkUsIGJsb2NrPUFQUl9C TE9DS19SRUFELCBy
ZWFkYnl0ZXM9MCkgYXQKPiBtb2RfbG9naW8uYzoxMTUKPiAjOSAgMHgwODBi M2FlMyBpbiBhcF9y
Z2V0bGluZV9jb3JlIChzPTB4OTVkMDJhMCwgbj04MTkyLCByZWFkPTB4YmZl NTU1MjgsCj4gICAg
IHI9MHg5NWQwMjg4LCBmb2xkPTAsIGJiPTB4OTVkMGY1MCkgYXQgcHJvdG9j b2wuYzoyMzAKPiAj
MTAgMHgwODBiNDU3NyBpbiBhcF9yZWFkX3JlcXVlc3QgKGNvbm49MHg5NTli NTgwKSBhdCBwcm90
b2NvbC5jOjU4Ngo+ICMxMSAweDA4MDhiYzZlIGluIGFwX3Byb2Nlc3NfaHR0 cF9jb25uZWN0aW9u
IChjPTB4OTU5YjU4MCkgYXQKPiBodHRwX2NvcmUuYzoyNzEKPiAtLS1UeXBl IDxyZXR1cm4+IHRv
IGNvbnRpbnVlLCBvciBxIDxyZXR1cm4+IHRvIHF1aXQtLS0KPiAjMTIgMHgw ODBiMGE2MiBpbiBh
cF9ydW5fcHJvY2Vzc19jb25uZWN0aW9uIChjPTB4OTU5YjU4MCkgYXQgY29u bmVjdGlvbi5jOjQz
Cj4gIzEzIDB4MDgwYTVhZDEgaW4gY2hpbGRfbWFpbiAoY2hpbGRfbnVtX2Fy Zz1WYXJpYWJsZSAi
Y2hpbGRfbnVtX2FyZyIgaXMgbm90Cj4gYXZhaWxhYmxlLgo+ICkgYXQgcHJl Zm9yay5jOjYxMAo+
ICMxNCAweDA4MGE1Y2ZiIGluIG1ha2VfY2hpbGQgKHM9VmFyaWFibGUgInMi IGlzIG5vdCBhdmFp
bGFibGUuCj4gKSBhdCBwcmVmb3JrLmM6NzA0Cj4gIzE1IDB4MDgwYTVkOGMg aW4gc3RhcnR1cF9j
aGlsZHJlbiAobnVtYmVyX3RvX3N0YXJ0PTUpIGF0IHByZWZvcmsuYzo3MjIK PiAjMTYgMHgwODBh
NjQ1ZiBpbiBhcF9tcG1fcnVuIChfcGNvbmY9MHg4YzJjMGE4LCBwbG9nPTB4 OGM2YzFhOCwgcz0w
eDhjMzJlZDApCj4gICAgIGF0IHByZWZvcmsuYzo5NDEKPiAjMTcgMHgwODBh YjgwMyBpbiBtYWlu
IChhcmdjPTQsIGFyZ3Y9MHhiZmU1NThjNCkgYXQgbWFpbi5jOjYzOAo+Cj4g aGF2ZW50IHRyaWVk
IG5ldHN0YXQgeWV0IGJlY2F1c2UgdGhlIHByb2JsZW0gaXNudCBvY2N1cmlu ZyByaWdodCBub3cu
Lgo+IEhlcmUgaXMgc29tZSBpbmZvIG9uIHRoZSBBcGFjaGUgd2UgYXJlIHVz aW5nOgo+IFNlcnZl
ciBWZXJzaW9uOiBBcGFjaGUvMi4wLjYxIChVbml4KSBtb2Rfc3NsLzIuMC42 MSBPcGVuU1NMLzAu
OS43YQo+IG1vZF9hdXRoX3Bhc3N0aHJvdWdoLzIuMSBGcm9udFBhZ2UvNS4w LjIuMjYzNSBtb2Rf
YndsaW1pdGVkLzEuNAo+Cj4gV2UgYXJlIHJ1bm5pbmcgUkVESEFUIEVudGVy cHJpc2UgNCBpNjg2
IGFsbW9zdCBhbGwgb3VyIHBhZ2VzIGFyZSBkeW5hbWljCj4gcGhwLi4gYW5k IGl0cyBydW5uaW5n
IGFib3V0IDIwMCB3ZWJzaXRlcy4uIHdlIGRvbnQgaGF2ZSBhIGxvYWQgYmFs YW5jZXIuLgo+Cj4g
QWxsIHRoZSByZXF1ZXN0cyBhcmUgbm90IGZyb20gdGhlIHNhbWUgaXAgYWRk cmVzcyB2ZXJ5IHJh
bmRvbS4KPgo+Cj4KPiBBbGYgSMO4Z2VtYXJrIHdyb3RlOgo+ID4KPiA+IEhp Cj4gPgo+ID4gVGhp
cyBzZWVtcyB0byBiZSBxdWl0ZSBzaW1pbGFyIHRvIHdoYXQgSSBoYXZlIHNl ZW4sIGFuZCBvdGhl
cnMgaGF2ZSBhbHNvCj4gPiBwb3N0ZWQgYWJvdXQgdGhlIGxhc3QgY291cGxl IG9mIHllYXJzLgo+
ID4gSSd2ZSBkb25lIGEgbG90IG9mIGludmVzdGlnYXRpb24sIGJ1dCBoYXZl IG5vdCBmb3VuZCBh
IHJlYWwgc29sdXRpb24uCj4gPiBNeSBwcm9ibGVtIGlzIHRoYXQgdGhlIHRo cmVhZHMgZ2V0cyBz
dHVjayBpbiAiVyIgLyAiU2VuZGluZyByZXBseSIgc3RhdGUuCj4gPgo+ID4g QXJlIHlvdSB1c2lu
ZyBtb2RfcGhwID8gSWYgc28sIGl0IG1pZ2h0IGJlIHRoYXQgb25lIHRocmVh ZCBpcyBkb2luZwo+
ID4gd29yaywgd2hpbGUgdGhlIG90aGVyIHRocmVhZHMgYXJlIHdhaXRpbmcg dG8gYWNxdWlyZSBh
IGxvY2sgb24gdGhlIHBocAo+ID4gc2Vzc2lvbiBmaWxlLgo+ID4gV2hlbiBQ SFAgc2Vzc2lvbiBm
aWxlcyBhcmUgdXNlZCwgdGhlIHJlcXVlc3QgYXJlIHByb2Nlc3NlZCBzZXF1 ZW50aWFsbHksCj4g
PiBhbmQgbm90IGluIHBhcmFsbGVsIGZvciBhIHVzZXIuCj4gPgo+ID4gSSBz dWdnZXN0IHlvdSBh
bHNvIHVzZSAic3RyYWNlIiB1bml4IGNvbW1hbmQgdG8gc2VlIHdoYXQgdGhl IGRpZmZlcmVudAo+
ID4gdGhyZWFkcyBhcmUgZG9pbmcsIHRoYXQgbWlnaHQgYmUgZWFzaWVyIHRo YW4gdXNpbmcgZ2Ri
Lgo+ID4gV2hlbiBJIHVzZSBzdHJhY2UsIEkgc2VlIHRoYXQgb25lIHRocmVh ZCBpcyBoYW5naW5n
IG9uICJwb2xsIiBzeXN0ZW0KPiA+IGNhbGwsIHVudGlsIHRoZSB0aW1lb3V0 IChUaW1lb3V0IEFw
YWNoZSBkaXJlY3RpdmUgLSBkZWZhdWx0IDMwMCBzZWNvbmRzKQo+ID4gb2Nj dXIuIFRoZW4gSSBz
ZWUgdGhhdCB0aGUgbmV4dCB0aHJlYWQgaGFuZ3MgaW4gInBvbGwiIGFuZCBz byBvbi4KPiA+IFNv
IGlmIHlvdSBzZXQgdGhlICJUaW1lb3V0IiBsb3dlciwgaXQgbWlnaHQgaW1w cm92ZSB5b3VyIHNp
dHVhdGlvbi4KPiA+Cj4gPiBZb3Ugc2hvdWxkIGFsc28gcnVuICJuZXRzdGF0 IiwgYW5kIHNlZSB3
aGF0IHRoZSBzdGF0dXMgb2YgdGhlIHNvY2tldHMKPiA+IGFyZS4gSWYgdGhl eSBhcmUgYWxtb3N0
IGFsbCBpbiBDTE9TRV9XQUlULCBJIHRoaW5rIHlvdSBhcmUgc2VlaW5nCj4g PiBzb21ldGhpbmcg
c2ltaWxhciB0byBtZS4KPiA+Cj4gPiBIZXJlIGFyZSBzb21lIGxpbmtzIHRv IHNpbWlsYXIgaXNz
dWVzLCBJJ20gbm90IHN1cmUgaWYgaXQgd2lsbCBoZWxwIHlvdSA6Cj4gPiBo dHRwOi8vd3d3Lm5h
YmJsZS5jb20vUkU6LXJlcXVlc3RzLXRpbWUtb3V0LXVuZGVyLWxvYWQsLW5v LXdhcm5pbmdzLWlu
LWxvZ3MtcDEzMDY1NDk1Lmh0bWwKPiA+IGh0dHA6Ly9tYWlsLWFyY2hpdmVz LmFwYWNoZS5vcmcv
bW9kX21ib3gvaHR0cGQtdXNlcnMvMjAwNjExLm1ib3gvJTNjMjAwNjExMTQw ODQzMzMuR0MxMDgw
OUBuZXh1cy5pbnRyYS5jb3JldHJlay5jb20lM2UKPiA+ICggYW5kIHRoZSBv dGhlciBwb3N0cyB0
byB0aGF0IHRocmVhZCkKPiA+IGh0dHA6Ly93d3cubGludXhmb3J1bXMub3Jn L2ZvcnVtL3NlcnZl
cnMvMjc1MTktYWxyaWdodC1pbS1keWluZy1oZXJlLWFwYWNoZS1oZWxwLW15 c3FsLXBocC5odG1s
Cj4gPiBodHRwOi8vb3NkaXIuY29tL21sL2FwYWNoZS5tb2Qtc3NsLnVzZXIv MjAwNC0xMC9tc2cw
MDAwOS5odG1sCj4gPgo+ID4gSSBoYXZlIGFsc28gc2VlbiBvdGhlciBwb3N0 cyBmcm9tIHBlb3Bs
ZSB3aXRoIHZlcnkgc2ltaWxhciBwcm9ibGVtcywKPiA+IGxvdHMgb2YgdGhy ZWFkIHN0dWNrIGlu
IGEgc3RhdGUuCj4gPgo+ID4gV2hhdCBpcyB5b3VyIGVudmlyb25tZW50ID8g SS5lLiB3aGF0IHZl
cnNpb24gb2YgQXBhY2hlLCB3aGF0IG9wZXJhdGluZwo+ID4gc3lzdGVtIGFu ZCB2ZXJzaW9uLCBh
cmUgeW91IGdlbmVyYXRpbmcgZHluYW1pYyBwYWdlcywgYW5kIGlmIHNvLCB3 aGF0Cj4gPiB0b29s
cyAoZm9yIGV4YW1wbGUgcGhwKSBhcmUgeW91IHVzaW5nID8KPiA+IERvIHlv dSBoYXZlIGEgZmly
ZXdhbGwgb3IgYSBsb2FkIGJhbGFuY2VyIGluIGZyb250IG9mIHlvdXIgd2Vi IHNlcnZlciA/Cj4g
Pgo+ID4gQXJlIGFsbW9zdCBhbGwgdGhlIHJlcXVlc3RzIGZyb20gdGhlIHNh bWUgSVAgYWRyZXNz
ZSA/IEFyZSBhbG1vc3QgYWxsCj4gPiB0aGUgcmVxdWVzdCBmb3IgdGhlIHNh bWUgVVJMID8KPiA+
Cj4gPiBXaGF0IG15IHRoZW9yeSBpcyBhdCB0aGUgbW9tZW50LCBhZnRlciB0 aHJvd2luZyBhd2F5
IHNvbWUgb3RoZXIKPiA+IHRoZW9yaWVzLCBhcmUgdGhhdCBzb21laG93IGEg Y2xpZW50IGNyZWF0
ZXMgc2V2ZXJhbCBodW5kcmVkIHJlcXVlc3RzIGZvcgo+ID4gdGhlIHNhbWUg VVJMLgo+ID4gU2lu
Y2UgaW4gbXkgY2FzZSB0aGlzIGhhcHBlbnMgZmFpcmx5IG9mdGVuLCBzZXZl cmFsIHRpbWVzIGEg
ZGF5IG9uIGEKPiA+IHVuY29udHJvdmVyc2lhbCBoaWdoIHRyYWZmaWMgc2l0 ZSwgZnJvbSBkaWZm
ZXJlbnQgdXNlcnMsIEkgZG8gbm90IHRoaW5rCj4gPiBpdCBpcyBzb21lb25l IGRvaW5nIGEgRG9T
IGF0dGFjay4KPiA+Cj4gPiBJIHRoaW5rIGl0IG1pZ2h0IGJlIGEgcHJvYmxl bSB3aXRoIGEgZmly
ZXdhbGwgb3IgYSBwcm94eSBvciBtYXliZSB0aGUKPiA+IGJyb3dzZXIgKEkg aGF2ZSBzZWVuIGl0
IGhhcHBlbiBmb3IgcGVvcGxlIHVzaW5nIGJvdGggV2luZG93cyBhbmQgTWFj LAo+ID4gYm90aCBG
aXJlZm94IGFuZCBJRSksIHRoYXQgc2l0cyBiZXR3ZWVuIHRoZSB3ZWIgc2Vy dmVyIGFuZCB0aGUg
ZW5kIHVzZXIuCj4gPiBTb21laG93IHRoZSBjbG9zaW5nIG9mIHRoZSBzb2Nr ZXQvY29ubmVjdGlv
biBieSB0aGUgYnJvd3NlciBpcyBub3QKPiA+IHBpY2tlZCB1cCBieSB0aGUg d2ViIHNlcnZlci4g
QW5kIHNvbWVob3cgdGhlIGJyb3dzZXIgaXNzdWVzIGxvdHMgb2YKPiA+IHJl cXVlc3RzLgo+ID4g
SSBoYXZlIG5vdCBjb21wbGV0ZWx5IHJ1bGVkIG91dCB0aGF0IGl0IG1pZ2h0 IGJlIGEgQXBhY2hl
IHdlYiBzZXJ2ZXIgYnVnCj4gPiBhcyB3ZWxsLgo+ID4gQnV0IHNpbmNlIEkg aGF2ZSBubyB3YXkg
b2YgcmVwcm9kdWNpbmcgdGhpcywgYXBhcnQgZnJvbSBvYnNlcnZpbmcgaXQK PiA+IGhhcHBlbmlu
ZyBxdWl0ZSBvZnRlbiwgaXQgaXMgdmVyeSBoYXJkIHRvIGdldCBhbnkgZnVy dGhlci4KPiA+IFNv
IGFueSBoZWxwIHdvdWxkIGJlIGdyZWF0bHkgYXBwcmVjaWF0ZWQuCj4gPgo+ ID4gUmVnYXJkcwo+
ID4gQWxmIEhvZ2VtYXJrCj4gPgo+ID4KPiA+Cj4gPiBTYW5kZXIgVGVtbWUg d3JvdGU6Cj4gPj4K
PiA+PiBPbiBOb3YgNywgMjAwNywgYXQgOTozMSBBTSwgQW5kcmV3IFJvc29s aW5vIHdyb3RlOgo+
ID4+Cj4gPj4+IEkgcmVzdGFydGVkIGFwYWNoZSBhbmQgYWxsIHdhcyBmaW5l Li4gYnV0IHRoZW4g
MjAgbWludXRlcyBsYXRlciB0aGV5Cj4gPj4+IHdlbnQKPiA+Pj4gYmFjayBh bGwgaW50byBhIHJl
YWRpbmcgc3RhdGUuLiBpdCBhcHBlYXJzIGFzIGlmIHNsb3dseSBlYWNoCj4g Pj4+IHByb2Nlc3Nl
cyBnb2VzCj4gPj4+IGludG8gdGhlIHJlYWRpbmcgc3RhdGU/PyBJIGRvbnQg dW5kZXJzdGFuZCB3
aGF0IHRoZSBwcm9ibGVtIGlzLgo+ID4+Cj4gPj4gSWYgeW91IGhhdmUgZ2Ri IG9uIHRoZSBib3gg
KGFuZCBoYXZlIHNvbWUgYWN1bWVuIHVzaW5nIGl0KSwgeW91IGNhbgo+ID4+ IGF0dGFjaCB0byBv
bmUgb2YgdGhlIHByb2Nlc3NlcyBhbmQgc2VlIGlmIGl0IGlzIGhhbmdpbmcg aW4gc29tZXRoaW5n
Cj4gPj4gb2J2aW91cy4KPiA+Pgo+ID4+IElmIG5vdCwgYW5kIHlvdXIgdHJh ZmZpYyBpcyBwbGFp
bnRleHQsIEkgcmVjb21tZW5kIHRoZSB0Y3BkdW1wCj4gPj4gYXBwcm9hY2gg dG8gc2VlIHdoYXQg
aXMgZ29pbmcgb24gdHJhZmZpYy13aXNlLCBhbmQgd2hhdCB0cmlnZ2VycyB0 aGUKPiA+PiBpc3N1
ZS4gIEknZCBkdW1wIHRoZSB0cmFmZmljIHRvIGRpc2sgKHRjcGR1bXAgLWkg ZXRoWCAtcyAwIC13
Cj4gPj4gL3doZXJldmVyL3RyYWZmaWMuZHVtcCBwb3J0IDgwKSBhbmQgdGFr ZSBpdCBvdmVyIHRv
IGEgYm94IHdpdGgKPiA+PiBFdGhlcmVhbC9XaXJlc2hhcmsgZm9yIHNvbWUg Y296eSBhbmFseXNp
cy4KPiA+Pgo+ID4+IElmIHlvdSdyZSBiZWluZyBET1MgYXR0YWNrZWQgYnkg dHJpY2tsZSByZXF1
ZXN0cywgeW91IGNvdWxkIHRyeQo+ID4+IHNldHRpbmcgYSB2ZXJ5IGxvdyB0 aW1lb3V0IChkZWZh
dWx0IGlzIDUgbWludXRlcyB3aGljaCBkb2Vzbid0IHNlZW0gdG8KPiA+PiBi ZSB3b3JraW5nIGZv
ciB5b3UpIGFuZCBwZXJoYXBzIHVzZSBtb2RfZXZhc2l2ZSBvciBzb21lc3Vj aCB0byBmbGFnCj4g
Pj4gYW5kIGZpcmV3YWxsIHRoZSBiYWQgY2xpZW50cy4KPiA+Pgo+ID4+IFMu Cj4gPj4KPiA+Cj4g
PiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0KPiA+IFRoZSBvZmZpY2lhbCBVc2VyLVRvLVVzZXIg c3VwcG9ydCBmb3J1
bSBvZiB0aGUgQXBhY2hlIEhUVFAgU2VydmVyIFByb2plY3QuCj4gPiBTZWUg PFVSTDpodHRwOi8v
aHR0cGQuYXBhY2hlLm9yZy91c2Vyc2xpc3QuaHRtbD4gZm9yIG1vcmUgaW5m by4KPiA+IFRvIHVu
c3Vic2NyaWJlLCBlLW1haWw6IHVzZXJzLXVuc3Vic2NyaWJlQGh0dHBkLmFw YWNoZS5vcmcKPiA+
ICAgICIgICBmcm9tIHRoZSBkaWdlc3Q6IHVzZXJzLWRpZ2VzdC11bnN1YnNj cmliZUBodHRwZC5h
cGFjaGUub3JnCj4gPiBGb3IgYWRkaXRpb25hbCBjb21tYW5kcywgZS1tYWls OiB1c2Vycy1oZWxw
QGh0dHBkLmFwYWNoZS5vcmcKPiA+Cj4gPgo+ID4KPgo+IC0tCj4gVmlldyB0 aGlzIG1lc3NhZ2Ug
aW4gY29udGV4dDogaHR0cDovL3d3dy5uYWJibGUuY29tL0FwYWNoZS1IYW5n cy4uLVNlcnZlci1T
dGF0dXMtc2hvd3MtYWxsLVJlYWRpbmctdGY0NzY2MTEwLmh0bWwjYTEzNjkz MTgyCj4gU2VudCBm
cm9tIHRoZSBBcGFjaGUgSFRUUCBTZXJ2ZXIgLSBVc2VycyBtYWlsaW5nIGxp c3QgYXJjaGl2ZSBh
dCBOYWJibGUuY29tLgo+Cj4KPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KPgo+IFRoZSBvZmZp Y2lhbCBVc2VyLVRv
LVVzZXIgc3VwcG9ydCBmb3J1bSBvZiB0aGUgQXBhY2hlIEhUVFAgU2VydmVy IFByb2plY3QuCj4g
U2VlIDxVUkw6aHR0cDovL2h0dHBkLmFwYWNoZS5vcmcvdXNlcnNsaXN0Lmh0 bWw+IGZvciBtb3Jl
IGluZm8uCj4gVG8gdW5zdWJzY3JpYmUsIGUtbWFpbDogdXNlcnMtdW5zdWJz Y3JpYmVAaHR0cGQu
YXBhY2hlLm9yZwo+ICAgICIgICBmcm9tIHRoZSBkaWdlc3Q6IHVzZXJzLWRp Z2VzdC11bnN1YnNj
cmliZUBodHRwZC5hcGFjaGUub3JnCj4gRm9yIGFkZGl0aW9uYWwgY29tbWFu ZHMsIGUtbWFpbDog
dXNlcnMtaGVscEBodHRwZC5hcGFjaGUub3JnCj4KPgoKCgotLSAKSmVyb21l IEV0ZXZlLgpqZXJv
bWVAZXRldmUubmV0Cmh0dHA6Ly9qZXJvbWUuZXRldmUuZnJlZS5mci8K

Re: Apache Hangs.. Server-Status shows all Reading

am 28.01.2008 21:35:50 von christian.koeberl

Hi!


Jérôme Etévé-2 wrote:
>=20
> I've got here exactly the same problem, but without any php modules. My
> server serves only static resources and It's flooded with reading state
> request sometime.
>=20

We've also got the same problem and as it seems the Apache foundation has
the same problem as well (see this server-status of issues.apache.org:=20
http://www.nabble.com/file/p15144733/server-status-issues-ap ache-2008-01-25=
..htm
server-status-issues-apache-2008-01-25.htm ). The actual problem is that
requests are stuck in "Reading Request" until the timeout occurs. In the
server-status from issues.apache.org you see "R"-request with more than 500=
s
running.

There are several people describing the same problem with different version=
s
of Apache (also 1.x) and different SSL implementation - but no solution.

I have made a JMeter test, wich only requests static resources on our serve=
r
to find the cause: I can reproduce the problem on our test server. There,
I've done some analysis with netstat and I've seen that there are lot of tc=
p
connections with "TIME_WAIT" - but I'm not sure if that is in connection
with this phenomenon.

Anybody else got this problem? Did anybody solve this?

Maybe we should file a bugzilla entry?

--=20
Chris
--=20
View this message in context: http://www.nabble.com/Apache-Hangs..-Server-S=
tatus-shows-all-Reading-tp13631744p15144733.html
Sent from the Apache HTTP Server - Users mailing list archive at Nabble.com=
..


------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org