strange random delays in a proxy configuration

strange random delays in a proxy configuration

am 23.11.2009 18:09:43 von aw

Hi guys.

A customer of ours have an internal Apache 2.2 server under Linux
Debian. Static pages of their applications are located on their own server.
Some URLs are proxied by their server to our own Apache 2.2 server
(Linux Debian), through the Internet, by means of a line like

ProxyPass /getthumb http://our-server.company.com
and the URLs are like
/getthumb?id=(thumb-id)

These URLs obtain thumbnails from our server, corresponding to
documents. Each thumbnail is generated on the fly and is about 10 KB in
size. It takes our server on average 150 ms to process such a URL and
return the thumbnail to their server.
The number of thumbnails to serve for one customer application page is
variable, between just a few and sometimes several hundred, with an
average of 20-30 per page.
In the vast majority of cases, each thumbnail request is served in
around 150 ms, and we serve thousands such requests per day.

Yet sometimes, for no apparent reason that we can link to the process of
generating and returning the thumbnail, nor to any load of our own
Apache server or theirs, returning one arbitrary thumbnail in the series
takes about 30 seconds.
It is always either about 150 ms, or about 30000 ms. It is never
in-between, like 10 or 20 seconds or so.

The delay does not always affect the same thumbnails either. For the
same requested page with the same series of thumbnails, sometimes the 30
seconds delay will happen at the first and second thumbnail, sometimes 4
of them in the middle, and so on. Sometimes the whole page-worth of
thumbnails loads in exactly (number of thumbnails X 150 ms), which is
what it should always be.

If I watch our server's processes with "top", and simultaneously the end
of the access log or the screen of one of their workstations, I can see
nothing untoward when the browser suddenly "hangs" for 30 seconds
waiting fo the next thumbnail. The Apache processes on our side seem
mostly idle at that time, and there are plenty of Apache processes
available to serve any request.
It looks as if the browser on the customer side just hangs waiting for
the thumbnail, and our server is just idle.

These times are per our own server's access log, and per their Firefox
browser, to which we added the HttpFox add-on, to see what happens from
their side.
Their browsers do issue the requests (almost all at once), and the
symptoms are the same whether they use IE or Firefox.

My first question is : does there exist any tool in Apache itself, that
would show me what is happening during such a 30-second pause ?
The access log does not, as it just adds a line at the end of the 30
seconds, when the request is terminated, which is too late. There are
no errors in our error log.
mod_log_io and mod_forensic, from their documentation, do not seem to
provide much more relevant info.

For example, since their server proxies these calls to ours, and we see
nothing in our logs, is there a way whereby I can set the logging of the
proxy calls (and only those, preferably) to some "debug" setting ?

(I have full access to their and our Apache servers configurations)

My second question would be : does anyone have an idea of what could
explain sudden random 30 second pauses in such a schema ?

I am trying to avoid having to go through a TCP protocol analyser,
because I am unfamiliar with them, at least at the Linux level.


Thanks

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: [SOLVED] strange random delays in a proxy configuration

am 24.11.2009 00:58:52 von aw

André Warnier wrote:
> Hi guys.
>
> A customer of ours have an internal Apache 2.2 server
....
Never mind, I believe we found the problem, and it has nothing to do
with the network or Apache or proxying per se.
Sorry to have wasted your time if I did.

It seems to be the application that is misbehaving, reloading some big
table in each Apache child from time to time, when it has no reason to.
That explains why the pattern seemed random too, because when it happens
depends on how many requests are sent by each client and directed to
which Apache child, and the settings of MaxKeepAliveRequests and so on
with which I have been fiddling in attempts to circumscribe the issue.



------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org