Consider mod_proxy used to serve as a proxy gateway (aka reverse proxy) to a
set of backend servers. Seems that it would be useful to maintain a pool of
connections to the backend servers and multiplex HTTP requests over these
connections (to save connection setup and takedown). Anyone have any ideas
on whether this would be a worthwhile performance improvement?
Bill
On Mon, 2002-08-19 at 07:13, Bill Stoddard wrote:
> Consider mod_proxy used to serve as a proxy gateway (aka reverse proxy) to a
> set of backend servers. Seems that it would be useful to maintain a pool of
> connections to the backend servers and multiplex HTTP requests over these
> connections (to save connection setup and takedown). Anyone have any ideas
> on whether this would be a worthwhile performance improvement?
>
I was thinking of doing something like this for quite some time.
the only issues I can see are
keepalives..
* do they handle things like cookies per request or per
connection?
* authentication
* handling of keepalive timeouts might be tricky
* ie.. do keepalives keep some state, I am never sure
there are also some other minor problems if you want to load balance the
backendservers, as that would break.
The reason I never pursued it was due to the backends we use being so
pathetic that they can't even handle simple Keep-alives without falling
over.
Oh apr_reslist would be a good starting point for this.
> Bill
--
Ian Holsman
Performance Measurement & Analysis
CNET Networks
PH: 415-344-2608
On 19 Aug 2002, Ian Holsman wrote:
> I was thinking of doing something like this for quite some time. the
> only issues I can see are keepalives.
There should be no problem with this approach as far as I know. Many
proxies, including open-source Squid, maintain the client-side
connection pool virtually independent from the server-side pool.
> * do they handle things like cookies per request or per
> connection?
Cookies are end-to-end things that are not related to hop-by-hop
connection management.
> * authentication
Same here unless you are talking about some proprietary authentication
methods that require certain connection reuse patterns (but those are
usually not proxied anyway).
> * handling of keepalive timeouts might be tricky
Not in general, AFAIK. You just maintain two connection pools. The
timeout and other algorithms are pretty much the same for each side.
> * ie.. do keepalives keep some state, I am never sure
Not sure what you mean by state here. There is no HTTP state, but,
yes, since a connection may survive several HTTP transactions, there
is some internal state to be kept (e.g., destination address, idle
timer, etc.)
> there are also some other minor problems if you want to load balance the
> backendservers, as that would break.
Yes, but load balancing needs special options and algorithms anyway.
Most of them are not really closely related to idle connection
management.
$0.02,
Alex.
--
| HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
| all of the above - PolyBox appliance
On Mon, 2002-08-19 at 09:00, Alex Rousskov wrote:
> On 19 Aug 2002, Ian Holsman wrote:
>
Good .. thats fantastic news Alex.
The other thing to remember is that in >our< case (yours may vary)
connection setup/teardown may account for ~10ms, while the actual thing
being reversed proxied is a order of magnitude (or more) larger, and
this stuff just gets lost in the noise.
--Ian
> > I was thinking of doing something like this for quite some time. the
> > only issues I can see are keepalives.
>
> There should be no problem with this approach as far as I know. Many
> proxies, including open-source Squid, maintain the client-side
> connection pool virtually independent from the server-side pool.
>
> > * do they handle things like cookies per request or per
> > connection?
>
> Cookies are end-to-end things that are not related to hop-by-hop
> connection management.
>
> > * authentication
>
> Same here unless you are talking about some proprietary authentication
> methods that require certain connection reuse patterns (but those are
> usually not proxied anyway).
>
> > * handling of keepalive timeouts might be tricky
>
> Not in general, AFAIK. You just maintain two connection pools. The
> timeout and other algorithms are pretty much the same for each side.
>
> > * ie.. do keepalives keep some state, I am never sure
>
> Not sure what you mean by state here. There is no HTTP state, but,
> yes, since a connection may survive several HTTP transactions, there
> is some internal state to be kept (e.g., destination address, idle
> timer, etc.)
>
> > there are also some other minor problems if you want to load balance the
> > backendservers, as that would break.
>
> Yes, but load balancing needs special options and algorithms anyway.
> Most of them are not really closely related to idle connection
> management.
>
>
> $0.02,
>
> Alex.
>
> --
> | HTTP performance - Web Polygraph benchmark
> www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
> | all of the above - PolyBox appliance
--
Ian Holsman
Performance Measurement & Analysis
CNET Networks
PH: 415-344-2608
Ian Holsman wrote:
> I was thinking of doing something like this for quite some time.
> the only issues I can see are
> keepalives..
> * do they handle things like cookies per request or per
> connection?
Servers don't handle cookies - they're just Yet Another Header.
> * authentication
Authentication happens on a per request basis, not per connection, so no
problems there.
> * handling of keepalive timeouts might be tricky
> * ie.. do keepalives keep some state, I am never sure
There is no state, no. The next request follows the previous, and each
request has a well defined length (either using content-length, or
chunked encoding) so the receiving end always knows when the previous
request ended the the current began.
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm
"There's a moon
over Bourbon Street
tonight..."
On Mon, 19 Aug 2002, Graham Leggett wrote:
> Ian Holsman wrote:
>
> > I was thinking of doing something like this for quite some time.
> > the only issues I can see are
> > keepalives..
> > * do they handle things like cookies per request or per
> > connection?
>
> Servers don't handle cookies - they're just Yet Another Header.
Well, not so if you run your reverse proxy with caching enabled. Then
it becomes a Yet Another Header To Worry About :-).
Alex.
--
| HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
| all of the above - PolyBox appliance
On 19 Aug 2002, Ian Holsman wrote:
> The other thing to remember is that in >our< case (yours may vary)
> connection setup/teardown may account for ~10ms, while the actual
> thing being reversed proxied is a order of magnitude (or more)
> larger, and this stuff just gets lost in the noise.
It is not just about response time overhead for connection setup. One
should also account for the number of open connections that need to be
supported (on the proxy and on the origins), related DoS attacks, and
such. I am not saying persistency and "multiplexing" is a big
performance win by default. It may be, in some situations, with proper
tuning.
Alex.
--
| HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
| all of the above - PolyBox appliance
Alex Rousskov wrote:
> Well, not so if you run your reverse proxy with caching enabled. Then
> it becomes a Yet Another Header To Worry About :-).
But then that's the cache's problem, not the proxy's.
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm
"There's a moon
over Bourbon Street
tonight..."
On Mon, 19 Aug 2002, Alex Rousskov wrote:
> > * ie.. do keepalives keep some state, I am never sure
>
> Not sure what you mean by state here. There is no HTTP state, but,
> yes, since a connection may survive several HTTP transactions, there
> is some internal state to be kept (e.g., destination address, idle
> timer, etc.)
eh; usually it's not an unreasonable assumption that one connection
will have one client in the other end. If the backend uses that for
optimization it would break.
--
ask bjoern hansen, http://www.askbjoernhansen.com/ !try; do();
On Tue, 20 Aug 2002, Ask Bjoern Hansen wrote:
> eh; usually it's not an unreasonable assumption that one connection
> will have one client in the other end. If the backend uses that for
> optimization it would break.
I assume you are talking about "end clients" or "user agents". In my
experience, such optimizations are extremely dangerous to the point of
being broken. There are many client-side proxies out there. A single
connection does not mean a single end client.
Alex.
--
| HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
| all of the above - PolyBox appliance
On Tue, 20 Aug 2002, Alex Rousskov wrote:
> On Tue, 20 Aug 2002, Ask Bjoern Hansen wrote:
>
> > eh; usually it's not an unreasonable assumption that one connection
> > will have one client in the other end. If the backend uses that for
> > optimization it would break.
>
> I assume you are talking about "end clients" or "user agents". In my
> experience, such optimizations are extremely dangerous to the point of
> being broken. There are many client-side proxies out there. A single
> connection does not mean a single end client.
Really? What proxies aggregates requests from multiple end clients?
- ask
--
ask bjoern hansen, http://www.askbjoernhansen.com/ !try; do();
On Wed, 21 Aug 2002, Ask Bjoern Hansen wrote:
> Really? What proxies aggregates requests from multiple end clients?
I know that Squid does. Some say Squid covers at least 50% of forward
HTTP proxy installations. I do not know for sure, but would be
surprised if most commercial caches (NetApp, Cisco, Cacheflow, iMimic,
etc.) do not maintain separate client- and server-side connection
pools. AOL probably does that too. Again, I cannot be certain about
commercial implementations (would have to check cache-off results for
a proof). Possible performance gains aside, it simply makes for a
better, more flexible design, I guess.
Alex.
--
| HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
| all of the above - PolyBox appliance