Strange behaviour with Pseudo-Proxy script

This is a multi-part message in MIME format.

------=_NextPart_000_004A_01CB49DC.ADF15C20
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit

Hi,



I am having a strange issue with a mod_perl handler which I've written
lately.



A little background. we are using a mod_perl script for our self-developed
MS .NET application.

The application connects to the frontend server, where the mod_perl
"proxy" is running. The script

does some kind of load_balancing and then proxies the request to the
backend and providest the

answer from the backend back to the application. This concept is working
fine, but the initial version

of the proxy script is poorly written. It first collects all data from the
client in memory (which might be

a pretty high amount of data (up to 100MB)), then send the stuff to the
backend and then provides

the answer. This causes bad memory consumption- especially if there are
more than 100 concurrent

users. Also the script is no real mod_perl script, but uses
ModPerl::Registry instead.



So I've re-written the whole script from scratch. I've written it as
"real" mod_perl handler module.

It's now working with chunks of data to be sent and received. Everything
seems to working fine-

except of one thing. The .NET app has the ability to upload a file to the
backend. With the original

script the upload usually has at least about 1-2MB/sec bandwidth. but with
the new script that

I've written, it only provides a throughput of about 300kb/sec. I did
couple of profiling already,

but wasn't able to find the cause of this.



So I am wondering if someone of the mod_perl community has some advices or
hints, how to resolve

this issue. Every help or thought-provoking impulse is highly appreciated.




Thanks

Winni


------=_NextPart_000_004A_01CB49DC.ADF15C20
Content-Type: text/html;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" =
xmlns=3D"http://www.w3.org/TR/REC-html40">


charset=3Dus-ascii">

Re: Strange behaviour with Pseudo-Proxy script

Aside from posting the source code so that we can peruse and say "That
might be it", you might try putting a non blocking reverse proxy such
as Perlbal (my favorite, and darn tootin' fast), Nginx, or Varnish in
front of your mod_perl instance so that it does the dirty work of
connection handling and mod_perl isn't stuck spoon feeding slow
clients.


On Wed, Sep 1, 2010 at 4:50 AM, Winfried Neessen
wrote:
> Hi,
>
>
>
> I am having a strange issue with a mod_perl handler which I=92ve written
> lately.
>
>
>
> A little background=85 we are using a mod_perl script for our self-develo=
ped
> MS .NET application.
>
> The application connects to the frontend server, where the mod_perl =93pr=
oxy=94
> is running. The script
>
> does some kind of load_balancing and then proxies the request to the back=
end
> and providest the
>
> answer from the backend back to the application. This concept is working
> fine, but the initial version
>
> of the proxy script is poorly written. It first collects all data from th=
e
> client in memory (which might be
>
> a pretty high amount of data (up to 100MB)), then send the stuff to the
> backend and then provides
>
> the answer=85 This causes bad memory consumption- especially if there are=
more
> than 100 concurrent
>
> users. Also the script is no real mod_perl script, but uses
> ModPerl::Registry instead.
>
>
>
> So I=92ve re-written the whole script from scratch. I=92ve written it as =
=93real=94
> mod_perl handler module.
>
> It=92s now working with chunks of data to be sent and received. Everythin=
g
> seems to working fine-
>
> except of one thing=85 The .NET app has the ability to upload a file to t=
he
> backend. With the original
>
> script the upload usually has at least about 1-2MB/sec bandwidth=85 but w=
ith
> the new script that
>
> I=92ve written, it only provides a throughput of about 300kb/sec. I did c=
ouple
> of profiling already,
>
> but wasn=92t able to find the cause of this.
>
>
>
> So I am wondering if someone of the mod_perl community has some advices o=
r
> hints, how to resolve
>
> this issue. Every help or thought-provoking impulse is highly appreciated=
..
>
>
>
> Thanks
>
> Winni

RE: Strange behaviour with Pseudo-Proxy script

Hi Fred,

here is the code:
Proxy.pm: http://dokuleser.privatepaste.com/dd9f56cb09/vu0bxLIICY
Proxy/AppServer.pm:
http://dokuleser.privatepaste.com/36b60ba9e2/byzjGiLki0
Proxy/Header.pm: http://dokuleser.privatepaste.com/e306920a14/rhxc8kRUFB

Thanks
Winni

-----Original Message-----
From: Fred Moyer [mailto:fred@redhotpenguin.com]
Sent: Wednesday, September 01, 2010 8:21 PM
To: Winfried Neessen
Cc: modperl@perl.apache.org
Subject: Re: Strange behaviour with Pseudo-Proxy script

Aside from posting the source code so that we can peruse and say "That
might be it", you might try putting a non blocking reverse proxy such
as Perlbal (my favorite, and darn tootin' fast), Nginx, or Varnish in
front of your mod_perl instance so that it does the dirty work of
connection handling and mod_perl isn't stuck spoon feeding slow
clients.


On Wed, Sep 1, 2010 at 4:50 AM, Winfried Neessen
wrote:
> Hi,
>
>
>
> I am having a strange issue with a mod_perl handler which I've written
> lately.
>
>
>
> A little background. we are using a mod_perl script for our
self-developed
> MS .NET application.
>
> The application connects to the frontend server, where the mod_perl
"proxy"
> is running. The script
>
> does some kind of load_balancing and then proxies the request to the
backend
> and providest the
>
> answer from the backend back to the application. This concept is working
> fine, but the initial version
>
> of the proxy script is poorly written. It first collects all data from
the
> client in memory (which might be
>
> a pretty high amount of data (up to 100MB)), then send the stuff to the
> backend and then provides
>
> the answer. This causes bad memory consumption- especially if there are
more
> than 100 concurrent
>
> users. Also the script is no real mod_perl script, but uses
> ModPerl::Registry instead.
>
>
>
> So I've re-written the whole script from scratch. I've written it as
"real"
> mod_perl handler module.
>
> It's now working with chunks of data to be sent and received. Everything
> seems to working fine-
>
> except of one thing. The .NET app has the ability to upload a file to
the
> backend. With the original
>
> script the upload usually has at least about 1-2MB/sec bandwidth. but
with
> the new script that
>
> I've written, it only provides a throughput of about 300kb/sec. I did
couple
> of profiling already,
>
> but wasn't able to find the cause of this.
>
>
>
> So I am wondering if someone of the mod_perl community has some advices
or
> hints, how to resolve
>
> this issue. Every help or thought-provoking impulse is highly
appreciated.
>
>
>
> Thanks
>
> Winni

RE: Strange behaviour with Pseudo-Proxy script

Hi again,

sorry forgot Proxy/Sender.pm in my last mail.
http://dokuleser.privatepaste.com/05e9b4f124/QHvZO0RMoi


Winni

-----Original Message-----
From: Winfried Neessen [mailto:neessen@cleverbridge.com]
Sent: Thursday, September 09, 2010 11:07 AM
To: modperl@perl.apache.org
Subject: RE: Strange behaviour with Pseudo-Proxy script

Hi Fred,

here is the code:
Proxy.pm: http://dokuleser.privatepaste.com/dd9f56cb09/vu0bxLIICY
Proxy/AppServer.pm:
http://dokuleser.privatepaste.com/36b60ba9e2/byzjGiLki0
Proxy/Header.pm: http://dokuleser.privatepaste.com/e306920a14/rhxc8kRUFB

Thanks
Winni

-----Original Message-----
From: Fred Moyer [mailto:fred@redhotpenguin.com]
Sent: Wednesday, September 01, 2010 8:21 PM
To: Winfried Neessen
Cc: modperl@perl.apache.org
Subject: Re: Strange behaviour with Pseudo-Proxy script

Aside from posting the source code so that we can peruse and say "That
might be it", you might try putting a non blocking reverse proxy such
as Perlbal (my favorite, and darn tootin' fast), Nginx, or Varnish in
front of your mod_perl instance so that it does the dirty work of
connection handling and mod_perl isn't stuck spoon feeding slow
clients.


On Wed, Sep 1, 2010 at 4:50 AM, Winfried Neessen
wrote:
> Hi,
>
>
>
> I am having a strange issue with a mod_perl handler which I've written
> lately.
>
>
>
> A little background. we are using a mod_perl script for our
self-developed
> MS .NET application.
>
> The application connects to the frontend server, where the mod_perl
"proxy"
> is running. The script
>
> does some kind of load_balancing and then proxies the request to the
backend
> and providest the
>
> answer from the backend back to the application. This concept is working
> fine, but the initial version
>
> of the proxy script is poorly written. It first collects all data from
the
> client in memory (which might be
>
> a pretty high amount of data (up to 100MB)), then send the stuff to the
> backend and then provides
>
> the answer. This causes bad memory consumption- especially if there are
more
> than 100 concurrent
>
> users. Also the script is no real mod_perl script, but uses
> ModPerl::Registry instead.
>
>
>
> So I've re-written the whole script from scratch. I've written it as
"real"
> mod_perl handler module.
>
> It's now working with chunks of data to be sent and received. Everything
> seems to working fine-
>
> except of one thing. The .NET app has the ability to upload a file to
the
> backend. With the original
>
> script the upload usually has at least about 1-2MB/sec bandwidth. but
with
> the new script that
>
> I've written, it only provides a throughput of about 300kb/sec. I did
couple
> of profiling already,
>
> but wasn't able to find the cause of this.
>
>
>
> So I am wondering if someone of the mod_perl community has some advices
or
> hints, how to resolve
>
> this issue. Every help or thought-provoking impulse is highly
appreciated.
>
>
>
> Thanks
>
> Winni

RE: Strange behaviour with Pseudo-Proxy script

Hi again,

I was able to figure out the problem on my own. After re-reading the
mod_perl2
documentation of Apache2::RequestIO I figured that I forgot to enable
autoflush
on STDOUT, which causes the $r->print() call to be buffered.

Regards
Winni


-----Original Message-----
From: Winfried Neessen [mailto:neessen@cleverbridge.com]
Sent: Thursday, September 09, 2010 11:22 AM
To: modperl@perl.apache.org
Subject: RE: Strange behaviour with Pseudo-Proxy script

Hi again,

sorry forgot Proxy/Sender.pm in my last mail.
http://dokuleser.privatepaste.com/05e9b4f124/QHvZO0RMoi


Winni

-----Original Message-----
From: Winfried Neessen [mailto:neessen@cleverbridge.com]
Sent: Thursday, September 09, 2010 11:07 AM
To: modperl@perl.apache.org
Subject: RE: Strange behaviour with Pseudo-Proxy script

Hi Fred,

here is the code:
Proxy.pm: http://dokuleser.privatepaste.com/dd9f56cb09/vu0bxLIICY
Proxy/AppServer.pm:
http://dokuleser.privatepaste.com/36b60ba9e2/byzjGiLki0
Proxy/Header.pm: http://dokuleser.privatepaste.com/e306920a14/rhxc8kRUFB

Thanks
Winni

-----Original Message-----
From: Fred Moyer [mailto:fred@redhotpenguin.com]
Sent: Wednesday, September 01, 2010 8:21 PM
To: Winfried Neessen
Cc: modperl@perl.apache.org
Subject: Re: Strange behaviour with Pseudo-Proxy script

Aside from posting the source code so that we can peruse and say "That
might be it", you might try putting a non blocking reverse proxy such
as Perlbal (my favorite, and darn tootin' fast), Nginx, or Varnish in
front of your mod_perl instance so that it does the dirty work of
connection handling and mod_perl isn't stuck spoon feeding slow
clients.


On Wed, Sep 1, 2010 at 4:50 AM, Winfried Neessen
wrote:
> Hi,
>
>
>
> I am having a strange issue with a mod_perl handler which I've written
> lately.
>
>
>
> A little background. we are using a mod_perl script for our
self-developed
> MS .NET application.
>
> The application connects to the frontend server, where the mod_perl
"proxy"
> is running. The script
>
> does some kind of load_balancing and then proxies the request to the
backend
> and providest the
>
> answer from the backend back to the application. This concept is working
> fine, but the initial version
>
> of the proxy script is poorly written. It first collects all data from
the
> client in memory (which might be
>
> a pretty high amount of data (up to 100MB)), then send the stuff to the
> backend and then provides
>
> the answer. This causes bad memory consumption- especially if there are
more
> than 100 concurrent
>
> users. Also the script is no real mod_perl script, but uses
> ModPerl::Registry instead.
>
>
>
> So I've re-written the whole script from scratch. I've written it as
"real"
> mod_perl handler module.
>
> It's now working with chunks of data to be sent and received. Everything
> seems to working fine-
>
> except of one thing. The .NET app has the ability to upload a file to
the
> backend. With the original
>
> script the upload usually has at least about 1-2MB/sec bandwidth. but
with
> the new script that
>
> I've written, it only provides a throughput of about 300kb/sec. I did
couple
> of profiling already,
>
> but wasn't able to find the cause of this.
>
>
>
> So I am wondering if someone of the mod_perl community has some advices
or
> hints, how to resolve
>
> this issue. Every help or thought-provoking impulse is highly
appreciated.
>
>
>
> Thanks
>
> Winni

Re: Strange behaviour with Pseudo-Proxy script

Winfried Neessen schrieb am 09.09.2010 um 12:39 (+0200):
> After re-reading the mod_perl2 documentation of Apache2::RequestIO I
> figured that I forgot to enable autoflush on STDOUT, which causes the
> $r->print() call to be buffered.

So to sum it up, no autoflush, hence no buffering, hence the performance
drop you mentioned in your original mail. Correct?

--
Michael Ludwig

RE: Strange behaviour with Pseudo-Proxy script

No, not exactly...

No autoflush -> buffered output -> weird performance issues.


Winni


-----Original Message-----
From: Michael Ludwig [mailto:milu71@gmx.de]
Sent: Thursday, September 09, 2010 8:41 PM
To: modperl@perl.apache.org
Subject: Re: Strange behaviour with Pseudo-Proxy script

Winfried Neessen schrieb am 09.09.2010 um 12:39 (+0200):
> After re-reading the mod_perl2 documentation of Apache2::RequestIO I
> figured that I forgot to enable autoflush on STDOUT, which causes the
> $r->print() call to be buffered.

So to sum it up, no autoflush, hence no buffering, hence the performance
drop you mentioned in your original mail. Correct?

--
Michael Ludwig