which reverse proxy for modperl?

which reverse proxy for modperl?

am 13.12.2008 03:02:00 von Jeff Pang

------=_Part_149063_24897033.1229133720629
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hello,

I have a modperl application on a host which is running with heavy load.
I have the plan to put a reverse proxy before it.
There are two well known reverse proxy software, one is Squid, another is n=
ginx.
Which one is better for modperl application? or is there any others which a=
re better than these two?

Thanks.

Créez votre adresse électronique prenom.nom@laposte.net=20
1 Go d'espace de stockage, anti-spam et anti-virus intégrés.

------=_Part_149063_24897033.1229133720629
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hello,

I have a modperl application on =
a host which is running with heavy load.
I have the plan to put a reve=
rse proxy before it.
There are two well known reverse proxy software, =
one is Squid, another is nginx.
Which one is better for modperl applic=
ation? or is there any others which are better than these two?

T=
hanks.



arget=3D_blank href=3Dhttp://www.laposte.net>Créez votre adresse =
électronique prenom.nom@laposte.net
1 Go d'espace de stockage, ant=
i-spam et anti-virus intégrés.


------=_Part_149063_24897033.1229133720629--

Re: which reverse proxy for modperl?

am 13.12.2008 03:06:18 von wrowe

Jeff Pang wrote:
> Hello,
>
> I have a modperl application on a host which is running with heavy load.
> I have the plan to put a reverse proxy before it.
> There are two well known reverse proxy software, one is Squid, another
> is nginx.
> Which one is better for modperl application? or is there any others
> which are better than these two?

And there is httpd which you are already running, and you can run a very
low cost, separate instance with very minimal modules loaded. With the
mod_proxy_balancer, you can shuffle off some of the traffic to a second,
third machine for good measure.

Is either better? Google both for problems to determine how many people
are irritated with either.

Re: which reverse proxy for modperl?

am 13.12.2008 04:50:00 von Cees Hek

On Sat, Dec 13, 2008 at 1:02 PM, Jeff Pang wrote:
> Hello,
>
> I have a modperl application on a host which is running with heavy load.
> I have the plan to put a reverse proxy before it.
> There are two well known reverse proxy software, one is Squid, another is
> nginx.
> Which one is better for modperl application? or is there any others which
> are better than these two?

I can't tell you which is better, but I can tell you that we have been
very happy with squid. We run four load balanced squid proxies in
front of all of our web servers and they never break a sweat. Squid
is very efficient and can handle a massive amount of concurrent
connections with minimal hardware requirements. We run them on old
school single proc PIII DL360s as squid barely uses the CPU and just
load them up with RAM since squid will use as much as you throw at it
to serve cached content out of RAM instead of the disk.

I'll include a few numbers below in case it helps out (these were
taken on a Saturday afternoon, so definitely not peak time). We run
an average of 10mb/s through our internet feed where most of the
traffic is web traffic which all goes through the squids.

Load balancer connection stats (notice that one of our squids is
actually down for maintenance today)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP xxx.xxx.xxx.xxx:8090 lc
-> syd-webcache-04 Route 1 297 882
-> syd-webcache-02 Route 1 297 816
-> syd-webcache-01 Route 1 291 1169
-> syd-webcache-03 Route 0 0 0

syd-webcache-01: $ uptime
14:28:28 up 50 days, 22:17, 1 user, load average: 0.14, 0.12, 0.09

syd-webcache-01: $ vmstat 5
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 750428 53064 38388 243800 8 7 4 1 9 5 4 2 91 3
0 0 750428 52224 38540 244844 0 0 222 43 845 620 3 4 88 4
0 0 750416 53492 38100 243768 5 0 270 72 865 571 5 3 88 4
1 1 750396 54108 37776 243300 19 0 351 82 988 770 7 6 82 5

800Mhz Pentium III (Coppermine) with 2Gb RAM and stripped SCSI disks

We've been running this setup for several years now and it has never
been a bottle neck for us (mind you we have never been slashdotted
either :) )

Cheers,

Cees Hek

Re: which reverse proxy for modperl?

am 13.12.2008 04:59:39 von Perrin Harkins

I'm proposing to give a talk at OSCON this year about choosing your
reverse proxy, comparing proxy servers and other parts of the stack
like mod_perl and FastCGI. I haven't seen a good comparison that hits
all the popular proxy servers (perlbal, pound, nginx, lighttpd,
apache, squid... I think I'm forgetting some) but I've wanted one
before.

- Perrin

Re: which reverse proxy for modperl?

am 13.12.2008 16:38:19 von mpeters

Perrin Harkins wrote:
> I haven't seen a good comparison that hits
> all the popular proxy servers (perlbal, pound, nginx, lighttpd,
> apache, squid... I think I'm forgetting some) but I've wanted one
> before.

If you could include varnish, I'd be really happy :)

--
Michael Peters
Plus Three, LP

Re: which reverse proxy for modperl?

am 14.12.2008 03:06:30 von Adam Prime

William A. Rowe, Jr. wrote:
> Jeff Pang wrote:
>> Hello,
>>
>> I have a modperl application on a host which is running with heavy load.
>> I have the plan to put a reverse proxy before it.
>> There are two well known reverse proxy software, one is Squid, another
>> is nginx.
>> Which one is better for modperl application? or is there any others
>> which are better than these two?
>
> And there is httpd which you are already running, and you can run a very
> low cost, separate instance with very minimal modules loaded. With the
> mod_proxy_balancer, you can shuffle off some of the traffic to a second,
> third machine for good measure.
>
> Is either better? Google both for problems to determine how many people
> are irritated with either.

I believe that there are a number of people that use lighttpd on the
frontend too, so you can add it to the list. All of them have their
pluses and minuses, and all of them will help relieve the load on your
server.

Adam

Re: which reverse proxy for modperl?

am 16.12.2008 19:47:24 von Frank Wiles

On Sat, 13 Dec 2008 03:02:00 +0100 (CET)
Jeff Pang wrote:

> Hello,
>
> I have a modperl application on a host which is running with heavy
> load. I have the plan to put a reverse proxy before it.
> There are two well known reverse proxy software, one is Squid,
> another is nginx. Which one is better for modperl application? or is
> there any others which are better than these two?

I've used mod_proxy, Perlbal, and nginx. These days nginx is my
favorite, it just works and has a ton of features. Also it appears to
have the most active community development, but that's subjective as I
don't follow them all very closely.

-------------------------------------------------------
Frank Wiles, Revolution Systems, LLC.
Personal : frank@wiles.org http://www.wiles.org
Work : frank@revsys.com http://www.revsys.com

Re: which reverse proxy for modperl?

am 17.12.2008 10:08:23 von Fred Moyer

On Fri, Dec 12, 2008 at 6:02 PM, Jeff Pang wrote:
> Hello,
>
> I have a modperl application on a host which is running with heavy load.
> I have the plan to put a reverse proxy before it.
> There are two well known reverse proxy software, one is Squid, another is
> nginx.
> Which one is better for modperl application? or is there any others which
> are better than these two?

Perlbal is my reverse proxy of choice. In addition to being able to
handle tens of thousands of connections to the client without breaking
a sweat, it is written in Perl and isn't that hard to customize once
you grok the event driven programming in it; that is a really big
selling point for me as a Perl programmer.

It also has a cool feature called reproxying. Take for example,
perlbal running as a reverse proxy to mod_perl. If you get a request
for a file or url resource such as an image, you can code mod_perl to
return headers with 'X-Reproxy-Url: http://someurl.com/file.txt' and
perlbal will asynchronously proxy that file from the resource to the
client, leaving mod_perl free to service new requests.

The load balancing feature in perlbal is excellent. It keeps track of
the number of busy httpd backends, automatically removing those which
don't respond, and allowing you to run many mod_perl servers behind
one perlbal server.

And Perlbal has a plugin architecture that you can use to code up
request rewrite or similar functionality as plugins.

Re: which reverse proxy for modperl?

am 17.12.2008 11:08:01 von Jeff Pang

------=_Part_96039_22071319.1229508481447
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Thanks for all the kind info and replying.
When you use reverse proxy, do you generally have more than one modperl bac=
kend servers?
In my case, I have only one modperl server. When I put a reverse-proxy in f=
ront of it, will it improve the performance?

Thanks.



> On Fri, Dec 12, 2008 at 6:02 PM, Jeff Pang wrote:
> > Hello,
> >
> > I have a modperl application on a host which is running with heavy load=
..
> > I have the plan to put a reverse proxy before it.
> > There are two well known reverse proxy software, one is Squid, another =
is
> > nginx.
> > Which one is better for modperl application? or is there any others whi=
ch
> > are better than these two?
>


Créez votre adresse électronique prenom.nom@laposte.net=20
1 Go d'espace de stockage, anti-spam et anti-virus intégrés.

------=_Part_96039_22071319.1229508481447
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Thanks for all the kind info and replying.
Wh=
en you use reverse proxy, do you generally have more than one modperl backe=
nd servers?
In my case, I have only one modperl server. When I put a r=
everse-proxy in front of it, will it improve the performance?

Th=
anks.



> On Fri, Dec 12, 2008 at 6:02 PM, Jeff Pan=
g wrote:
> > Hello,
> >
> > I have a modp=
erl application on a host which is running with heavy load.
> > =
I have the plan to put a reverse proxy before it.
> > There are =
two well known reverse proxy software, one is Squid, another is
> &=
gt; nginx.
> > Which one is better for modperl application? or i=
s there any others which
> > are better than these two?
>=
;



=3D_blank href=3Dhttp://www.laposte.net>Créez votre adresse é=
lectronique prenom.nom@laposte.net
1 Go d'espace de stockage, anti-spam=
et anti-virus intégrés.


------=_Part_96039_22071319.1229508481447--

Re: which reverse proxy for modperl?

am 17.12.2008 18:45:19 von Frank Wiles

On Wed, 17 Dec 2008 11:08:01 +0100 (CET)
Jeff Pang wrote:

> Thanks for all the kind info and replying.
> When you use reverse proxy, do you generally have more than one
> modperl backend servers? In my case, I have only one modperl server.
> When I put a reverse-proxy in front of it, will it improve the
> performance?

You can do it any way you like. I have systems that are setup using
Perlbal and/or nginx ( which I will call proxy ) in the following
situations:

1) proxy on 80 to apache1/mp1 on localhost:8000
2) proxy on 80 and 443 to apache1/mp1 on localhost:8000 and
apache2/mp2/perl 5.8 on localhost:8001
3) proxy on 80 and 443 to several different apache2/mp2 servers on a
local network.

On one system I even have several "instances" of Apache running on
localhost on various ports basically one instance per mod_perl
"application" due to the traffic patterns/memory usage of the
applications. Since one app is a HUGE memory hog and very light on
traffic it is setup with just two Apache child processes and the other
apps are setup with more children.

Hope this helps.

-------------------------------------------------------
Frank Wiles, Revolution Systems, LLC.
Personal : frank@wiles.org http://www.wiles.org
Work : frank@revsys.com http://www.revsys.com

Re: which reverse proxy for modperl?

am 17.12.2008 19:41:22 von Amiri Barksdale

I was just grappling with this problem last week, and after trying out
lighttpd, nginx, and varnish, I chose varnish.

I too run one single mod-perl server, but it's sort of complicated, with
file uploads and customizable pages and stuff. I found that neither lighttpd
or nginx really allowed me to handle large file uploads transparently
without strange pauses and what not--both pre-cache file uploads in full
before passing them back to the Apache/mod_perl server. I am sure I simply
couldn't figure out how to configure those features properly, but ease of
use is part of the game.

Varnish, on the other hand, was no harder to set up and begin to understand,
and it handled uploads well. There's some trickiness to setting up the rules
in your default.vcl, but it works nicely, and there was no need to fiddle
with the config just to allow uploads to go through quickly.

I'm contentedly serving static files and some dynamic-but-rarely-updated
html from my varnish cache now.

Amiri


Frank Wiles wrote:
>
> On Sat, 13 Dec 2008 03:02:00 +0100 (CET)
> Jeff Pang wrote:
>
>> Hello,
>>
>> I have a modperl application on a host which is running with heavy
>> load. I have the plan to put a reverse proxy before it.
>> There are two well known reverse proxy software, one is Squid,
>> another is nginx. Which one is better for modperl application? or is
>> there any others which are better than these two?
>
> I've used mod_proxy, Perlbal, and nginx. These days nginx is my
> favorite, it just works and has a ton of features. Also it appears to
> have the most active community development, but that's subjective as I
> don't follow them all very closely.
>
> -------------------------------------------------------
> Frank Wiles, Revolution Systems, LLC.
> Personal : frank@wiles.org http://www.wiles.org
> Work : frank@revsys.com http://www.revsys.com
>
>
>

--
View this message in context: http://www.nabble.com/which-reverse-proxy-for-modperl--tp209 86586p21058840.html
Sent from the mod_perl - General mailing list archive at Nabble.com.

Re: which reverse proxy for modperl?

am 17.12.2008 19:46:53 von Amiri Barksdale

Oh, one other wrinkle is ssl. I had to forgo proxying my ssl pages using
nginx, varnish or lighttpd. In all three cases I had to make apache listen
on my real IP address to port 443 for https from the internet, and only
allow it to listen on localhost:8080 for whatever was sent from the
reverse_proxy.

Amiri



amiribarksdale wrote:
>
> I was just grappling with this problem last week, and after trying out
> lighttpd, nginx, and varnish, I chose varnish.
>
> I too run one single mod-perl server, but it's sort of complicated, with
> file uploads and customizable pages and stuff. I found that neither
> lighttpd or nginx really allowed me to handle large file uploads
> transparently without strange pauses and what not--both pre-cache file
> uploads in full before passing them back to the Apache/mod_perl server. I
> am sure I simply couldn't figure out how to configure those features
> properly, but ease of use is part of the game.
>
> Varnish, on the other hand, was no harder to set up and begin to
> understand, and it handled uploads well. There's some trickiness to
> setting up the rules in your default.vcl, but it works nicely, and there
> was no need to fiddle with the config just to allow uploads to go through
> quickly.
>
> I'm contentedly serving static files and some dynamic-but-rarely-updated
> html from my varnish cache now.
>
> Amiri
>
>
> Frank Wiles wrote:
>>
>> On Sat, 13 Dec 2008 03:02:00 +0100 (CET)
>> Jeff Pang wrote:
>>
>>> Hello,
>>>
>>> I have a modperl application on a host which is running with heavy
>>> load. I have the plan to put a reverse proxy before it.
>>> There are two well known reverse proxy software, one is Squid,
>>> another is nginx. Which one is better for modperl application? or is
>>> there any others which are better than these two?
>>
>> I've used mod_proxy, Perlbal, and nginx. These days nginx is my
>> favorite, it just works and has a ton of features. Also it appears to
>> have the most active community development, but that's subjective as I
>> don't follow them all very closely.
>>
>> -------------------------------------------------------
>> Frank Wiles, Revolution Systems, LLC.
>> Personal : frank@wiles.org http://www.wiles.org
>> Work : frank@revsys.com http://www.revsys.com
>>
>>
>>
>
>

--
View this message in context: http://www.nabble.com/which-reverse-proxy-for-modperl--tp209 86586p21058934.html
Sent from the mod_perl - General mailing list archive at Nabble.com.

Re: which reverse proxy for modperl?

am 17.12.2008 20:04:45 von Frank Wiles

On Wed, 17 Dec 2008 10:46:53 -0800 (PST)
amiribarksdale wrote:

>
> Oh, one other wrinkle is ssl. I had to forgo proxying my ssl pages
> using nginx, varnish or lighttpd. In all three cases I had to make
> apache listen on my real IP address to port 443 for https from the
> internet, and only allow it to listen on localhost:8080 for whatever
> was sent from the reverse_proxy.

I agree with your last message that Varnish is another good option.
I'm betting you just had some configuration issues, as I use nginx and
Perlbal all the time for proxying both 80 and 443.

-------------------------------------------------------
Frank Wiles, Revolution Systems, LLC.
Personal : frank@wiles.org http://www.wiles.org
Work : frank@revsys.com http://www.revsys.com

Re: which reverse proxy for modperl?

am 17.12.2008 20:10:21 von Kurt Hansen

Hi,

amiribarksdale wrote:
> Oh, one other wrinkle is ssl. I had to forgo proxying my ssl pages using
> nginx, varnish or lighttpd. In all three cases I had to make apache listen
> on my real IP address to port 443 for https from the internet, and only
> allow it to listen on localhost:8080 for whatever was sent from the
> reverse_proxy.
>
> Amiri
>
I've only read some docs and reviews for nginx so far, but I thought it
supported ssl. Does it not? Do none of the 3?

Right now, I'm using the dual-apache setup on one server. The light
apache has ssl compiled in. It is the only one listening on the public
ports of 80 and 443; it then forwards everything to the mod_perl apache
on port 81.

I've been assuming that I could replace my light apache with ssl with
nginx with ssl with essentially the same setup. I'd move nginx to a
separate machine, though, so I would do same more load balancing and
fault tolerance. nginx would be listening in on port 80 and 443 and
forwarding things on.

Is this not possible?

Thanks!

Kurt

Re: which reverse proxy for modperl?

am 17.12.2008 20:17:46 von Frank Wiles

On Wed, 17 Dec 2008 14:10:21 -0500
Kurt Hansen wrote:

> Hi,
>
> amiribarksdale wrote:
> > Oh, one other wrinkle is ssl. I had to forgo proxying my ssl pages
> > using nginx, varnish or lighttpd. In all three cases I had to make
> > apache listen on my real IP address to port 443 for https from the
> > internet, and only allow it to listen on localhost:8080 for
> > whatever was sent from the reverse_proxy.
> >
> > Amiri
> >
> I've only read some docs and reviews for nginx so far, but I thought
> it supported ssl. Does it not? Do none of the 3?
>
> Right now, I'm using the dual-apache setup on one server. The light
> apache has ssl compiled in. It is the only one listening on the
> public ports of 80 and 443; it then forwards everything to the
> mod_perl apache on port 81.
>
> I've been assuming that I could replace my light apache with ssl with
> nginx with ssl with essentially the same setup. I'd move nginx to a
> separate machine, though, so I would do same more load balancing and
> fault tolerance. nginx would be listening in on port 80 and 443 and
> forwarding things on.
>
> Is this not possible?

It's definitely possible, I use nginx on both 80 and 443 all the time.

-------------------------------------------------------
Frank Wiles, Revolution Systems, LLC.
Personal : frank@wiles.org http://www.wiles.org
Work : frank@revsys.com http://www.revsys.com

Re: which reverse proxy for modperl?

am 17.12.2008 20:28:39 von Kurt Hansen

This is a multi-part message in MIME format.
--------------050207010503000602060809
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Thanks, Frank!

Good to know!

Take care,

Kurt


Frank Wiles wrote:
> On Wed, 17 Dec 2008 14:10:21 -0500
> Kurt Hansen wrote:
>
>
>> Hi,
>>
>> amiribarksdale wrote:
>>
>>> Oh, one other wrinkle is ssl. I had to forgo proxying my ssl pages
>>> using nginx, varnish or lighttpd. In all three cases I had to make
>>> apache listen on my real IP address to port 443 for https from the
>>> internet, and only allow it to listen on localhost:8080 for
>>> whatever was sent from the reverse_proxy.
>>>
>>> Amiri
>>>
>>>
>> I've only read some docs and reviews for nginx so far, but I thought
>> it supported ssl. Does it not? Do none of the 3?
>>
>> Right now, I'm using the dual-apache setup on one server. The light
>> apache has ssl compiled in. It is the only one listening on the
>> public ports of 80 and 443; it then forwards everything to the
>> mod_perl apache on port 81.
>>
>> I've been assuming that I could replace my light apache with ssl with
>> nginx with ssl with essentially the same setup. I'd move nginx to a
>> separate machine, though, so I would do same more load balancing and
>> fault tolerance. nginx would be listening in on port 80 and 443 and
>> forwarding things on.
>>
>> Is this not possible?
>>
>
> It's definitely possible, I use nginx on both 80 and 443 all the time.
>
> -------------------------------------------------------
> Frank Wiles, Revolution Systems, LLC.
> Personal : frank@wiles.org http://www.wiles.org
> Work : frank@revsys.com http://www.revsys.com
>
>
>

--------------050207010503000602060809
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit







Thanks, Frank!



Good to know!



Take care,



Kurt





Frank Wiles wrote:

type="cite">
On Wed, 17 Dec 2008 14:10:21 -0500
Kurt Hansen wrote:



Hi,

amiribarksdale wrote:


Oh, one other wrinkle is ssl. I had to forgo proxying my ssl pages
using nginx, varnish or lighttpd. In all three cases I had to make
apache listen on my real IP address to port 443 for https from the
internet, and only allow it to listen on localhost:8080 for
whatever was sent from the reverse_proxy.

Amiri



I've only read some docs and reviews for nginx so far, but I thought
it supported ssl. Does it not? Do none of the 3?

Right now, I'm using the dual-apache setup on one server. The light
apache has ssl compiled in. It is the only one listening on the
public ports of 80 and 443; it then forwards everything to the
mod_perl apache on port 81.

I've been assuming that I could replace my light apache with ssl with
nginx with ssl with essentially the same setup. I'd move nginx to a
separate machine, though, so I would do same more load balancing and
fault tolerance. nginx would be listening in on port 80 and 443 and
forwarding things on.

Is this not possible?



It's definitely possible, I use nginx on both 80 and 443 all the time.

-------------------------------------------------------
Frank Wiles, Revolution Systems, LLC.
Personal :
Work :







--------------050207010503000602060809--

Re: which reverse proxy for modperl?

am 17.12.2008 21:53:41 von David Nicol

ldirectord will load-balance any number of back-ends; state has to be
maintained on all of them the same though

Re: which reverse proxy for modperl?

am 18.12.2008 03:00:10 von Jeff Pang

------=_Part_133767_14950354.1229565610630
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


> Message du 17/12/08 21:54
> De : "David Nicol"
> ldirectord will load-balance any number of back-ends; state has to be
> maintained on all of them the same though
>


Yup, ldirectord (a scheduler for LVS) is right for load-balancing.
But it is only a tcp-header-rewrite (or so called NAT or reverse NAT), not =
a proxy server.

One of our applications, have two LVS before 200+ Squid boxes, the backend =
are hundreds of fastcgi servers. LVS and Squid run with DR mode, could reac=
h to 10G+ traffic.


--
Jeff Pang
http://home.arcor.de/pangj/

Créez votre adresse électronique prenom.nom@laposte.net=20
1 Go d'espace de stockage, anti-spam et anti-virus intégrés.

------=_Part_133767_14950354.1229565610630
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


> Message du 17/12/08 21:54
> De =
: "David Nicol"
> ldirectord will load-balance any number=
of back-ends; state has to be
> maintained on all of them the same=
though
>


Yup, ldirectord (a scheduler for LVS) is=
right for load-balancing.
But it is only a tcp-header-rewrite (or so =
called NAT or reverse NAT), not a proxy server.

One of our appli=
cations, have two LVS before 200+ Squid boxes, the backend are hundreds of =
fastcgi servers. LVS and Squid run with DR mode, could reach to 10G+ traffi=
c.


--
Jeff Pang
http://home.arcor.de/pangj/

<=
BR>
ank href=3Dhttp://www.laposte.net>Créez votre adresse électro=
nique prenom.nom@laposte.net
1 Go d'espace de stockage, anti-spam et an=
ti-virus intégrés.


------=_Part_133767_14950354.1229565610630--