experiencing Out of memory errors

We have an Apache/mod_perl application running under Windows Server 2003
that periodically experiences
"Out of memory" errors (they appear in the Apache error logs) which
forces Apache to restart. Our application does, in fact, use a lot of
memory, and we believe that this is not due to a bug or memory leak.
What we would like is to run this application in a 64 bit environment, thus
allowing us to use more than the 2 GB memory that the Apache process ('httpd')
is limited to (the "Out of memory" errors always occur as the memory usage
of the httpd process approaches this 2 GB limit).

We know that we will need to run a 64 bit version of Windows on 64 bit
hardware to accomplish this. Furthermore, we know that ActiveState has a
binary Perl install of a 64 bit Perl. However, we're not aware of either a
64 bit Apache (though recently I found the blackdot site -
http://www.blackdot.be/ - that provides a Windows compatible binary build
of 64 bit apache, though I know very little about it) or a 64 bit mod_perl.
However, we're also not sure if either of these are needed to escape the
2 GB memory limitation. E.g., can 64 bit Perl be used with a 32 bit
Apache and/or mod_perl?

Any help with accomplishing what we need would be greatly appreciated,
including the possibility of hiring someone on a contract basis to
help us.



OS: Microsoft Windows Server 2003
Perl: v 5.8.9 ActiveState build 826
Apache 2.2.11
mod_perl 2 (not sure of the exact version, but very recent)

Re: experiencing Out of memory errors

Hi John,

On Wed, 2011-01-26 at 10:17 -0500, John Deighan wrote:
> However, we're also not sure if either of these are needed to escape the
> 2 GB memory limitation. E.g., can 64 bit Perl be used with a 32 bit
> Apache and/or mod_perl?

You should use the same architecture for Perl and Apache.

> Any help with accomplishing what we need would be greatly appreciated,
> including the possibility of hiring someone on a contract basis to
> help us.

Probably because of the fact that the Apache foundation does not ship
64-bit builds for Windows (which is a shame!), ActiveState does not ship
a mod_perl with their 64- bit Perl. Compiling your own mod_perl would be
possible but you need to use preferably the same compiler that was used
for Apache + perl.

If you would want to switch to StrawberryPerl 5.12.x 64-bit, you can use
this pre-compiled mod_perl:
http://strawberryperl.com/package/kmx/mod_perl/

--
Mike

Re: experiencing Out of memory errors

Hi

Am 27.01.2011 um 12:57 schrieb Michiel Beijen :

> Hi John,
>
> On Wed, 2011-01-26 at 10:17 -0500, John Deighan wrote:
>> However, we're also not sure if either of these are needed to escape the
>> 2 GB memory limitation. E.g., can 64 bit Perl be used with a 32 bit
>> Apache and/or mod_perl?
>
> You should use the same architecture for Perl and Apache.
>
>> Any help with accomplishing what we need would be greatly appreciated,
>> including the possibility of hiring someone on a contract basis to
>> help us.
>
> Probably because of the fact that the Apache foundation does not ship
> 64-bit builds for Windows (which is a shame!), ActiveState does not ship
> a mod_perl with their 64- bit Perl. Compiling your own mod_perl would be
> possible but you need to use preferably the same compiler that was used
> for Apache + perl.
>
> If you would want to switch to StrawberryPerl 5.12.x 64-bit, you can use
> this pre-compiled mod_perl:
> http://strawberryperl.com/package/kmx/mod_perl/

Is mod_perl compiled with MinGW or with cl.exe for windows?

Seems that apxs suggest MinGW.

I got the same problem with mod_ruby for windows.

Best
Zeno

Re: experiencing Out of memory errors

I'd forgotten about Strawberry Perl (came across it about a year ago, but never got around to trying
it). One question, though: I'm thinking there's not much point to using a 64-bit version of mod_perl
unless I'm using it with a 64-bit version of Apache. Where can I get a 64-bit version of Apache that
will with with the mod_perl below?

On 1/27/2011 6:57 AM, Michiel Beijen wrote:
> Hi John,
>
> ... etc.
>
> If you would want to switch to StrawberryPerl 5.12.x 64-bit, you can use
> this pre-compiled mod_perl:
> http://strawberryperl.com/package/kmx/mod_perl/
>
> --
> Mike

------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: experiencing Out of memory errors

Hi John,

On Thu, January 27, 2011 15:10, John Deighan wrote:
> One question, though: I'm thinking there's not much point to using a
> 64-bit version of mod_perl
> unless I'm using it with a 64-bit version of Apache. Where can I get a
> 64-bit version of Apache that
> will with with the mod_perl below?

You're right, you should not use 64-bit perl with a 32-bit apache
You should get the one from Blackdot probably:
http://www.blackdot.be/?inc=apache/binaries

--
Mike

Re: experiencing Out of memory errors

--Apple-Mail-31-38365877
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=us-ascii


On 26 Jan 2011, at 15:17, John Deighan wrote:

> What we would like is to run this application in a 64 bit environment, =
thus
> allowing us to use more than the 2 GB memory that the Apache process =
('httpd')
> is limited to (the "Out of memory" errors always occur as the memory =
usage
> of the httpd process approaches this 2 GB limit).

Can I just say: WTF? 2G in an Apache? Surely there's a better way of =
architecting
this?=

--Apple-Mail-31-38365877
Content-Disposition: attachment;
filename=smime.p7s
Content-Type: application/pkcs7-signature;
name=smime.p7s
Content-Transfer-Encoding: base64

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEH AQAAoIIDWjCCA1Yw
ggI+oAMCAQICAQEwCwYJKoZIhvcNAQEFMEoxGDAWBgNVBAMMD0RhdmUgSG9k Z2tpbnNvbjELMAkG
A1UEBhMCR0IxITAfBgkqhkiG9w0BCQEWEmRhdmVob2RnQGdtYWlsLmNvbTAe Fw0xMDA1MjYxMjM3
NDVaFw0xMTA1MjYxMjM3NDVaMEoxGDAWBgNVBAMMD0RhdmUgSG9kZ2tpbnNv bjELMAkGA1UEBhMC
R0IxITAfBgkqhkiG9w0BCQEWEmRhdmVob2RnQGdtYWlsLmNvbTCCASIwDQYJ KoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBAKNEsd4Pz8mjFrM97NLC6WwfXMJEFGgw9+1j5RPhVvFC D+jPATGVzON1lVaB
9C5vMQov/hzC6/B0bynpNzQnZC+v3Vy+Flgro+XNh1rMPtqqK757bjwEQk/3 deB3yHuT6qCzj3Mb
ze5uuGYGJFyzOeKjteUp0UqrgiMl587qE1OGSrvMWCKSAo1nNgXA8FXnn/nN jgLrocwM8GrzMfvG
pA3bpjdKeZxSN4KgWnHZNYMb50CNOs6epET0snvEpzjojjhCdHTQUWZ+FZqF nN8aNmm2/hB/D2vO
8XBF+wNzsbljDXxL7wvgJEIqvvEFhvRYzJfkC5iWsSfss5nxeNINV2MCAwEA AaNJMEcwDgYDVR0P
AQH/BAQDAgeAMBYGA1UdJQEB/wQMMAoGCCsGAQUFBwMEMB0GA1UdEQQWMBSB EmRhdmVob2RnQGdt
YWlsLmNvbTANBgkqhkiG9w0BAQUFAAOCAQEAiwQ36Blz4Ud5zViOOt13qTf6 trx+mD2Q4a21C7rJ
WOjSclwaCa97VW9ZNxdXkPTG+QQY0kDNNEp601rNbmRUoSoHOAKtWwdeDL7C U1jStW5Wld6443Bk
lgepnkuK5Wgau0FZo20L9seKWkG1A9PpbanwSsyazVaPl8hNVzao5SCjvly0 9x9rE2ba4uv05bZj
0AZbOjKX25cNoBlc8YBgrdY0UpMz8tMwQEeIlJtzzF/YN3Hcvd279wSuVRZK j0UCufduHILVkdit
xWKgWMKRoGbBuV6KUQEs8JR6XwHq6bSCu326MIMdSx7HpqCfr+BtLeeCy6Cm 58o/jyt9cpFgUTGC
ApkwggKVAgEBME8wSjEYMBYGA1UEAwwPRGF2ZSBIb2Rna2luc29uMQswCQYD VQQGEwJHQjEhMB8G
CSqGSIb3DQEJARYSZGF2ZWhvZGdAZ21haWwuY29tAgEBMAkGBSsOAwIaBQCg ggEfMBgGCSqGSIb3
DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTExMDEyODAwMDU0 NlowIwYJKoZIhvcN
AQkEMRYEFNKhYScNuHa39mH2eo8mo+oflphlMF4GCSsGAQQBgjcQBDFRME8w SjEYMBYGA1UEAwwP
RGF2ZSBIb2Rna2luc29uMQswCQYDVQQGEwJHQjEhMB8GCSqGSIb3DQEJARYS ZGF2ZWhvZGdAZ21h
aWwuY29tAgEBMGAGCyqGSIb3DQEJEAILMVGgTzBKMRgwFgYDVQQDDA9EYXZl IEhvZGdraW5zb24x
CzAJBgNVBAYTAkdCMSEwHwYJKoZIhvcNAQkBFhJkYXZlaG9kZ0BnbWFpbC5j b20CAQEwDQYJKoZI
hvcNAQEBBQAEggEAXMZSd2Ey5foE1LHw1cn3acZ+2VlOaJRYKuvGvVph6sHO 0rLtwfneFWrbHYuB
wOTthIK8/3OWMaS0Wq0gOfUJEhmRWJd27cz/swXOxOerWcDls5IuRv7j02Ek naTVaP/xeOfwDy5s
lWkLOKl1kuLq8NZg6IJHLLOA7XkpaP+J6IaC7rTxCAdyWmrv0gw2pGW90tSx PqsgQJsstDCxPumq
hpohHtxD0j781sqEjXVpM1dYyYk1YF0UZ4zAjtk1qXePV5yLCkvkx9gUcV9a iBs9GnUCi8vwIXFc
x5vuDOd/HInyEd53IhsLJHgTK5VDcG4ldfF9RNbtzuUI+3L9qLCzkAAAAAAA AA==

--Apple-Mail-31-38365877--

Re: experiencing Out of memory errors

On 01/27/2011 07:05 PM, Dave Hodgkinson wrote:

> Can I just say: WTF? 2G in an Apache? Surely there's a better way of architecting
> this?

One thing to remember is that he's running Windows which doesn't have
Copy-On-Write memory, so depending on what he's doing it might not take
up as much memory if it were being run on Linux (or other OS with COW).

Another thing that maybe the OP should look at (if he hasn't already) is
to run a proxy in front of the main mod_perl application. Even if the
proxy is on the same machine it will help because you can reduce the
number of memory-heavy mod_perl processes/threads and handle the same
number of connections.

But, even after all that I have applications where we consistently run
3-4G just for mod_perl/Apache.

--
Michael Peters
Plus Three, LP

Re: experiencing Out of memory errors

Michael Peters schrieb am 27.01.2011 um 19:14 (-0500):

> But, even after all that I have applications where we consistently
> run 3-4G just for mod_perl/Apache.

But surely not in one process as the OP said he'd like to do?

--
Michael Ludwig

Re: experiencing Out of memory errors

On 01/27/2011 07:41 PM, Michael Ludwig wrote:
> Michael Peters schrieb am 27.01.2011 um 19:14 (-0500):
>
>> But, even after all that I have applications where we consistently
>> run 3-4G just for mod_perl/Apache.
>
> But surely not in one process as the OP said he'd like to do?

No you're right, but I'm guessing he might be running a threaded MPM, so
single process, multiple threads.

--
Michael Peters
Plus Three, LP

Re: experiencing Out of memory errors

On 1/27/2011 7:16 PM, Michael Peters wrote:
> On 01/27/2011 07:41 PM, Michael Ludwig wrote:
>> Michael Peters schrieb am 27.01.2011 um 19:14 (-0500):
>>
>>> But, even after all that I have applications where we consistently
>>> run 3-4G just for mod_perl/Apache.
>>
>> But surely not in one process as the OP said he'd like to do?
>
> No you're right, but I'm guessing he might be running a threaded MPM, so single process,
> multiple threads.

Exactly. Even constraining httpd to smaller stacks is unlikely to be wise
with mod_perl running, consider the default is 256kb IIRC. Shrinking this
to 128k obviously is a big help, but is unrealistic.

I agree with Peters, run a proxy in front of the server hosting mod_perl.
Even when we ship binary 64 bit for win32, there are far too many broken
perl and similar modules which don't expect sizeof(long*) > sizeof(long),
which has tripped up many porters. Most of these defects are gone from
apr and httpd, and perl and modperl can likely catch up quickly, but to
isolate every possible modperl XS package on cpan and identify all those
which make stupid long x = (long)xptr; assignments will be arduous.

Running a proxy in front of mod_perl on Win32

On Fri, January 28, 2011 01:14, Michael Peters wrote:
> Another thing that maybe the OP should look at (if he hasn't already) is
> to run a proxy in front of the main mod_perl application. Even if the
> proxy is on the same machine it will help because you can reduce the
> number of memory-heavy mod_perl processes/threads and handle the same
> number of connections.

I have read that advice before; then it was because of this bug concerning
running mod_ssl + mod_perl at the same time on Win32:
https://issues.apache.org/bugzilla/show_bug.cgi?id=36751

Unfortunately the bug still exists but the bug report was set to invalid
because it did not get proper attention.

What would be the best (windows-style) way of running a proxy in front of
apache?

Insights are welcome!
--
Mike

Re: Running a proxy in front of mod_perl on Win32

Am Fr, 28.01.2011, 09:31, schrieb Michiel Beijen:
> On Fri, January 28, 2011 01:14, Michael Peters wrote:
>> Another thing that maybe the OP should look at (if he hasn't already) is
>> to run a proxy in front of the main mod_perl application. Even if the
>> proxy is on the same machine it will help because you can reduce the
>> number of memory-heavy mod_perl processes/threads and handle the same
>> number of connections.
>
> I have read that advice before; then it was because of this bug concerning
> running mod_ssl + mod_perl at the same time on Win32:
> https://issues.apache.org/bugzilla/show_bug.cgi?id=36751
>
> Unfortunately the bug still exists but the bug report was set to invalid
> because it did not get proper attention.
>
> What would be the best (windows-style) way of running a proxy in front of
> apache?
>
> Insights are welcome!
> --
> Mike
>
>

(Linux-style) I use nginx and it works great (by coping with ssl and
keep-alives) whether you allow it to cache requests or not. It can serve
thousands of simultaneous connections and still has just a memory
footprint of a few mb. I dont know if the event-model works as great under
windows but there are windows builds available on the website
(http://nginx.org) so it should be worth a try.

Hendrik

Re: Running a proxy in front of mod_perl on Win32

On Fri, Jan 28, 2011 at 3:31 AM, Michiel Beijen wrote:
> On Fri, January 28, 2011 01:14, Michael Peters wrote:
>> Another thing that maybe the OP should look at (if he hasn't already) is
>> to run a proxy in front of the main mod_perl application. Even if the
>> proxy is on the same machine it will help because you can reduce the
>> number of memory-heavy mod_perl processes/threads and handle the same
>> number of connections.
>
> I have read that advice before; then it was because of this bug concerning
> running mod_ssl + mod_perl at the same time on Win32:
> https://issues.apache.org/bugzilla/show_bug.cgi?id=36751

Even if that problem is fixed, the advice to use a proxy server is good.

> What would be the best (windows-style) way of running a proxy in front of
> apache?

There are many proxy servers available. If you search the list
archives for "proxy" you should find many posts discussing the pros
and cons of various ones.

- Perrin