huge httpd processes

Hi clint

This script looks helpful. I don't completely understand (even after lookin=
g at the documentation for Linux::Smaps) it though

Looking at this output:

##before a bit hit on an expensive part of the web site
VMSIZE: 119472 kb
RSS: 96184 kb total
94904 kb shared
176 kb private clean
1104 kb private dirty

##after said hit
VMSIZE: 120408 kb
RSS: 98876 kb total
89136 kb shared
324 kb private clean
9416 kb private dirty


What this seems to tell me is:

i. The total VIRT size of the process before was 119 Mb
ii. Before - the process shared 94 Mb with other processes
iii. After - shared has gone down and private dirty up - does this mean tha=
t this process now 'owns' this memory and it can't be used by other process=
es?

The other thing which confuses me is that this seems to be completely diffe=
rent from TOP.


PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16210 apache 16 0 117m 97m 4788 S 0.0 9.6 0:01.04 httpd

In particular TOP'S concept of SHR seems to be completely different from Sm=
ap's.

Do either of these bear any relation to mod_perl's shared memory which you =
can use by preloading modules at startup?

Sorry; I am a complete beginner at this

Thanks

Justin
(ps this is sent from my work account for this mailing list justin.wyllie@s=
ophos.com )









> Hi Justin
>
>>
>> I'm wondering if anyone can advise me on how I could go about trying
>> to understand where this 90 Mbs is comming from? Some of it must be
>> the mod_perl and apache binaries - but how much should they be, and
>> apart from the 6mb in shared memory for my pre-loaded modules, where
>> is it all comming from?
>
> You don't mention what platform you are on, but if it is linux, then
> check out this blog
>
> http://bmaurer.blogspot.com/2006/03/memory-usage-with-smaps. html
>
> and specifically this script
>
> http://www.contrib.andrew.cmu.edu/%7Ebmaurer/memory/smem.pl
>
>
> which will give you a much more accurate idea about how much memory is
> shared and where it is being used.
>
> As Perrin said, Perl data and code both go on the heap, so you won't be
> able to separate those two out with this tool, but combining smem.pl
> with loading modules one by one will get you a long way to a diagnosis.
>
> clint

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United=
Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: huge httpd processes

> i. The total VIRT size of the process before was 119 Mb
> ii. Before - the process shared 94 Mb with other processes
> iii. After - shared has gone down and private dirty up - does this
> mean that this process now 'owns' this memory and it can't be used by
> other processes?

Correct
>
> The other thing which confuses me is that this seems to be completely
> different from TOP.

> In particular TOP'S concept of SHR seems to be completely different
> from Smap's.

Correct - top does not report shared memory correctly.
>
> Do either of these bear any relation to mod_perl's shared memory which
> you can use by preloading modules at startup?

Yes - as I understand it (somebody please correct me if I'm wrong), all
of the C libraries (eg XS modules) that you preload will remain shared
between your processes. Also, your heap (Perl code and vars) will start
off as shared in a new process. As each page in the heap becomes dirty,
the heap will become less shared.

But given that forking a new process is cheap with copy-on-write, just
make sure that your children exit on a regular basis, eg after 1000
requests. That makes it unlikely that they will grow too much before
being replaced by a fresh new maximally shared process.

And, as has been discuss in other emails, don't consume lots of extra
memory in your parent process when you don't need to.

For reference, here's the smem.pl output from my live site:

PARENT:
-------
VMSIZE: 160212 kb
RSS: 66516 kb total
62716 kb shared
784 kb private clean
3016 kb private dirty

CHILD:
------
VMSIZE: 156856 kb
RSS: 63992 kb total
45512 kb shared
0 kb private clean
18480 kb private dirty

>
clint

Re: huge httpd processes

On Wed, Sep 30, 2009 at 10:47 AM, Clinton Gormley wr=
ote:
>> Do either of these bear any relation to mod_perl's shared memory which
>> you can use by preloading modules at startup?
>
> Yes - as I understand it (somebody please correct me if I'm wrong), all
> of the C libraries (eg XS modules) that you preload will remain shared
> between your processes. =A0Also, your heap (Perl code and vars) will star=
t
> off as shared in a new process. =A0As each page in the heap becomes dirty=
,
> the heap will become less shared.

The short answer is that absolutely everything starts off shared
whenever you fork any program, because of copy-on-write. Over time,
as pages get written to, they are no longer shared.

> But given that forking a new process is cheap with copy-on-write, just
> make sure that your children exit on a regular basis, eg after 1000
> requests.

I'd start with something really low and work up, like 10 requests, or
use Apache::SizeLimit.

- Perrin