Caching a hash - am I missing something?

Caching a hash - am I missing something?

am 20.08.2008 01:35:29 von Chris Faust

This is a multi-part message in MIME format.

------=_NextPart_000_0321_01C90232.BE331950
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit

Hi,



This might be a little off topic, I hope it's OK to post. I'm not positive
if mod_perl matters or not because it's a little confusing to me.



I've taken over some pretty old code that I'm updating and making mp2
content handlers out of. The main script is a standard cgi script
"start.cgi" there is nothing special in the apache conf for it.





SetHandler perl-script

PerlFixupHandler My::Fixup

PerlResponseHandler ModPerl::PerlRun

PerlOptions +ParseHeaders

DirectoryIndex start.cgi

Options +ExecCGI +Indexes

allow from all





start.cgi calls a custom module (use CustomModule;) which exports a bunch of
subs, for example foobar and all over the place in the subs that are
exported from CustomModule I see code like





sub foobar {

my $key = @_;



if ($cache{$key}) {

return $cache{$key};

} else {

my $do_some_query = xxxx;

$cache{$key} = $do_some_query_results

return $cache{$key};

}

}



My question is isn't the "else" in foobar always going to be true anyplace
where start.cgi is calling "&foobar('somekey')"??????

I don't understand how %cache could already be populate from a previous
browser request or something? I'm I just missing something stupid?



FYI this was all running before on a version of Debian with old apache and
old mod_perl (early 1.99 and 2.0.x). I've updated mod_perl to 2.0.2 and
apache to 2.2 and everything still works.



TIA!!

-Chris
















------=_NextPart_000_0321_01C90232.BE331950
Content-Type: text/html;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns=3D"http://www.w3.org/TR/REC-html40">


charset=3Dus-ascii">









style=3D'font-size:10.0pt;
font-family:Arial'>Hi,



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'>This might be a little off topic, I hope it’s =
OK to
post. I’m not positive if mod_perl matters or not because =
it’s a little
confusing to me.



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'>I’ve taken over some pretty old code that =
I’m updating
and making mp2 content handlers out of. The main script is a standard =
cgi
script “start.cgi” there is nothing special in the apache =
conf for
it.



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'>        =
<Directory /xxx/>



style=3D'font-size:10.0pt;
font-family:Arial'>        SetHandler
perl-script



style=3D'font-size:10.0pt;
font-family:Arial'>        =
PerlFixupHandler
My::Fixup



style=3D'font-size:10.0pt;
font-family:Arial'>       
PerlResponseHandler ModPerl::PerlRun



style=3D'font-size:10.0pt;
font-family:Arial'>        =
PerlOptions
+ParseHeaders



style=3D'font-size:10.0pt;
font-family:Arial'>        =
DirectoryIndex start.cgi



style=3D'font-size:10.0pt;
font-family:Arial'>        Options =
+ExecCGI
+Indexes



style=3D'font-size:10.0pt;
font-family:Arial'>        allow from =
all



style=3D'font-size:10.0pt;
font-family:Arial'>       
</Directory>



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'>start.cgi calls a custom module (use CustomModule;) =
which
exports a bunch of subs, for example foobar and all over the place in =
the subs
that are exported from CustomModule I see code =
like



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'>sub foobar {



style=3D'font-size:10.0pt;
font-family:Arial'>         =
   my
$key =3D @_;



style=3D'font-size:10.0pt;
font-family:Arial'>         =
  



style=3D'font-size:10.0pt;
font-family:Arial'>         =
   if
($cache{$key}) {



style=3D'font-size:10.0pt;
font-family:Arial'>         =
            &=
nbsp;  return
$cache{$key};



style=3D'font-size:10.0pt;
font-family:Arial'>         =
   }
else {



style=3D'font-size:10.0pt;
font-family:Arial'>         =
            &=
nbsp;  my
$do_some_query =3D xxxx;



style=3D'font-size:10.0pt;
font-family:Arial'>         =
            &=
nbsp;  $cache{$key}
=3D $do_some_query_results



style=3D'font-size:10.0pt;
font-family:Arial'>         =
            &=
nbsp;  return
$cache{$key};



style=3D'font-size:10.0pt;
font-family:Arial'>         =
   }



style=3D'font-size:10.0pt;
font-family:Arial'>}



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'>My question is isn’t the “else” in =
foobar
always going to be true anyplace where start.cgi is calling =
“&foobar(‘somekey’)”??????
<=
/font>



style=3D'font-size:10.0pt;
font-family:Arial'>I don’t understand how %cache could already be
populate from a previous browser request or something? I’m I just =
missing
something stupid?



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'>FYI this was all running before on a version of =
Debian with old
apache and old mod_perl (early 1.99 and 2.0.x). I’ve updated =
mod_perl to 2.0.2
and apache to 2.2 and everything still =
works.



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'>TIA!!



style=3D'font-size:10.0pt;
font-family:Arial'>-Chris



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'> 



style=3D'font-size:10.0pt;
font-family:Arial'> 









------=_NextPart_000_0321_01C90232.BE331950--

Re: Caching a hash - am I missing something?

am 20.08.2008 01:39:16 von Tyler Gee

On Tue, Aug 19, 2008 at 5:35 PM, Chris Faust wrote:
> Hi,
>
>
>
> This might be a little off topic, I hope it's OK to post. I'm not positive
> if mod_perl matters or not because it's a little confusing to me.
>
>
>
> I've taken over some pretty old code that I'm updating and making mp2
> content handlers out of. The main script is a standard cgi script
> "start.cgi" there is nothing special in the apache conf for it.
>
>
>
>
>
> SetHandler perl-script
>
> PerlFixupHandler My::Fixup
>
> PerlResponseHandler ModPerl::PerlRun
>
> PerlOptions +ParseHeaders
>
> DirectoryIndex start.cgi
>
> Options +ExecCGI +Indexes
>
> allow from all
>
>

>
>
>
> start.cgi calls a custom module (use CustomModule;) which exports a bunch of
> subs, for example foobar and all over the place in the subs that are
> exported from CustomModule I see code like
>
>
>
>
>
> sub foobar {
>
> my $key = @_;
>
>
>
> if ($cache{$key}) {
>
> return $cache{$key};
>
> } else {
>
> my $do_some_query = xxxx;
>
> $cache{$key} = $do_some_query_results
>
> return $cache{$key};
>
> }
>
> }
>
>
>
> My question is isn't the "else" in foobar always going to be true anyplace
> where start.cgi is calling "&foobar('somekey')"??????
>
> I don't understand how %cache could already be populate from a previous
> browser request or something? I'm I just missing something stupid?

%cache is defined outside the scope of the sub so it will persist for
the lifetime of the apache server. The very first time
foobar('somekey') is called it will do the query lookup, the next time
it will return from cache.

>
>
>
> FYI this was all running before on a version of Debian with old apache and
> old mod_perl (early 1.99 and 2.0.x). I've updated mod_perl to 2.0.2 and
> apache to 2.2 and everything still works.
>
>
>
> TIA!!
>
> -Chris
>
>
>
>
>
>
>
>
>
>
>
>
>
>



--
~Tyler

Re: Caching a hash - am I missing something?

am 20.08.2008 09:28:58 von aw

W. Tyler Gee wrote:
> On Tue, Aug 19, 2008 at 5:35 PM, Chris Faust wrote:
>> Hi,
>>
>>
>>
>> This might be a little off topic, I hope it's OK to post. I'm not positive
>> if mod_perl matters or not because it's a little confusing to me.
>>
>>
>>
>> I've taken over some pretty old code that I'm updating and making mp2
>> content handlers out of. The main script is a standard cgi script
>> "start.cgi" there is nothing special in the apache conf for it.
>>
>>
>>
>>
>>
>> SetHandler perl-script
>>
>> PerlFixupHandler My::Fixup
>>
>> PerlResponseHandler ModPerl::PerlRun
>>
>> PerlOptions +ParseHeaders
>>
>> DirectoryIndex start.cgi
>>
>> Options +ExecCGI +Indexes
>>
>> allow from all
>>
>>

>>
>>
>>
>> start.cgi calls a custom module (use CustomModule;) which exports a bunch of
>> subs, for example foobar and all over the place in the subs that are
>> exported from CustomModule I see code like
>>
>>
>>
>>
>>
>> sub foobar {
>>
>> my $key = @_;
>>
>>
>>
>> if ($cache{$key}) {
>>
>> return $cache{$key};
>>
>> } else {
>>
>> my $do_some_query = xxxx;
>>
>> $cache{$key} = $do_some_query_results
>>
>> return $cache{$key};
>>
>> }
>>
>> }
>>
>>
>>
>> My question is isn't the "else" in foobar always going to be true anyplace
>> where start.cgi is calling "&foobar('somekey')"??????
>>
>> I don't understand how %cache could already be populate from a previous
>> browser request or something? I'm I just missing something stupid?
>
> %cache is defined outside the scope of the sub so it will persist for
> the lifetime of the apache server. The very first time
> foobar('somekey') is called it will do the query lookup, the next time
> it will return from cache.
>
I believe the above is almost, but not totally true. It should probably
be "for the lifetime of this particular apache child".
Each apache child process has it's own copy of the above code, and it's
own copy of the above "global" (*) %cache hash. Thus whether the first
or second part of the if will run, depends of the previous history of
the particular apache child which handles the current request. If this
particular child has already previously accessed the same hash key, it
will server it from (it's own) cache, and otherwise it will execute the
query to create the key (in it's own cache).
Apache children are created, and die, as directed by the main Apache
process configuration, and HTTP requests are served more or less at
random, by whichever child is available when the request comes in.
Is is thus quite possible for instance that the first 10 requests for a
particular hash key would each be handled by a different apache child,
and would each result in a query; then the 11th request would be handled
by a child that has already accessed this same key before, and thus
served from cache; the 13th request would be handled by a brand-new
apache child just created, thus would re-do the query, etc..

I this would be useful, I believe that it would be possible to avoid
this, by "priming" the mod_perl module during the initial start of
Apache, before it forks into children. Then each new child (being a
copy of the main apache) would start it's life with a number of keys
already in its cache. Of course this would work only if the contents of
the keys of the hash are never modified while apache is running.
And it probably involves some delicate mod_perl programming to do the
priming process.

(*) like the previous contributor, I guess that the %cache hash is
somehow global, because it is not declared within the sub that you show.
Whether it really is though, may depend on other code that we don't
see here. But anyway, "global" would mean only "global to this apache
child", not to the whole apache server.

And something that I don't know at all, is how this all works out with a
threaded apache, such as under Windows e.g.

André

RE: Caching a hash - am I missing something?

am 20.08.2008 12:22:38 von Chris Faust

This is a multi-part message in MIME format.

------_=_NextPart_001_01C902AE.AB8BDB4E
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Thanks Andre and Tyler.
=20
Believe it or not I never even thought of that until you guys pointed it =
out (that the hash would live for the the life of the apache child). I =
can be such a fool sometimes :)
=20
In this case I assume %cache is global - it only appers in that sub, =
there is no "my %cache =3D ()" any place else in the pm file and there =
is no "Use strict".
=20
I guess I'll just keep it the same way except I'll "use strict" and I'll =
declare "my %cache =3D ()" outside that sub.
=20
Thanks Guys!!
-Chris

________________________________

From: Andr=E9 Warnier [mailto:aw@ice-sa.com]
Sent: Wed 8/20/2008 3:28 AM
To: mod_perl list
Cc: W. Tyler Gee
Subject: Re: Caching a hash - am I missing something?



W. Tyler Gee wrote:
> On Tue, Aug 19, 2008 at 5:35 PM, Chris Faust =
wrote:
>> Hi,
>>
>>
>>
>> This might be a little off topic, I hope it's OK to post. I'm not =
positive
>> if mod_perl matters or not because it's a little confusing to me.
>>
>>
>>
>> I've taken over some pretty old code that I'm updating and making mp2
>> content handlers out of. The main script is a standard cgi script
>> "start.cgi" there is nothing special in the apache conf for it.
>>
>>
>>
>>
>>
>> SetHandler perl-script
>>
>> PerlFixupHandler My::Fixup
>>
>> PerlResponseHandler ModPerl::PerlRun
>>
>> PerlOptions +ParseHeaders
>>
>> DirectoryIndex start.cgi
>>
>> Options +ExecCGI +Indexes
>>
>> allow from all
>>
>>

>>
>>
>>
>> start.cgi calls a custom module (use CustomModule;) which exports a =
bunch of
>> subs, for example foobar and all over the place in the subs that are
>> exported from CustomModule I see code like
>>
>>
>>
>>
>>
>> sub foobar {
>>
>> my $key =3D @_;
>>
>>
>>
>> if ($cache{$key}) {
>>
>> return $cache{$key};
>>
>> } else {
>>
>> my $do_some_query =3D xxxx;
>>
>> $cache{$key} =3D $do_some_query_results
>>
>> return $cache{$key};
>>
>> }
>>
>> }
>>
>>
>>
>> My question is isn't the "else" in foobar always going to be true =
anyplace
>> where start.cgi is calling "&foobar('somekey')"??????
>>
>> I don't understand how %cache could already be populate from a =
previous
>> browser request or something? I'm I just missing something stupid?
>
> %cache is defined outside the scope of the sub so it will persist for
> the lifetime of the apache server. The very first time
> foobar('somekey') is called it will do the query lookup, the next time
> it will return from cache.
>
I believe the above is almost, but not totally true. It should probably
be "for the lifetime of this particular apache child".
Each apache child process has it's own copy of the above code, and it's
own copy of the above "global" (*) %cache hash. Thus whether the first
or second part of the if will run, depends of the previous history of
the particular apache child which handles the current request. If this
particular child has already previously accessed the same hash key, it
will server it from (it's own) cache, and otherwise it will execute the
query to create the key (in it's own cache).
Apache children are created, and die, as directed by the main Apache
process configuration, and HTTP requests are served more or less at
random, by whichever child is available when the request comes in.
Is is thus quite possible for instance that the first 10 requests for a
particular hash key would each be handled by a different apache child,
and would each result in a query; then the 11th request would be handled
by a child that has already accessed this same key before, and thus
served from cache; the 13th request would be handled by a brand-new
apache child just created, thus would re-do the query, etc..

I this would be useful, I believe that it would be possible to avoid
this, by "priming" the mod_perl module during the initial start of
Apache, before it forks into children. Then each new child (being a
copy of the main apache) would start it's life with a number of keys
already in its cache. Of course this would work only if the contents of
the keys of the hash are never modified while apache is running.
And it probably involves some delicate mod_perl programming to do the
priming process.

(*) like the previous contributor, I guess that the %cache hash is
somehow global, because it is not declared within the sub that you show.
Whether it really is though, may depend on other code that we don't
see here. But anyway, "global" would mean only "global to this apache
child", not to the whole apache server.

And something that I don't know at all, is how this all works out with a
threaded apache, such as under Windows e.g.

Andr=E9




------_=_NextPart_001_01C902AE.AB8BDB4E
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Re: Caching a hash - am I missing =<br /> something?=0A=
=0A=
=0A=
=0A=

=0A=
Thanks Andre =
and Tyler.
=0A=
 
=0A=
Believe it or not I never =
even thought of that until you guys pointed it out (that the hash =
would live for the the life of the apache child). I can be such a =
fool sometimes :)
=0A=
 
=0A=
In this case I assume %cache =
is global - it only appers in that sub, there is no "my %cache =3D ()" =
any place else in the pm file and there is no "Use strict".
=0A=
 
=0A=
I guess I'll just keep it the =
same way except I'll "use strict" and I'll declare "my %cache =3D ()" =
outside that sub.
=0A=
 
=0A=
Thanks Guys!!
=0A=
-Chris
=0A=

=0A=

=0A=
From: Andr=E9 Warnier =
[mailto:aw@ice-sa.com]
Sent: Wed 8/20/2008 3:28 =
AM
To: mod_perl list
Cc: W. Tyler =
Gee
Subject: Re: Caching a hash - am I missing =
something?

=0A=
=0A=

W. Tyler Gee wrote:
> On Tue, Aug 19, 2008 at =
5:35 PM, Chris Faust <cfaust@doyougot.com> wrote:
>> =
Hi,
>>
>>
>>
>> This might be a =
little off topic, I hope it's OK to post. I'm not positive
>> =
if mod_perl matters or not because it's a little confusing to =
me.
>>
>>
>>
>> I've taken over some =
pretty old code that I'm updating and making mp2
>> content =
handlers out of. The main script is a standard cgi script
>> =
"start.cgi" there is nothing special in the apache conf for =
it.
>>
>>
>>
>>   &nbs=
p;     <Directory =
/xxx/>
>>
>>      &nbs=
p;  SetHandler =
perl-script
>>
>>      &n=
bsp;  PerlFixupHandler =
My::Fixup
>>
>>      &nbs=
p;  PerlResponseHandler =
ModPerl::PerlRun
>>
>>     &nb=
sp;   PerlOptions =
+ParseHeaders
>>
>>      =
   DirectoryIndex =
start.cgi
>>
>>      &nbs=
p;  Options +ExecCGI =
+Indexes
>>
>>       =
;  allow from =
all
>>
>>       &nbs=
p; </Directory>
>>
>>
>>
>> =
start.cgi calls a custom module (use CustomModule;) which exports a =
bunch of
>> subs, for example foobar and all over the place in =
the subs that are
>> exported from CustomModule I see code =
like
>>
>>
>>
>>
>>
>&=
gt; sub foobar =
{
>>
>>        =
     my $key =3D =
@_;
>>
>>
>>
>>   &nbs=
p;         if ($cache{$key}) =
{
>>
>>        =
            &=
nbsp;    return =
$cache{$key};
>>
>>      =
       } else =
{
>>
>>        =
            &=
nbsp;    my $do_some_query =3D =
xxxx;
>>
>>       &n=
bsp;           &nb=
sp;     $cache{$key} =3D =
$do_some_query_results
>>
>>    &nb=
sp;           &nbs=
p;        return =
$cache{$key};
>>
>>      =
       }
>>
>> =
}
>>
>>
>>
>> My question is isn't =
the "else" in foobar always going to be true anyplace
>> where =
start.cgi is calling =
"&foobar('somekey')"??????
>>
>> I don't =
understand how %cache could already be populate from a =
previous
>> browser request or something? I'm I just missing =
something stupid?
>
> %cache is defined outside the scope of =
the sub so it will persist for
> the lifetime of the apache =
server.  The very first time
> foobar('somekey') is called it =
will do the query lookup, the next time
> it will return from =
cache.
>
I believe the above is almost, but not totally =
true.  It should probably
be "for the lifetime of this =
particular apache child".
Each apache child process has it's own copy =
of the above code, and it's
own copy of the above "global" (*) %cache =
hash.  Thus whether the first
or second part of the if will run, =
depends of the previous history of
the particular apache child which =
handles the current request.  If this
particular child has =
already previously accessed the same hash key, it
will server it from =
(it's own) cache, and otherwise it will execute the
query to create =
the key (in it's own cache).
Apache children are created, and die, as =
directed by the main Apache
process configuration, and HTTP requests =
are served more or less at
random, by whichever child is available =
when the request comes in.
Is is thus quite possible for instance =
that the first 10 requests for a
particular hash key would each be =
handled by a different apache child,
and would each result in a =
query; then the 11th request would be handled
by a child that has =
already accessed this same key before, and thus
served from cache; =
the 13th request would be handled by a brand-new
apache child just =
created, thus would re-do the query, etc..

I this would be =
useful, I believe that it would be possible to avoid
this, by =
"priming" the mod_perl module during the initial start of
Apache, =
before it forks into children.  Then each new child (being =
a
copy of the main apache) would start it's life with a number of =
keys
already in its cache.  Of course this would work only if =
the contents of
the keys of the hash are never modified while apache =
is running.
And it probably involves some delicate mod_perl =
programming to do the
priming process.

(*) like the previous =
contributor, I guess that the %cache hash is
somehow global, because =
it is not declared within the sub that you show.
  Whether it =
really is though, may depend on other code that we don't
see =
here.  But anyway, "global" would mean only "global to this =
apache
child", not to the whole apache server.

And something =
that I don't know at all, is how this all works out with a
threaded =
apache, such as under Windows =
e.g.

Andr=E9


------_=_NextPart_001_01C902AE.AB8BDB4E--