Apache::Reload - ModPerl::Util::unload_package under perl 5.10 usebase directive malfunction

1. Problem Description:

While developing with CGI::Application and utilizing Apache::Reload, we
encountered an issue where our modules were not being succesfully
reinitialized on reload. It was traced down to @ISA not containing the
proper values after a 'use base' directive, but only on automatic
reload, and freshly on perl 5.10, not our previous version of perl 5.8.8!

I determined that when ModPerl::Util::unload_package is used to unload a
package, when it is then re-required in perl 5.10, 'use base' directives
don't function as expected.

I recreated the problem with this simple use case. It happens when a
sub-module (required by something else) is unloaded then re-required. I
included the workaround using the push @ISA method to contrast working
with not working.

This is the output when the program below is run.

$ perl t.pl
X
X
V
Not a CODE reference at t.pl line 14.

Where did the inheritance go after the require of v? Is this a bug in
the do-file functionality utilized by require? Did
ModPerl::Util::unload_package not do something it was supposed to? Is
there a subtle interaction between 5.10 and ModPerl::Util?

I was initially going to submit this to the perl bug database, but
realized that the unload_package function was a ModPerl functionality,
so this is my first place to go. Five files to test with itemized
below. As you can see, I can recreate it without utilizing the mod_perl
environment specifically.

--- begin t.pl ---
#!/usr/bin/perl
use ModPerl::Util;
# Using push @ISA
require w;
w->x();
ModPerl::Util::unload_package('w');
require w;
w->x();
# Using use base
require u;
u->v();
ModPerl::Util::unload_package('v');
require u;
u->v();
-- end t.pl --
--- begin u.pm ---
package u;
use v;
use base 'v';
1;
--- end u.pm
--- begin v.pm ---
package v;
sub v { print "V\n";}
1;
--- end v.pm
--- begin w.pm ---
package w;
use x;
BEGIN { push @ISA, 'x'; }
1;
--- end w.pm
--- begin x.pm ---
package x;
sub x { print "X\n";}
1;
--- end x.pm


2. Used Components and their Configuration:

*** mod_perl version 2.000004

*** using /usr/lib/perl5/Apache2/BuildConfig.pm

*** Makefile.PL options:
MP_APR_LIB => aprext
MP_APXS => /usr/bin/apxs2
MP_CCOPTS => -g -Wall
MP_COMPAT_1X => 1
MP_GENERATE_XS => 1
MP_INCLUDE_DIR => /usr/include/apache2 /usr/include/apr-1.0
MP_LIBNAME => mod_perl
MP_TRACE => 0
MP_USE_DSO => 1
MP_USE_GTOP => 1
MP_USE_STATIC => 0


*** The httpd binary was not found


*** (apr|apu)-config linking info

-L/usr/lib -laprutil-1
-L/usr/lib -lapr-1 -luuid -lrt -lcrypt -lpthread -ldl



*** /usr/bin/perl -V
Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
Platform:
osname=linux, osvers=2.6.24-16-server,
archname=x86_64-linux-gnu-thread-multi
uname='linux yellow 2.6.24-16-server #1 smp thu apr 10 13:15:38 utc
2008 x86_64 gnulinux '
config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.10.0
-Dsitearch=/usr/local/lib/perl/5.10.0 -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio
-Uusenm -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib
-Dlibperl=libperl.so.5.10.0 -Dd_dosuid -des'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN
-fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
optimize='-O2 -g',
cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing
-pipe -I/usr/local/include'
ccversion='', gccversion='4.3.2', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib /lib64 /usr/lib64
libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
perllibs=-ldl -lm -lpthread -lc -lcrypt
libc=/lib/libc-2.8.90.so, so=so, useshrplib=true,
libperl=libperl.so.5.10.0
gnulibc_version='2.8.90'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib'


Characteristics of this binary (from libperl):
Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
USE_64_BIT_ALL
USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES
USE_PERLIO USE_REENTRANT_API
Built under linux
Compiled at Dec 23 2008 02:55:19
%ENV:
PERL_LWP_USE_HTTP_10="1"
@INC:
/etc/perl
/usr/local/lib/perl/5.10.0
/usr/local/share/perl/5.10.0
/usr/lib/perl5
/usr/share/perl5
/usr/lib/perl/5.10
/usr/share/perl/5.10
/usr/local/lib/site_perl
.

*** Packages of interest status:

Apache2 : -
Apache2::Request : -
CGI : 3.29
ExtUtils::MakeMaker: 6.42, 6.48
LWP : 5.812
mod_perl : -
mod_perl2 : 2.000004


3. This is the core dump trace: (if you get a core dump):

Not relevant

This report was generated by /usr/share/libapache2-mod-perl2/mp2bug on
Sat Dec 27 01:11:46 2008 GMT.


---
David Ihnen
Perl Programmer
Norchem Laboratories, Inc

Re: Apache::Reload - ModPerl::Util::unload_package under perl 5.10use base directive malfunction

David Ihnen wrote:
> 1. Problem Description:
>
> While developing with CGI::Application and utilizing Apache::Reload, we
> encountered an issue where our modules were not being succesfully
> reinitialized on reload. It was traced down to @ISA not containing the
> proper values after a 'use base' directive, but only on automatic
> reload, and freshly on perl 5.10, not our previous version of perl 5.8.8!
Are you able to try 5.8.9, its at least a smaller change set to look at if
5.8.9 breaks too.




--
------------------------------------------------------------ ------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci@p6m7g8.com) c: 703.336.9354
Consultant - P6M7G8 Inc. http://p6m7g8.net
Director IT - RideCharge, Inc. http://ridecharge.com
Contractor - PositiveEnergyUSA http://positiveenergyusa.com
ASF Member - Apache Software Foundation http://apache.org
FreeBSD Committer - FreeBSD Foundation http://freebsd.org

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.

Apache::Reload/ModPerl::Util/base.pm incompatibility prospectivepatch

This is a multi-part message in MIME format.
--------------090106060009050505090303
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Philip M. Gollucci wrote:
> David Ihnen wrote:
>> 1. Problem Description:
>>
>> While developing with CGI::Application and utilizing Apache::Reload,
>> we encountered an issue where our modules were not being succesfully
>> reinitialized on reload. It was traced down to @ISA not containing
>> the proper values after a 'use base' directive, but only on automatic
>> reload, and freshly on perl 5.10, not our previous version of perl
>> 5.8.8!
> Are you able to try 5.8.9, its at least a smaller change set to look
> at if 5.8.9 breaks too.
>
No, not easily when I considered the mod_perl ramifications
particularly. I did find a way around this particular problem on the
web server though.

I noticed line 79. of base.pm

79 next if grep $_->isa($base), ($inheritor, @bases);

Experimentally, if I commented it out, the web server could reload
properly! Score!

Hm. I know ->isa is a UNIVERSAL function. UNIVERSAL.pm is... not
implimented in perl. Must be in the C code.

So I found UNIVERSAL.c

whose isa function implimentation said after some error checks essentially

ST(0) = boolSV(sv_derived_from(sv, name));


which function had the documentation

=head1 SV Manipulation Functions

=for apidoc sv_derived_from

Returns a boolean indicating whether the SV is derived from the specified class
I. To check derivation at the Perl level, call C as a
normal Perl method.

=cut


Ooh. at the **C** level. I bet the unload utility doesn't even *affect*
the C level!

Those who understand perl C code might be better able to tell if
unloading in the C levels is possible, a good idea, or just bad mojo.
Having taken some medicine to deal with the C-inspired headache that
inspection brought on, I returned to base.pm.

ISA is a package local variable list that lists the parents. In order
to see all-the-parents, or replicate ->isa, I have to get the list of
all the packages in ISA, all THEIR ISAs, and so on, as long as there are
contents therein. And my scan will pay attention only to the
interpreter state, not the C state...

I should be able to do that in perl easily enough. PATCH

--- 510base.pm 2008-12-29 12:48:27.000000000 -0700
+++ /usr/share/perl/5.10.0/base.pm 2008-12-29 12:48:48.000000000 -0700
@@ -76,7 +76,20 @@
warn "Class '$inheritor' tried to inherit from itself\n";
}

- next if grep $_->isa($base), ($inheritor, @bases);
+ my @classlist = ($inheritor, @bases);
+ foreach ($inheritor, @bases) {
+ my @parents = @$_::ISA;
+ do {
+ my $class = shift @parents;
+ foreach my $parent (@$class::ISA) {
+ unless (grep { $_ eq $parent } @classlist) {
+ push @classlist, $parent;
+ push @parents, @$parent::ISA;
+ }
+ }
+ } while (scalar @parents);
+ }
+ next if grep {$_ eq $base} @classlist;

if (has_version($base)) {
${$base.'::VERSION'} = '-1, set by base.pm'

I know thats kind of ugly, and verbose, and maybe not very performant.
But it works for the web server. Its not in any of the ISAs at
re-require time so it does not skip loading the module even if the C HAS
seen it before, and the application continues happily.

So at any rate, its a patch that works for my case (much as I hate
patching base.pm) - and to my somewhat weak understanding of the C code
implimentation, does the same thing in perl that the isa-iteration did
in c. What is the proper path here?

David Ihnen
Perl Programmer
Norchem Laboratories, Inc


--------------090106060009050505090303
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit







Philip M. Gollucci wrote:

David
Ihnen wrote:

1. Problem Description:




While developing with CGI::Application and utilizing Apache::Reload, we
encountered an issue where our modules were not being succesfully
reinitialized on reload.  It was traced down to @ISA not containing the
proper values after a 'use base' directive, but only on automatic
reload, and freshly on perl 5.10, not our previous version of perl
5.8.8!

Are you able to try 5.8.9, its at least a smaller change set to look at
if 5.8.9 breaks too.

ST(0) = boolSV(sv_derived_from(sv, name));

=head1 SV Manipulation Functions



=for apidoc sv_derived_from



Returns a boolean indicating whether the SV is derived from the specified class

I<at the C level>.  To check derivation at the Perl level, call C<isa()> as a

normal Perl method.



=cut

Re: Apache::Reload/ModPerl::Util/base.pm incompatibility prospectivepatch

Have you tried the alternate unload_package implementation, the one in C/XS?

You can try it out by doing

BEGIN {
$ModPerl::Util::DEFAULT_UNLOAD_METHOD = 'unload_package_xs';
}
use ModPerl::Util;

Somewhere early in your server startup scripts.

On 29/12/08 15:08, David Ihnen wrote:
> Philip M. Gollucci wrote:
>> David Ihnen wrote:
>>> 1. Problem Description:
>>>
>>> While developing with CGI::Application and utilizing Apache::Reload,
>>> we encountered an issue where our modules were not being succesfully
>>> reinitialized on reload. It was traced down to @ISA not containing
>>> the proper values after a 'use base' directive, but only on automatic
>>> reload, and freshly on perl 5.10, not our previous version of perl
>>> 5.8.8!
>> Are you able to try 5.8.9, its at least a smaller change set to look
>> at if 5.8.9 breaks too.
>>
> No, not easily when I considered the mod_perl ramifications
> particularly. I did find a way around this particular problem on the web
> server though.
>
> I noticed line 79. of base.pm
>
> 79 next if grep $_->isa($base), ($inheritor, @bases);
>
> Experimentally, if I commented it out, the web server could reload
> properly! Score!
>
> Hm. I know ->isa is a UNIVERSAL function. UNIVERSAL.pm is... not
> implimented in perl. Must be in the C code.
>
> So I found UNIVERSAL.c
>
> whose isa function implimentation said after some error checks essentially
>
> ST(0) = boolSV(sv_derived_from(sv, name));
>
>
> which function had the documentation
>
> =head1 SV Manipulation Functions
>
> =for apidoc sv_derived_from
>
> Returns a boolean indicating whether the SV is derived from the specified class
> I. To check derivation at the Perl level, call C as a
> normal Perl method.
>
> =cut
>
>
> Ooh. at the **C** level. I bet the unload utility doesn't even *affect*
> the C level!
>
> Those who understand perl C code might be better able to tell if
> unloading in the C levels is possible, a good idea, or just bad mojo.
> Having taken some medicine to deal with the C-inspired headache that
> inspection brought on, I returned to base.pm.
>
> ISA is a package local variable list that lists the parents. In order to
> see all-the-parents, or replicate ->isa, I have to get the list of all
> the packages in ISA, all THEIR ISAs, and so on, as long as there are
> contents therein. And my scan will pay attention only to the interpreter
> state, not the C state...
>
> I should be able to do that in perl easily enough. PATCH
>
> --- 510base.pm 2008-12-29 12:48:27.000000000 -0700
> +++ /usr/share/perl/5.10.0/base.pm 2008-12-29 12:48:48.000000000 -0700
> @@ -76,7 +76,20 @@
> warn "Class '$inheritor' tried to inherit from itself\n";
> }
>
> - next if grep $_->isa($base), ($inheritor, @bases);
> + my @classlist = ($inheritor, @bases);
> + foreach ($inheritor, @bases) {
> + my @parents = @$_::ISA;
> + do {
> + my $class = shift @parents;
> + foreach my $parent (@$class::ISA) {
> + unless (grep { $_ eq $parent } @classlist) {
> + push @classlist, $parent;
> + push @parents, @$parent::ISA;
> + }
> + }
> + } while (scalar @parents);
> + }
> + next if grep {$_ eq $base} @classlist;
>
> if (has_version($base)) {
> ${$base.'::VERSION'} = '-1, set by base.pm'
>
> I know thats kind of ugly, and verbose, and maybe not very performant.
> But it works for the web server. Its not in any of the ISAs at
> re-require time so it does not skip loading the module even if the C HAS
> seen it before, and the application continues happily.
>
> So at any rate, its a patch that works for my case (much as I hate
> patching base.pm) - and to my somewhat weak understanding of the C code
> implimentation, does the same thing in perl that the isa-iteration did
> in c. What is the proper path here?
>
> David Ihnen
> Perl Programmer
> Norchem Laboratories, Inc
>

--
Philippe M. Chiasson GPG: F9BFE0C2480E7680 1AE53631CB32A107 88C3A5A5
http://gozer.ectoplasm.org/ m/gozer\@(apache|cpan|ectoplasm)\.org/

Re: Apache::Reload - ModPerl::Util::unload_package under perl 5.10use base directive malfunction

On 26/12/08 20:39, David Ihnen wrote:
> 1. Problem Description:
>
> While developing with CGI::Application and utilizing Apache::Reload, we
> encountered an issue where our modules were not being succesfully
> reinitialized on reload. It was traced down to @ISA not containing the
> proper values after a 'use base' directive, but only on automatic
> reload, and freshly on perl 5.10, not our previous version of perl 5.8.8!
>
> I determined that when ModPerl::Util::unload_package is used to unload a
> package, when it is then re-required in perl 5.10, 'use base' directives
> don't function as expected.
>
> I recreated the problem with this simple use case. It happens when a
> sub-module (required by something else) is unloaded then re-required. I
> included the workaround using the push @ISA method to contrast working
> with not working.
>
> This is the output when the program below is run.
>
> $ perl t.pl
> X
> X
> V
> Not a CODE reference at t.pl line 14.
>
> Where did the inheritance go after the require of v? Is this a bug in
> the do-file functionality utilized by require? Did
> ModPerl::Util::unload_package not do something it was supposed to? Is
> there a subtle interaction between 5.10 and ModPerl::Util?
>
> I was initially going to submit this to the perl bug database, but
> realized that the unload_package function was a ModPerl functionality,
> so this is my first place to go. Five files to test with itemized
> below. As you can see, I can recreate it without utilizing the mod_perl
> environment specifically.
>
> --- begin t.pl ---
> #!/usr/bin/perl
> use ModPerl::Util;
> # Using push @ISA
> require w;
> w->x();
> ModPerl::Util::unload_package('w');
> require w;
> w->x();
> # Using use base
> require u;
> u->v();
> ModPerl::Util::unload_package('v');
> require u;

This is strange, as you are unloading 'v' and reloading 'u', probably not
what was intended.

--
Philippe M. Chiasson GPG: F9BFE0C2480E7680 1AE53631CB32A107 88C3A5A5
http://gozer.ectoplasm.org/ m/gozer\@(apache|cpan|ectoplasm)\.org/