Apache2::SubProcess subprocess not subprocessing properly

Hi after much trial and all error I am seeing that the browser
connection closing is also stopping my subprocess.

The main ModPerl::Registry program looks like this:
### file.pl ###
use Apache2::SubProcess ();
use JSON();

&main(shift);

sub main {
my $r = shift;
...
$r->spawn_proc_prog ('/web/html/file_fork.pl', \@argv);
return print $cgi->redirect('index.pl');
}
###

And

### file_fork.pl ###
use JSON();
use warnings;
use POSIX 'setsid';
chdir '/' or die "Can't chdir to /: $!";
open STDIN, '/dev/null' or die "Can't read /dev/null: $!";
open STDOUT, '+>>', '/tmp/debug.txt' or die "Can't write to
/tmp/debug.txt: $!";
open STDERR, '>&STDOUT' or die "Can't dup stdout: $!";
setsid or die "Can't start a new session: $!";

# run your code here or call exec to another program
foreach my $num (1..10) {
warn $num;
sleep(1);
}
###

When file.pl is executed it calls file_fork.pl and file_fork starts to
print 1, 2, 3 and then stops around 2 or 3 depending on how long my
browser is connected.

Are there any glaring errors that anyone sees? It should work shouldn't
it, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 after 10 seconds...

Thank-you all!

Tosh


--
McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/

Re: Apache2::SubProcess subprocess not subprocessing properly

On Tuesday 09 February 2010 14:21:57 Tosh Cooey wrote:
> use POSIX 'setsid';
if( fork ) { POSIX::_exit 0; CORE::exit 0 }
> chdir '/' or die "Can't chdir to /: $!";
>
simplest is to fork() another time to break the parent-child relationship.

Keep in mind that you make your service vulnerable if you simply fork off long
running processes.

What prevents a user from testing it:

ab -n 100000 -c 100 http://...

This will quite fast create a *lot* of processes.

I tend to use some kind of queue for such processing. But, of course, it
depends.

Torsten

Re: Apache2::SubProcess subprocess not subprocessing properly

Actually my solution to this problem was pretty rational. I just turned
mod_perl off for this particular program.


PerlHandler ModPerl::Registry


Will I a have possible performance problems? Sure, but nothing a few
more $25/month servers can't fix, and it saves me all the aggravation of
the past months trying to get this to work.

Problem solved!

Tosh


Torsten Förtsch wrote:
> On Tuesday 09 February 2010 14:21:57 Tosh Cooey wrote:
>> use POSIX 'setsid';
> if( fork ) { POSIX::_exit 0; CORE::exit 0 }
>> chdir '/' or die "Can't chdir to /: $!";
>>
> simplest is to fork() another time to break the parent-child relationship.
>
> Keep in mind that you make your service vulnerable if you simply fork off long
> running processes.
>
> What prevents a user from testing it:
>
> ab -n 100000 -c 100 http://...
>
> This will quite fast create a *lot* of processes.
>
> I tend to use some kind of queue for such processing. But, of course, it
> depends.
>
> Torsten
>

--
McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/

RE: Apache2::SubProcess subprocess not subprocessing properly

We are configured to have mod_perl serve the .pl scripts and leave the .c=
gi scripts to be served as regular old CGI, just as a point of interest.

Eric

> -----Original Message-----
> From: Tosh Cooey [mailto:tosh@1200group.com]=20
> Sent: Thursday, February 11, 2010 6:08 AM
> To: Torsten Förtsch
> Cc: modperl@perl.apache.org
> Subject: Re: Apache2::SubProcess subprocess not subprocessing properly
>=20
> Actually my solution to this problem was pretty rational. I=20
> just turned=20
> mod_perl off for this particular program.
>=20
>
> PerlHandler ModPerl::Registry
>
>=20
> Will I a have possible performance problems? Sure, but nothing a few=20
> more $25/month servers can't fix, and it saves me all the=20
> aggravation of=20
> the past months trying to get this to work.
>=20
> Problem solved!
>=20
> Tosh
>=20
>=20
> Torsten Förtsch wrote:
> > On Tuesday 09 February 2010 14:21:57 Tosh Cooey wrote:
> >> use POSIX 'setsid';
> > if( fork ) { POSIX::_exit 0; CORE::exit 0 }
> >> chdir '/' or die "Can't chdir to /: $!";
> >>
> > simplest is to fork() another time to break the=20
> parent-child relationship.
> >=20
> > Keep in mind that you make your service vulnerable if you=20
> simply fork off long=20
> > running processes.
> >=20
> > What prevents a user from testing it:
> >=20
> > ab -n 100000 -c 100 http://...
> >=20
> > This will quite fast create a *lot* of processes.
> >=20
> > I tend to use some kind of queue for such processing. But,=20
> of course, it=20
> > depends.
> >=20
> > Torsten
> >=20
>=20
> --=20
> McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/
>=20
_______________________________________________

This e-mail may contain information that is confidential, privileged or o=
therwise protected from disclosure. If you are not an intended recipient =
of this e-mail, do not duplicate or redistribute it by any means. Please =
delete it and any attachments and notify the sender that you have receive=
d it in error. Unless specifically indicated, this e-mail is not an offer=
=20to buy or sell or a solicitation to buy or sell any securities, invest=
ment products or other financial product or service, an official confirma=
tion of any transaction, or an official statement of Barclays. Any views =
or opinions presented are solely those of the author and do not necessari=
ly represent those of Barclays. This e-mail is subject to terms available=
=20at the following link: www.barcap.com/emaildisclaimer. By messaging wi=
th Barclays you consent to the foregoing. Barclays Capital is the invest=
ment banking division of Barclays Bank PLC, a company registered in Engla=
nd (number 1026167) with its registered office at 1 Churchill Place, Lond=
on, E14 5HP. This email may relate to or be sent from other members of t=
he Barclays Group.
_______________________________________________

Apache2::SubProcess sucks

Seriously, you know what, the code below doesn't even work.

And if I get rid of the redirect and replace it with some nice pretty text:

print "Content-type: text/html\n\nTesting";

You know what? The warning for 1 (from the 'for' loop) is printed and
then it all stops, probably as the browser connection "closes".

Hiding it behind more forks made it work once but then never again.

So it is my belief that Apache2::SubProcess sucks dirty things, and that
belief won't change until I see proof to the contrary.

On a related note, anyone have tips for avoiding zombies in CGI forking?

Tosh



Tosh Cooey wrote:
> Hi after much trial and all error I am seeing that the browser
> connection closing is also stopping my subprocess.
>
> The main ModPerl::Registry program looks like this:
> ### file.pl ###
> use Apache2::SubProcess ();
> use JSON();
>
> &main(shift);
>
> sub main {
> my $r = shift;
> ...
> $r->spawn_proc_prog ('/web/html/file_fork.pl', \@argv);
> return print $cgi->redirect('index.pl');
> }
> ###
>
> And
>
> ### file_fork.pl ###
> use JSON();
> use warnings;
> use POSIX 'setsid';
> chdir '/' or die "Can't chdir to /: $!";
> open STDIN, '/dev/null' or die "Can't read /dev/null: $!";
> open STDOUT, '+>>', '/tmp/debug.txt' or die "Can't write to
> /tmp/debug.txt: $!";
> open STDERR, '>&STDOUT' or die "Can't dup stdout: $!";
> setsid or die "Can't start a new session: $!";
>
> # run your code here or call exec to another program
> foreach my $num (1..10) {
> warn $num;
> sleep(1);
> }
> ###
>
> When file.pl is executed it calls file_fork.pl and file_fork starts to
> print 1, 2, 3 and then stops around 2 or 3 depending on how long my
> browser is connected.
>
> Are there any glaring errors that anyone sees? It should work shouldn't
> it, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 after 10 seconds...
>
> Thank-you all!
>
> Tosh
>
>

--
McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/

Re: Apache2::SubProcess sucks

I haven't been following this thread too closely, but is there a
reason you aren't using PerlHandlers instead of ModPerl::Registry?
MP::R is meant mostly for CGI migration, so it is probably not widely
tested with Apache2::SubProcess calls. I'd suggest trying your
external call under a PerlHandler instead and see if that works.

On Fri, Feb 19, 2010 at 1:52 PM, Tosh Cooey wrote:
> Seriously, you know what, the code below doesn't even work.
>
> And if I get rid of the redirect and replace it with some nice pretty tex=
t:
>
> print "Content-type: text/html\n\nTesting";
>
> You know what? =A0The warning for 1 (from the 'for' loop) is printed and =
then
> it all stops, probably as the browser connection "closes".
>
> Hiding it behind more forks made it work once but then never again.
>
> So it is my belief that Apache2::SubProcess sucks dirty things, and that
> belief won't change until I see proof to the contrary.
>
> On a related note, anyone have tips for avoiding zombies in CGI forking?
>
> Tosh
>
>
>
> Tosh Cooey wrote:
>>
>> Hi after much trial and all error I am seeing that the browser connectio=
n
>> closing is also stopping my subprocess.
>>
>> The main ModPerl::Registry program looks like this:
>> ### file.pl ###
>> use Apache2::SubProcess ();
>> use JSON();
>>
>> &main(shift);
>>
>> sub main {
>> =A0my $r =3D shift;
>> =A0...
>> =A0$r->spawn_proc_prog ('/web/html/file_fork.pl', \@argv);
>> =A0return print $cgi->redirect('index.pl');
>> }
>> ###
>>
>> And
>>
>> ### file_fork.pl ###
>> use JSON();
>> use warnings;
>> use POSIX 'setsid';
>> chdir '/' =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0or die "Can't chdir to /: $!";
>> open STDIN, '/dev/null' =A0or die "Can't read /dev/null: $!";
>> open STDOUT, '+>>', '/tmp/debug.txt' or die "Can't write to
>> /tmp/debug.txt: $!";
>> open STDERR, '>&STDOUT' =A0or die "Can't dup stdout: $!";
>> setsid or die "Can't start a new session: $!";
>>
>> =A0# run your code here or call exec to another program
>> foreach my $num (1..10) {
>> =A0warn $num;
>> =A0sleep(1);
>> }
>> ###
>>
>> When file.pl is executed it calls file_fork.pl and file_fork starts to
>> print 1, 2, 3 and then stops around 2 or 3 depending on how long my brow=
ser
>> is connected.
>>
>> Are there any glaring errors that anyone sees? =A0It should work shouldn=
't
>> it, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 after 10 seconds...
>>
>> Thank-you all!
>>
>> Tosh
>>
>>
>
> --
> McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/
>

Re: Apache2::SubProcess subprocess not subprocessing properly

Tosh,

On Tuesday 09 February 2010 14:21:57 Tosh Cooey wrote:
> Hi after much trial and all error I am seeing that the browser
> connection closing is also stopping my subprocess.
>

I don't know what you are trying to achieve and I don't use
Apache2::Subprocess much. What I do to fork off processes instead is this:

my $close_fd=sub {
use POSIX ();
my %save=(2=>1); # keep STDERR
undef @save{@_} if( @_ );
opendir my $d, "/proc/self/fd" or do {
warn "Cannot open directory /proc/self/fd: $!\n";
POSIX::_exit -1; # try this first to avoid buffer flushing
CORE::exit -1;
};

while (defined(my $fd=readdir $d)) {
next unless $fd=~/^\d+$/;
POSIX::close $fd unless exists $save{$fd};
}
};

my $spawn=sub {
use POSIX ();
my ($daemon_should_survive_apache_restart, @args)=@_;

local $SIG{CHLD}='IGNORE';
my $pid;
# yes, even fork can fail
select undef, undef, undef, .1 while( !defined($pid=fork) );
unless( $pid ) { # child
# 2nd fork to cut parent relationship with a mod_perl apache
select undef, undef, undef, .1 while( !defined($pid=fork) );
if( $pid ) {
POSIX::_exit 0;
CORE::exit 0;
} else {
if( ref($daemon_should_survive_apache_restart) ) {
$close_fd->($daemon_should_survive_apache_restart->{fd});
POSIX::setsid if( $daemon_should_survive_apache_restart->{survive} );
} else {
$close_fd->();
POSIX::setsid if( $daemon_should_survive_apache_restart );
}

if( 'CODE' eq ref $args[0] ) {
my $f=shift @args;
# TODO: restore %ENV and exit() behavior
eval {$f->(@args)};
CORE::exit 0;
} else {
{exec @args;} # extra block to suppress a warning
POSIX::_exit -1;
CORE::exit -1;
}
}
}
waitpid $pid, 0; # avoid a zombie on some OS
};

And this is how to use it:

$spawn->({survive=>1}, sub {
my ($n, $sleep)=@_;
for(1..$n) {
warn "sleeping $sleep\n";
sleep $sleep;
}
warn "Done\n";
}, 10, 2 );

or

$spawn->
(undef, # don't survive apache stop
qw/bash -c/,'for (( i=0; i<10; i++ )); do echo "opi" >&2; sleep 2; done')

The first argument to $spawn is an options hash with 2 possible options:

survive=>BOOLEAN # should the subprocess survive an apache stop or not
fd=>\@list_of_open_file_descriptors_to_preserve

Normally the $spawn will close all files save for STDERR. With the fd option
one can preserve other files such as database connections or other.

With the survive option true the child process will be the leader of a new
process group.

NOTES:

- The $close_fd function is Linux-specific. I couldn't think of a really
portable (at least across UNICES) way to get all open file descriptors of the
current process.

- The waitpid at the end of $spawn is not necessary under Linux since
$SIG{CHLD}='IGNORE' already takes care of zombies. But other systems need it.

- Keep in mind that %ENV and exit() work different under mod_perl. If you pass
a subroutine this behavior remains.

Torsten

Re: Apache2::SubProcess subprocess not subprocessing properly

You aren't going to like this, and maybe what you're doing is for a
really complex situation, but it really shouldn't be so complex to just
spawn off a long running process that you don't care to ever hear back
from, honestly.

Tosh


Torsten Förtsch wrote:
> Tosh,
>
> On Tuesday 09 February 2010 14:21:57 Tosh Cooey wrote:
>> Hi after much trial and all error I am seeing that the browser
>> connection closing is also stopping my subprocess.
>>
>
> I don't know what you are trying to achieve and I don't use
> Apache2::Subprocess much. What I do to fork off processes instead is this:
>
> my $close_fd=sub {
> use POSIX ();
> my %save=(2=>1); # keep STDERR
> undef @save{@_} if( @_ );
> opendir my $d, "/proc/self/fd" or do {
> warn "Cannot open directory /proc/self/fd: $!\n";
> POSIX::_exit -1; # try this first to avoid buffer flushing
> CORE::exit -1;
> };
>
> while (defined(my $fd=readdir $d)) {
> next unless $fd=~/^\d+$/;
> POSIX::close $fd unless exists $save{$fd};
> }
> };
>
> my $spawn=sub {
> use POSIX ();
> my ($daemon_should_survive_apache_restart, @args)=@_;
>
> local $SIG{CHLD}='IGNORE';
> my $pid;
> # yes, even fork can fail
> select undef, undef, undef, .1 while( !defined($pid=fork) );
> unless( $pid ) { # child
> # 2nd fork to cut parent relationship with a mod_perl apache
> select undef, undef, undef, .1 while( !defined($pid=fork) );
> if( $pid ) {
> POSIX::_exit 0;
> CORE::exit 0;
> } else {
> if( ref($daemon_should_survive_apache_restart) ) {
> $close_fd->($daemon_should_survive_apache_restart->{fd});
> POSIX::setsid if( $daemon_should_survive_apache_restart->{survive} );
> } else {
> $close_fd->();
> POSIX::setsid if( $daemon_should_survive_apache_restart );
> }
>
> if( 'CODE' eq ref $args[0] ) {
> my $f=shift @args;
> # TODO: restore %ENV and exit() behavior
> eval {$f->(@args)};
> CORE::exit 0;
> } else {
> {exec @args;} # extra block to suppress a warning
> POSIX::_exit -1;
> CORE::exit -1;
> }
> }
> }
> waitpid $pid, 0; # avoid a zombie on some OS
> };
>
> And this is how to use it:
>
> $spawn->({survive=>1}, sub {
> my ($n, $sleep)=@_;
> for(1..$n) {
> warn "sleeping $sleep\n";
> sleep $sleep;
> }
> warn "Done\n";
> }, 10, 2 );
>
> or
>
> $spawn->
> (undef, # don't survive apache stop
> qw/bash -c/,'for (( i=0; i<10; i++ )); do echo "opi" >&2; sleep 2; done')
>
> The first argument to $spawn is an options hash with 2 possible options:
>
> survive=>BOOLEAN # should the subprocess survive an apache stop or not
> fd=>\@list_of_open_file_descriptors_to_preserve
>
> Normally the $spawn will close all files save for STDERR. With the fd option
> one can preserve other files such as database connections or other.
>
> With the survive option true the child process will be the leader of a new
> process group.
>
> NOTES:
>
> - The $close_fd function is Linux-specific. I couldn't think of a really
> portable (at least across UNICES) way to get all open file descriptors of the
> current process.
>
> - The waitpid at the end of $spawn is not necessary under Linux since
> $SIG{CHLD}='IGNORE' already takes care of zombies. But other systems need it.
>
> - Keep in mind that %ENV and exit() work different under mod_perl. If you pass
> a subroutine this behavior remains.
>
> Torsten
>

--
McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/

Re: Apache2::SubProcess sucks

Maybe it does or doesn't work under MP::R, the answer to that is perhaps
individual.

Anyway, the solution, at least so far until I run into other problems,
seems to be to just make a system() call and the called program uses
Proc::Daemon and things *seem* to work fine in testing, we'll see when
it hits production...

Tosh



Fred Moyer wrote:
> I haven't been following this thread too closely, but is there a
> reason you aren't using PerlHandlers instead of ModPerl::Registry?
> MP::R is meant mostly for CGI migration, so it is probably not widely
> tested with Apache2::SubProcess calls. I'd suggest trying your
> external call under a PerlHandler instead and see if that works.
>
> On Fri, Feb 19, 2010 at 1:52 PM, Tosh Cooey wrote:
>> Seriously, you know what, the code below doesn't even work.
>>
>> And if I get rid of the redirect and replace it with some nice pretty text:
>>
>> print "Content-type: text/html\n\nTesting";
>>
>> You know what? The warning for 1 (from the 'for' loop) is printed and then
>> it all stops, probably as the browser connection "closes".
>>
>> Hiding it behind more forks made it work once but then never again.
>>
>> So it is my belief that Apache2::SubProcess sucks dirty things, and that
>> belief won't change until I see proof to the contrary.
>>
>> On a related note, anyone have tips for avoiding zombies in CGI forking?
>>
>> Tosh
>>
>>
>>
>> Tosh Cooey wrote:
>>> Hi after much trial and all error I am seeing that the browser connection
>>> closing is also stopping my subprocess.
>>>
>>> The main ModPerl::Registry program looks like this:
>>> ### file.pl ###
>>> use Apache2::SubProcess ();
>>> use JSON();
>>>
>>> &main(shift);
>>>
>>> sub main {
>>> my $r = shift;
>>> ...
>>> $r->spawn_proc_prog ('/web/html/file_fork.pl', \@argv);
>>> return print $cgi->redirect('index.pl');
>>> }
>>> ###
>>>
>>> And
>>>
>>> ### file_fork.pl ###
>>> use JSON();
>>> use warnings;
>>> use POSIX 'setsid';
>>> chdir '/' or die "Can't chdir to /: $!";
>>> open STDIN, '/dev/null' or die "Can't read /dev/null: $!";
>>> open STDOUT, '+>>', '/tmp/debug.txt' or die "Can't write to
>>> /tmp/debug.txt: $!";
>>> open STDERR, '>&STDOUT' or die "Can't dup stdout: $!";
>>> setsid or die "Can't start a new session: $!";
>>>
>>> # run your code here or call exec to another program
>>> foreach my $num (1..10) {
>>> warn $num;
>>> sleep(1);
>>> }
>>> ###
>>>
>>> When file.pl is executed it calls file_fork.pl and file_fork starts to
>>> print 1, 2, 3 and then stops around 2 or 3 depending on how long my browser
>>> is connected.
>>>
>>> Are there any glaring errors that anyone sees? It should work shouldn't
>>> it, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 after 10 seconds...
>>>
>>> Thank-you all!
>>>
>>> Tosh
>>>
>>>
>> --
>> McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/
>>
>

--
McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/

Re: Apache2::SubProcess sucks

On Feb 20, 2010, at 7:01 AM, Tosh Cooey wrote:

> Anyway, the solution, at least so far until I run into other
> problems, seems to be to just make a system() call and the called
> program uses Proc::Daemon and things *seem* to work fine in
> testing, we'll see when it hits production...
>
> Tosh
>
Doesn't a process that calls 'system' wait for the child
process to complete? If the parent process is an Apache
child, is that in keeping with what you're trying to do?

cmac

Re: Apache2::SubProcess sucks

It does, but Proc::Daemon closes a billion file handles and sets new
process IDs and forks and forks and maybe forks a couple more times for
good measure... Fingers crossed!

I do enjoy the fact that nobody really seems to have a simple definitive
vanilla fork/spawn process down pat, it seems everyone does what I do,
trying this and that stumbling about until they come up with some
monstrosity like Torsen has that works under the sheer weight of tricks.

Tosh


mackenna@animalhead.com wrote:
> On Feb 20, 2010, at 7:01 AM, Tosh Cooey wrote:
>
>> Anyway, the solution, at least so far until I run into other problems,
>> seems to be to just make a system() call and the called program uses
>> Proc::Daemon and things *seem* to work fine in testing, we'll see when
>> it hits production...
>>
>> Tosh
>>
> Doesn't a process that calls 'system' wait for the child
> process to complete? If the parent process is an Apache
> child, is that in keeping with what you're trying to do?
>
> cmac
>
>

--
McIntosh Cooey - Twelve Hundred Group LLC - http://www.1200group.com/

Re: Apache2::SubProcess sucks

On Saturday 20 February 2010 19:25:39 Tosh Cooey wrote:
> I do enjoy the fact that nobody really seems to have a simple definitive
> vanilla fork/spawn process down pat, it seems everyone does what I do,
> trying this and that stumbling about until they come up with some
> monstrosity like Torsen has that works under the sheer weight of tricks.
>
Well, if I'd said: "Tosh take ModPerl::Something from CPAN and call
ModPerl::Something::spawn" you'd liked it (no matter what it does internally),
right?

That sheer weight of tricks comes down to 3 points:

- you have to close the client socket in the child process or the browser will
wait for more data

- if your process should survive an apache restart it must create its own
process group, hence the setsid().

- accidental leaking file descriptors to child processes is generally
considered a security hole

This is what my function does, it closes all file descriptors except for
STDERR and calls setsid, nothing special. Perhaps it would be good to reopen
STDIN and STDOUT as /dev/null because many programs don't like closed
STDIN/STDOUT but again, no tricks.

A tricky part would be to really restore the normal Perl environment if you
don't want to exec() another program but simply to call a Perl function.
Reverting the effect of *CORE::GLOBAL::exit=\&ModPerl::Util::exit in an
already compiled function, that is tricky. Or reestablish the connection
between %ENV and C-level char *environ[].

Torsten

Re: Apache2::SubProcess sucks

In data 20 febbraio 2010 alle ore 21:16:22, Torsten Förtsch
ha scritto:

> On Saturday 20 February 2010 19:25:39 Tosh Cooey wrote:
>> I do enjoy the fact that nobody really seems to have a simple definitive
>> vanilla fork/spawn process down pat, it seems everyone does what I do,
>> trying this and that stumbling about until they come up with some
>> monstrosity like Torsen has that works under the sheer weight of tricks.
>>
> Well, if I'd said: "Tosh take ModPerl::Something from CPAN and call
> ModPerl::Something::spawn" you'd liked it (no matter what it does
> internally), right?
>
> That sheer weight of tricks comes down to 3 points:
> [...]

That's really valuable, thanks for sharing.
However, a question:

is this something that should be included in Apache2::SubProcess
to make it better? Or is this something that could be published
another, separate, CPAN distribution?
Or maybe it's already out there?

--
Cosimo

Re: Apache2::SubProcess sucks

On Saturday 20 February 2010 21:24:31 Cosimo Streppone wrote:
> That's really valuable, thanks for sharing.
> However, a question:
>
> is this something that should be included in Apache2::SubProcess
> to make it better? Or is this something that could be published
> another, separate, CPAN distribution?
> Or maybe it's already out there?
>
Apache2::SubProcess' purpose is more for creating processes similar to CGI-
scripts. A CGI-script is killed by apache if it does not produce output for a
certain period of time. Also, the process gets killed I think at the end of
the request.

I think it could be an Apache2::SubProcess extension. That would mean adding
the stuff to xs/Apache2/SubProcess/SubProcess_pm in the source tree. But there
are a few things that are unsolved:

- is there a portable way to get all open file descriptors of the current
process? Under Linux one can readdir(/proc/self/fd). On Darwin I once simply
closed all fds from 0 to 1000. Some systems have getdtablesize(2), sometimes
it is getrlimit. Sometimes you can open /dev/something or /proc/self/something
and read the descriptor table at a certain offset. What the module should do
on Windows I simply don't know.

If you can solve this problem with apr-functions I think the it can really be
in Apache2::SubProcess.

- a way of really restoring the perl environment would be good, so that you
really can fork off subroutines.

But the real reason why it is not shaped out as a separate module yet is that
the code is really old and quite seldom needed. The main part was written
around year 2000 for mp1. At that time I didn't know about POSIX::close and
thus invented my own version like:

sub SYS_close {6;} # works on Linux and Darwin
sub sysclose {
syscall &SYS_close, shift()+0;
}

In short, the code was ugly.

And because my solution is Linux-only.

Torsten

Re: Apache2::SubProcess sucks

re: close all open file descriptors portably

well, if one is on a POSIX system, _SC_OPEN_MAX gives the max integer.
Then just close them all.
Here's my usual recipe for this:

# close all open file descriptors
my $max_fd =3D POSIX::sysconf(&POSIX::_SC_OPEN_MAX);
$max_fd =3D ((! defined $max_fd) || $max_fd < 0) ? 64 : $max_fd;
while (--$max_fd >=3D 0) {
next if ($max_fd == fileno($pid_fh));
POSIX::close($max_fd);
}

# now reopen std{in,out,err} on /dev/null
open(STDIN, "+>/dev/null");
open(STDOUT, "+>&STDIN");
open(STDERR, "+>&STDIN");

Proc::Daemon does the same thing...

HTH,
-- jeff


On 2/20/10 1:10 PM, "Torsten Förtsch" wrote:

> - is there a portable way to get all open file descriptors of the current
> process? Under Linux one can readdir(/proc/self/fd). On Darwin I once sim=
ply=20

Re: Apache2::SubProcess sucks

--0015175884d02c7cdd04800f292c
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

2010/2/20 Torsten Förtsch
[ ... ]

> - is there a portable way to get all open file descriptors of the current
> process? Under Linux one can readdir(/proc/self/fd). On Darwin I once
> simply
> closed all fds from 0 to 1000. Some systems have getdtablesize(2),
> sometimes
> it is getrlimit. Sometimes you can open /dev/something or
> /proc/self/something
> and read the descriptor table at a certain offset. What the module should
> do
> on Windows I simply don't know.
>

I think just closing stdin and stdout is likely to be enough, or maybe all
file descriptors from 0 to $^F . The OS will close any others that have th=
e
close-on-exec bit set, and the close-on-exec bit is set by default by Perl'=
s
open function. Anybody setting this to something different probably
understands the consequences.

----Scott.

--0015175884d02c7cdd04800f292c
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

2010/2/20 Torsten Förtsch < ..foertsch@gmx.net">torsten.foertsch@gmx.net>