Q on localizing *STDOUT and fork

Let me preface this question by making it clear that there's no
particular problem I'm trying to solve, but rather I'm trying to
clarify my understanding of how Perl works, at least under Unix.

In the following snippet, the key fragment is the block labeled
LOOK_HERE. There I first "save" STDOUT by duplicating the handle;
then I redirect STDOUT; then I execute a (forked) command that has
the effect of sending some output to (the now redirected) STDOUT;
and finally I "restore" STDOUT with another duplication.

my $file = 'somefile';
open my $out, '>', $file or die $!;

LOOK_HERE:
{
open my $save, '>&', STDOUT or die $!;
open STDOUT, '>&', $out or die $!;

{
open my $pipe, '|-', '/usr/bin/sort', '-n' or die $!;
print $pipe int( rand( ~0 ) ), "\n" for 1..1_000_000;
}

open STDOUT, '>&', $save or die $!;
}

print "OK\n";
close $out;
my @sorted = map { chomp; $_ } File::Slurp::read_file( $file );

This all works: the output of /usr/bin/sort does end up in 'somefile',
and "OK\n" gets printed to the terminal, confirming that STDOUT
was properly restored.

So far so good.

My question is this: is there a way to avoid the bothersome saving
and restoring of STDOUT. I naively thought that one could do so
by localizing *STDOUT. IOW, replace the LOOK_HERE block with:

{
local *STDOUT;
open STDOUT, '>&', $out or die $!;

{
open my $pipe, '|-', '/usr/bin/sort', '-n' or die $!;
print $pipe int( rand( ~0 ) ), "\n" for 1..1_000_000;
}
}

Very nice, except it doesn't work. Now the output /usr/bin/sort
(which, incidentally, in this example happens to be pretty big)
goes to the terminal. BTW, this same thing happens if instead of
redirecting STDOUT by duplicating the write-handle $out, I simply
re-open STDOUT like this:

{
local *STDOUT;
open STDOUT, '>', 'somefile' or die $!;

{
open my $pipe, '|-', '/usr/bin/sort', '-n' or die $!;
print $pipe int( rand( ~0 ) ), "\n" for 1..1_000_000;
}
}

Ditto if instead I select $out before execution of the sort command.

I suspect that the problem with these failed solutions has to do
with the implicit fork triggered by the '|-' mode in the call to
open. I.e., I'm guessing that the child process uses the default
STDOUT irrespective of the parent's maneuvers. But if this is the
case, then my confusion simply shifts to wondering how the first
approach could have worked at all!

Anyway, BTAIM, is there anyway to avoid the save/restore rigmarole?

TIA!

kj
--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.

Re: Q on localizing *STDOUT and fork

On Aug 2, 12:41 am, kj wrote:
> Let me preface this question by making it clear that there's no
> particular problem I'm trying to solve, but rather I'm trying to
> clarify my understanding of how Perl works, at least under Unix.

> My question is this: is there a way to avoid the bothersome saving
> and restoring of STDOUT. I naively thought that one could do so
> by localizing *STDOUT. IOW, replace the LOOK_HERE block with:
>
> {
> local *STDOUT;
> open STDOUT, '>&', $out or die $!;
>
> {
> open my $pipe, '|-', '/usr/bin/sort', '-n' or die $!;
> print $pipe int( rand( ~0 ) ), "\n" for 1..1_000_000;
> }
> }
>
> Very nice, except it doesn't work. Now the output /usr/bin/sort
> (which, incidentally, in this example happens to be pretty big)
> goes to the terminal. BTW, this same thing happens if instead of
> redirecting STDOUT by duplicating the write-handle $out, I simply
> re-open STDOUT like this:
>
> {
> local *STDOUT;
> open STDOUT, '>', 'somefile' or die $!;
>
> {
> open my $pipe, '|-', '/usr/bin/sort', '-n' or die $!;
> print $pipe int( rand( ~0 ) ), "\n" for 1..1_000_000;
> }
> }

I wrote a quite detailed explanation of this here...

http://groups.google.com/group/comp.lang.perl.misc/browse_fr m/thread/6d5c3d062c1e7608/fb1af6de388ee945

> Anyway, BTAIM, is there anyway to avoid the save/restore rigmarole?

I'm fairly sure I've seen modules on CPAN to wrap it up a bit but
under the hood AFAIK they'd still do the same thing.

Re: Q on localizing *STDOUT and fork

kj wrote:
>
> My question is this: is there a way to avoid the bothersome saving
> and restoring of STDOUT. I naively thought that one could do so
> by localizing *STDOUT. IOW, replace the LOOK_HERE block with:
>
> {
> local *STDOUT;
> open STDOUT, '>&', $out or die $!;
>
> {
> open my $pipe, '|-', '/usr/bin/sort', '-n' or die $!;
> print $pipe int( rand( ~0 ) ), "\n" for 1..1_000_000;
> }
> }
>
> Very nice, except it doesn't work. Now the output /usr/bin/sort
> (which, incidentally, in this example happens to be pretty big)
> goes to the terminal.

The piped "sort" doesn't inherit the Perl notion of STDOUT, it inherits the
C's notion of stdout. When Perl starts, Perl's STDOUT "points" to C's
stdout. When you localize STDOUT, you break that linkage, and so whatever
is done to Perl's STDOUT doesn't affect C's stdout.


>
> I suspect that the problem with these failed solutions has to do
> with the implicit fork triggered by the '|-' mode in the call to
> open. I.e., I'm guessing that the child process uses the default
> STDOUT irrespective of the parent's maneuvers. But if this is the
> case, then my confusion simply shifts to wondering how the first
> approach could have worked at all!

When you reopen STDOUT without localizing it, it does this by reopening
C's notion of stdout and leaving Perl's STDOUT pointing to C's stdout.
Forked commands inherit this C notion, complete with the change made to it.

>
> Anyway, BTAIM, is there anyway to avoid the save/restore rigmarole?

You could do the redirect in the open command:

open my $pipe, '|-', "/usr/bin/sort -n > $filename", or die $!;

Alas, this requires you to use the shell-interpreted-version of the open
rather than the shell-less version.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB

Re: Q on localizing *STDOUT and fork

In <1186036012.070979.170690@19g2000hsx.googlegroups.com> Brian McCauley writes:

>On Aug 2, 12:41 am, kj wrote:
>I wrote a quite detailed explanation of this here...

>http://groups.google.com/group/comp.lang.perl.misc/browse_f rm/thread/6d5c3d062c1e7608/fb1af6de388ee945

That was certainly illuminating. Thank you!

In that post you wrote:

> But if you do local(*STDOUT) you stash away the current contents of
> *STDOUT{IO} and make it empty. Now when you open(STDOUT,...) you get a
> new IO-thingy associated with *STDOUT{IO} but this IO-thingy is not
> associated with FD 1. As far as the current Perl process is concerned
> this is now the "standard output" but any child process that's created
> will still think of FD 1 as standard out.

Inspired by this I replaced

local *STDOUT;

with

local *STDOUT = *STDOUT;

The subsequent redirection of STDOUT now worked as desired, but
the original STDOUT was not restored after the end of the enclosing
block. My guess is that the "stashing away" that happens when one
uses local does not include the association of the file descriptor
with the file control block, which just gets lost after the
duplication with "open STDOUT, '>&', ...". If correct, this is a
shame, because it breaks the localization model. But I'm waaay
out of my depth here.

>I'm fairly sure I've seen modules on CPAN to wrap it up a bit...

If anyone happens to know the CPAN module that Brian is referring
to here please let me know. I looked for it without any luck. I
searched for terms like "redirect" and "redirection". (FWIW, I
did my search with Google restricted to site:search.cpan.org.)

TIA!

kj

--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.

Re: Q on localizing *STDOUT and fork

On Aug 3, 4:55 pm, kj wrote:
> In <1186036012.070979.170...@19g2000hsx.googlegroups.com> Brian McCauley

> >I'm fairly sure I've seen modules on CPAN to wrap it up a bit...
>
> If anyone happens to know the CPAN module that Brian is referring
> to here please let me know. I looked for it without any luck. I
> searched for terms like "redirect" and "redirection". (FWIW, I
> did my search with Google restricted to site:search.cpan.org.)

I think I was miss-remembering either Hook::Output::File or
SelectSaver - neither of which does what you want. Hook::Output::File
could be rewritten to do the right thing.

Re: Q on localizing *STDOUT and fork

In <1186167749.116827.250990@q75g2000hsh.googlegroups.com> Brian McCauley writes:

>On Aug 3, 4:55 pm, kj wrote:
>> In <1186036012.070979.170...@19g2000hsx.googlegroups.com> Brian McCauley

>> >I'm fairly sure I've seen modules on CPAN to wrap it up a bit...
>>
>> If anyone happens to know the CPAN module that Brian is referring
>> to here please let me know. I looked for it without any luck. I
>> searched for terms like "redirect" and "redirection". (FWIW, I
>> did my search with Google restricted to site:search.cpan.org.)

>I think I was miss-remembering either Hook::Output::File or
>SelectSaver - neither of which does what you want. Hook::Output::File
>could be rewritten to do the right thing.

Thanks! (That's one utterly mystifying chunk of code... And so
short too! I can't even begin to understand it; it sure is humbling.
Time for me to crawl back to my little world...)

kj

--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.

Re: Q on localizing *STDOUT and fork

On Aug 3, 4:55 pm, kj wrote:
> In <1186036012.070979.170...@19g2000hsx.googlegroups.com> Brian McCauley writes:
>
> >On Aug 2, 12:41 am, kj wrote:
> >I wrote a quite detailed explanation of this here...
> >http://groups.google.com/group/comp.lang.perl.misc/browse_f rm/thread/...
>
> That was certainly illuminating. Thank you!
>
> In that post you wrote:
>
> > But if you do local(*STDOUT) you stash away the current contents of
> > *STDOUT{IO} and make it empty. Now when you open(STDOUT,...) you get a
> > new IO-thingy associated with *STDOUT{IO} but this IO-thingy is not
> > associated with FD 1. As far as the current Perl process is concerned
> > this is now the "standard output" but any child process that's created
> > will still think of FD 1 as standard out.
>
> Inspired by this I replaced
>
> local *STDOUT;
>
> with
>
> local *STDOUT = *STDOUT;
>
> The subsequent redirection of STDOUT now worked as desired, but
> the original STDOUT was not restored after the end of the enclosing
> block. My guess is that the "stashing away" that happens when one
> uses local does not include the association of the file descriptor
> with the file control block, which just gets lost after the
> duplication with "open STDOUT, '>&', ...".

With local *STDOUT = *STDOUT all you are doing is manipulating the
very top layers of the stuff described in my other post.

You are not creating a new IO-thingy nor are you changing anything in
the IO-thingy, filedescriptor, FCB chain. There's still only one
filedescriptor (1) associated with the original FCB and as soon as you
reopen STDOUT your process looses all connection with that FCB.

Re: Q on localizing *STDOUT and fork

On Aug 3, 8:02 pm, Brian McCauley wrote:
> On Aug 3, 4:55 pm, kj wrote:
>
> > In <1186036012.070979.170...@19g2000hsx.googlegroups.com> Brian McCauley
> > >I'm fairly sure I've seen modules on CPAN to wrap it up a bit...
>
> > If anyone happens to know the CPAN module that Brian is referring
> > to here please let me know. I looked for it without any luck. I
> > searched for terms like "redirect" and "redirection". (FWIW, I
> > did my search with Google restricted to site:search.cpan.org.)
>
> I think I was miss-remembering either Hook::Output::File or
> SelectSaver - neither of which does what you want. Hook::Output::File
> could be rewritten to do the right thing.

On the other hand simply abstracting the code from my previous post
into a subroutine is not too hard:

use AtExit;
sub save_fd {
my $std = shift;
open ( my $saved,'>&', $std) or die $!;
my $mode = shift;
open $std, $mode, @_ and new AtExit sub {
# Actually the or die is of dubious utility here
open ($std,'>&', $saved) or die $!;
};
}

print "To terminal\n";
{
my $save_STDOUT = save_fd \*STDOUT, '>', 'x.log' or die $!;
print "To log\n";
}
print "To terminal\n";

Re: Q on localizing *STDOUT and fork

In <1186223134.723592.105510@d55g2000hsg.googlegroups.com> Brian McCauley writes:

>On Aug 3, 8:02 pm, Brian McCauley wrote:

>On the other hand simply abstracting the code from my previous post
>into a subroutine is not too hard:

>use AtExit;
>sub save_fd {
> my $std = shift;
> open ( my $saved,'>&', $std) or die $!;
> my $mode = shift;
> open $std, $mode, @_ and new AtExit sub {
> # Actually the or die is of dubious utility here
> open ($std,'>&', $saved) or die $!;
> };
>}

Thanks once more. AtExit is a handy module to know.

kj

--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.

Re: Q on localizing *STDOUT and fork

xhoster@gmail.com schreef:

> open my $pipe, '|-', "/usr/bin/sort -n > $filename", or die $!;
>
> Alas, this requires you to use the shell-interpreted-version of the
> open rather than the shell-less version.

Isn't there "concept" in Perl to change "SHELLMETAS"? (name as is used
in `man procmailrc`)

I often undefine SHELLMETAS in a .procmailrc, to prevent the shell being
used.

--
Affijn, Ruud

"Gewoon is een tijger."

Re: Q on localizing *STDOUT and fork

"Dr.Ruud" wrote:
> xhoster@gmail.com schreef:
>
> > open my $pipe, '|-', "/usr/bin/sort -n > $filename", or die $!;
> >
> > Alas, this requires you to use the shell-interpreted-version of the
> > open rather than the shell-less version.
>
> Isn't there "concept" in Perl to change "SHELLMETAS"? (name as is used
> in `man procmailrc`)

Well, there is the 4 or more form of open. That suppresses the shell.

>
> I often undefine SHELLMETAS in a .procmailrc, to prevent the shell being
> used.

But that would defeat the purpose. The ">" redirection has to be
interpreted by the shell, otherwise there is no point to doing it that way.

You could always pass $filename through quotemeta, although for all I know
that might backwhack some things which are ordinary to the shell but become
special when backwhacked.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB

Re: Q on localizing *STDOUT and fork

On 3 Aug, 20:02, Brian McCauley wrote:
> On Aug 3, 4:55 pm, kj wrote:

> > If anyone happens to know the CPAN module that Brian is referring
> > to here please let me know. I looked for it without any luck. I
> > searched for terms like "redirect" and "redirection". (FWIW, I
> > did my search with Google restricted to site:search.cpan.org.)
>
> I think I was miss-remembering either Hook::Output::File or
> SelectSaver - neither of which does what you want.

For the sake of the archives, I'd like to add that various modules
with "Capture" in the name do indeed do what the OP was seeking and
maybe this is what I was remembering.