open3 and STDERR blocking

am 08.02.2006 20:08:22 von scott.pham

I'm currently using open3 to launch ssh sessions, when I use -vvv with
ssh the process is blocked.

when I close STDERR before the open3 call, it works, except I lose all
information on STDERR. I've tried to dupe that handle to STDOUT and
still the same issues.

The error handle I pass to open3, doesn't seem to work if I close
STDERR. Anyone have a work around for this issue?

I've tried several non blocking code using IO::Select,
IO::Handle(->blocking(0)). Nothing seems to work like it should. Some
pointers would be greate :).

Re: open3 and STDERR blocking

am 08.02.2006 21:35:16 von xhoster

"Scott" wrote:
> I'm currently using open3 to launch ssh sessions, when I use -vvv with
> ssh the process is blocked.
>
> when I close STDERR before the open3 call, it works, except I lose all
> information on STDERR. I've tried to dupe that handle to STDOUT and
> still the same issues.
>
> The error handle I pass to open3, doesn't seem to work if I close
> STDERR. Anyone have a work around for this issue?

You are doing something wrong. We don't know what, because you haven't
shown us what you are doing.

>
> I've tried several non blocking code using IO::Select,
> IO::Handle(->blocking(0)). Nothing seems to work like it should. Some
> pointers would be greate :).

Some code would be great.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB

Re: open3 and STDERR blocking

am 08.02.2006 22:03:44 von scott.pham

$bootstrap = 'EOK';
'
use IPC::Open3;
require "flush.pl";
my $data=;
eval "$data";
exit 0;
'
EOK

$bootstrap =~ s/\n//g;
@sshargs = (
"ssh", "-vvv", '-l', 'root',
"-o",
"ChallengeResponseAuthentication=no", "-o",
"PreferredAuthentications=publickey", "-o",
"NumberOfPasswordPrompts=0", "-o",
"PasswordAuthentication=no", "-o",
"PubkeyAuthentication=yes", "-o",
"UseRsh=no", "-o",
"RhostsAuthentication=no", "-o",
"RhostsRSAAuthentication=no", "-o",
"FallBackToRsh=no", "-o",
"StrictHostKeyChecking=no", "-o",
"BatchMode=yes", $host,
'PATH=/usr/local/bin:/usr/bin:\$PATH; export PATH;',
'perl', "-e",
"$bootstrap"
);

open3( *WR, *RD, *RD , @sshargs );

print WR "$script\n";
close WR;

while(){
print;
}
close RD;

script stalls at this point
"[pid 3769] write(2, "debug2: mac_init: found hmac-md5"..., 34" from
strace -f

If I put a close STDERR before the open3, it works perfectly, or if I
only have '-v' instead of '-vvv'.

Not sure what I'm doing wrong.

Re: open3 and STDERR blocking

am 08.02.2006 22:19:27 von scott.pham

Looking at this further, thre 'stalling' only comes when it's a pretty
descent size script, anything over around 400 lines. So I believe the
blocking is currently in the 'writer'.

Re: open3 and STDERR blocking

am 09.02.2006 01:43:58 von xhoster

"Scott" wrote:
> Looking at this further, thre 'stalling' only comes when it's a pretty
> descent size script, anything over around 400 lines. So I believe the
> blocking is currently in the 'writer'.

It sounds like your classic deadlock. The original script won't read until
it is done writing, and the remote script eventually stops reading until it
is listened to. They are both trying to talk, and neither will listen
until the other one does.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB

Re: open3 and STDERR blocking

am 21.02.2006 18:47:52 von Charles DeRykus

Scott wrote:
> $bootstrap = 'EOK';
> '
> use IPC::Open3;
> require "flush.pl";
> my $data=;
> eval "$data";
> exit 0;
> '
> EOK
>
> $bootstrap =~ s/\n//g;
> @sshargs = (
> "ssh", "-vvv", '-l', 'root',
> "-o",
> "ChallengeResponseAuthentication=no", "-o",
> "PreferredAuthentications=publickey", "-o",
> "NumberOfPasswordPrompts=0", "-o",
> "PasswordAuthentication=no", "-o",
> "PubkeyAuthentication=yes", "-o",
> "UseRsh=no", "-o",
> "RhostsAuthentication=no", "-o",
> "RhostsRSAAuthentication=no", "-o",
> "FallBackToRsh=no", "-o",
> "StrictHostKeyChecking=no", "-o",
> "BatchMode=yes", $host,
> 'PATH=/usr/local/bin:/usr/bin:\$PATH; export PATH;',
> 'perl', "-e",
> "$bootstrap"
> );
>
> open3( *WR, *RD, *RD , @sshargs );
>
> print WR "$script\n";
> close WR;
>
> while(){
> print;
> }
> close RD;
>
> script stalls at this point
> "[pid 3769] write(2, "debug2: mac_init: found hmac-md5"..., 34" from
> strace -f
>
>
> If I put a close STDERR before the open3, it works perfectly, or if I
> only have '-v' instead of '-vvv'.
>

[posted and mailed]

I'm not sure why why deadlock would be an issue since you're merging
stdout/stderr but it may still be occurring.

From an Open3/deadlock thread:

: The *only* proper solution for avoiding deadlock (if one has no
: knowledge of when the child process will produce stuff on it's
stdout or
: when it will produce stuff on it's stderr), is to redirect one
handle or
: the other to a file (possibly a temp file), or else use IO::Select to
: read from both simultaneously.

You might try re-directing stderr and/or stdout to a temp file.

HTH,
--
Charles DeRykus

Re: open3 and STDERR blocking

am 21.02.2006 18:52:46 von Charles DeRykus

Re: open3 and STDERR blocking

am 21.02.2006 18:54:58 von Charles DeRykus

Scott wrote:
> I'm currently using open3 to launch ssh sessions, when I use -vvv with
> ssh the process is blocked.
>
> when I close STDERR before the open3 call, it works, except I lose all
> information on STDERR. I've tried to dupe that handle to STDOUT and
> still the same issues.
>
> The error handle I pass to open3, doesn't seem to work if I close
> STDERR. Anyone have a work around for this issue?
>
> I've tried several non blocking code using IO::Select,
> IO::Handle(->blocking(0)). Nothing seems to work like it should. Some
> pointers would be greate :).

Although I'm not sure why why deadlock would be an issue since you're
merging stdout/stderr but to rule it out:

From an Open3/deadlock thread I spotted:

: The *only* proper solution for avoiding deadlock (if one has no
: knowledge of when the child process will produce stuff on it's
stdout or
: when it will produce stuff on it's stderr), is to redirect one
handle or
: the other to a file (possibly a temp file), or else use IO::Select to
: read from both simultaneously.

You might try re-directing stderr and/or stdout to a temp file.

HTH,
--
Charles DeRykus

Re: open3 and STDERR blocking

am 22.02.2006 17:29:26 von xhoster

Charles DeRykus wrote:
> Scott wrote:
> > $bootstrap = 'EOK';
> > '
> > use IPC::Open3;
> > require "flush.pl";
> > my $data=;
> > eval "$data";
> > exit 0;
> > '
> > EOK
> >
> > $bootstrap =~ s/\n//g;
> > @sshargs = (
> > "ssh", "-vvv", '-l', 'root',
> > "-o",
> > "ChallengeResponseAuthentication=no", "-o",
> > "PreferredAuthentications=publickey", "-o",
> > "NumberOfPasswordPrompts=0", "-o",
> > "PasswordAuthentication=no", "-o",
> > "PubkeyAuthentication=yes", "-o",
> > "UseRsh=no", "-o",
> > "RhostsAuthentication=no", "-o",
> > "RhostsRSAAuthentication=no", "-o",
> > "FallBackToRsh=no", "-o",
> > "StrictHostKeyChecking=no", "-o",
> > "BatchMode=yes", $host,
> > 'PATH=/usr/local/bin:/usr/bin:\$PATH; export PATH;',
> > 'perl', "-e",
> > "$bootstrap"
> > );
> >
> > open3( *WR, *RD, *RD , @sshargs );
> >
> > print WR "$script\n";
> > close WR;
> >
> > while(){
> > print;
> > }
> > close RD;
> >
> > script stalls at this point
> > "[pid 3769] write(2, "debug2: mac_init: found hmac-md5"..., 34" from
> > strace -f
> >
> >
> > If I put a close STDERR before the open3, it works perfectly, or if I
> > only have '-v' instead of '-vvv'.
> >
>
> [posted and mailed]
>
> I'm not sure why why deadlock would be an issue since you're merging
> stdout/stderr but it may still be occurring.

There are 3!/1! = 6 ways for deadlocks to occur under open3. Merging
stdout and stderr only eliminates 4 of those ways. He is running into one
of the other 2.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB

Re: open3 and STDERR blocking

am 23.02.2006 10:39:43 von Charles DeRykus

xhoster@gmail.com wrote:
> Charles DeRykus wrote:
>> Scott wrote:
>>> $bootstrap = 'EOK';
>>> '
>>> use IPC::Open3;
>>> require "flush.pl";
>>> my $data=;
>>> eval "$data";
>>> exit 0;
>>> '
>>> EOK
>>>
>>> $bootstrap =~ s/\n//g;
>>> @sshargs = (
>>> "ssh", "-vvv", '-l', 'root',
>>> "-o",
>>> "ChallengeResponseAuthentication=no", "-o",
>>> "PreferredAuthentications=publickey", "-o",
>>> "NumberOfPasswordPrompts=0", "-o",
>>> "PasswordAuthentication=no", "-o",
>>> "PubkeyAuthentication=yes", "-o",
>>> "UseRsh=no", "-o",
>>> "RhostsAuthentication=no", "-o",
>>> "RhostsRSAAuthentication=no", "-o",
>>> "FallBackToRsh=no", "-o",
>>> "StrictHostKeyChecking=no", "-o",
>>> "BatchMode=yes", $host,
>>> 'PATH=/usr/local/bin:/usr/bin:\$PATH; export PATH;',
>>> 'perl', "-e",
>>> "$bootstrap"
>>> );
>>>
>>> open3( *WR, *RD, *RD , @sshargs );
>>>
>>> print WR "$script\n";
>>> close WR;
>>>
>>> while(){
>>> print;
>>> }
>>> close RD;
>>>
>>> script stalls at this point
>>> "[pid 3769] write(2, "debug2: mac_init: found hmac-md5"..., 34" from
>>> strace -f
>>>
>>>
>>> If I put a close STDERR before the open3, it works perfectly, or if I
>>> only have '-v' instead of '-vvv'.
>>>
>> [posted and mailed]
>>
>> I'm not sure why why deadlock would be an issue since you're merging
>> stdout/stderr but it may still be occurring.
>
> There are 3!/1! = 6 ways for deadlocks to occur under open3. Merging
> stdout and stderr only eliminates 4 of those ways. He is running into one
> of the other 2.

Yes, I see what you mean. The massive 400 line $bootstrap the OP
mentions could be doing wild things that'd increase the risk of
deadlock too. It'd probably help to have more visibility about
what's there. I could imagine some very slow and/or intermittent
writes being problematic. Hopefully output to temp files would help though.

(Sorry for the earlier multiple posts.)

--
Charles DeRykus