WaitForExit() times out when I try to capture multi-line Stdout

I have written a short VB.NET function (see below) which captures
StandardOutput from a particular Windows program (myProgram.exe).

The problem is this: if myProgram.exe returns more than a single
line of text then WaitForExit() always times out! The function
works fine if myProgram.exe return only one line of text. It also
works fine with multi-line output -- returning a String with embedded
newlines -- provided I remove the test on WaitForExit(). Note that
if I execute myProgram.exe from the command line it returns quickly,
no matter how many lines of text I have asked for.

My understanding of WaitForExit() is that it returns True when the
process that it is waiting on exits, or False if the process has
not exited after a specified number of milliseconds.

Clearly I must have misunderstood something, but I see nothing in
the online documentation to suggest why it would matter whether
the the output from the program contains one line or many. Can
anyone tell me what I am doing wrong?


Public Function myFunction(ByVal myOptions As String) As String

Dim psInfo As New Diagnostics.ProcessStartInfo
psInfo.FileName = "myProgram.exe"
psInfo.Arguments = myOptions
psInfo.CreateNoWindow = True
psInfo.UseShellExecute = False
psInfo.RedirectStandardOutput = True

Dim newProc As Diagnostics.Process = Diagnostics.Process.Start(psInfo)

Dim myOutput As String = ""

If newProc.WaitForExit(6000) Then myOutput = newProc.StandardOutput.ReadToEnd

newProc.StandardOutput.Close()
newProc.Close()
newProc.Dispose()

Return myOutput
End Function
--
John Brock
jbrock@panix.com

Re: WaitForExit() times out when I try to capture multi-line Stdout

On Mon, 07 Apr 2008 20:26:17 -0700, John Brock wrote:

> [...]
> Clearly I must have misunderstood something, but I see nothing in
> the online documentation to suggest why it would matter whether
> the the output from the program contains one line or many. Can
> anyone tell me what I am doing wrong?

You have redirected the standard output, but you are not reading from it
when you are blocked on the call to WaitForExit(). So, when the other
process fills the relatively small output buffer for standard output, it
blocks. It can't proceed until you read data from the standard output,
freeing up the buffer space so that it can emit more output.

Since you are using ReadToEnd(), there's really no reason for you to call
WaitForExit(). By definition, the standard output stream won't end until
the process has exited. So you can just call ReadToEnd() itself, and it
will block until the other process has exited, just as WaitForExit() would
have had you not redirected the output.

The docs do in fact describe this issue. From the page for
ProcessStartInfo.RedirectStandardOutput:
http://msdn2.microsoft.com/en-us/library/system.diagnostics. processstartinfo.redirectstandardoutput.aspx

When the child process writes enough data to fill its
redirected stream, it is dependent on the parent. The
child process waits for the next write operation until
the parent reads from the full stream or closes the
stream. The deadlock condition results when the caller
and child process wait for each other to complete an
operation, and neither can continue.

Your code doesn't quite deadlock, because you've included a timeout that
will break the deadlock. But otherwise, that's the exact scenario you're
running into.

Pete

Re: WaitForExit() times out when I try to capture multi-line Stdout

You must use a worker thread to read the standart input/ouput
while waiting for the program end in the main thread.

Re: WaitForExit() times out when I try to capture multi-line Stdout

On Tue, 08 Apr 2008 03:13:41 -0700, SoftLion
wrote:

> You must use a worker thread to read the standart input/ouput
> while waiting for the program end in the main thread.

That's not true. See my own reply to the OP.

Re: WaitForExit() times out when I try to capture multi-line Stdout

In article ,
Peter Duniho wrote:
>On Mon, 07 Apr 2008 20:26:17 -0700, John Brock wrote:

>> [...]
>> Clearly I must have misunderstood something, but I see nothing in
>> the online documentation to suggest why it would matter whether
>> the the output from the program contains one line or many. Can
>> anyone tell me what I am doing wrong?

>You have redirected the standard output, but you are not reading from it
>when you are blocked on the call to WaitForExit(). So, when the other
>process fills the relatively small output buffer for standard output, it
>blocks. It can't proceed until you read data from the standard output,
>freeing up the buffer space so that it can emit more output.
>
>Since you are using ReadToEnd(), there's really no reason for you to call
>WaitForExit(). By definition, the standard output stream won't end until
>the process has exited. So you can just call ReadToEnd() itself, and it
>will block until the other process has exited, just as WaitForExit() would
>have had you not redirected the output.
>
>The docs do in fact describe this issue. From the page for
>ProcessStartInfo.RedirectStandardOutput:
>http://msdn2.microsoft.com/en-us/library/system.diagnostics .processstartinfo.redirectstandardoutput.aspx
>
> When the child process writes enough data to fill its
> redirected stream, it is dependent on the parent. The
> child process waits for the next write operation until
> the parent reads from the full stream or closes the
> stream. The deadlock condition results when the caller
> and child process wait for each other to complete an
> operation, and neither can continue.
>
>Your code doesn't quite deadlock, because you've included a timeout that
>will break the deadlock. But otherwise, that's the exact scenario you're
>running into.

Thank you, that is very helpful! I'm still wondering about something
though...

I put the timeout in my function to mimic the timeout in the Shell()
function, which will break if the program it is running hangs.
The document you pointed me to seems to suggest (in the first
example) that I can avoid deadlocks by calling WaitForExit after
ReadToEnd().

The code example avoids a deadlock condition by calling
p.StandardOutput.ReadToEnd before p.WaitForExit. A deadlock
condition can result if the parent process calls p.WaitForExit
before p.StandardOutput.ReadToEnd and the child process
writes enough text to fill the redirected stream.

But what if the program hangs? How can I Read To End if the program
I am running doesn't end, if it just stops writing to Stdout, but
doesn't exit or close the stream? How would I ever even reach the
WaitForExit() statement?

Am I still misunderstanding something? Or is there just no way to
implement a fully functional Shell() style timeout without resorting
to threads?
--
John Brock
jbrock@panix.com

Re: WaitForExit() times out when I try to capture multi-line Stdout

On Tue, 08 Apr 2008 12:13:22 -0700, John Brock wrote:

> [...]
> I put the timeout in my function to mimic the timeout in the Shell()
> function, which will break if the program it is running hangs.
> The document you pointed me to seems to suggest (in the first
> example) that I can avoid deadlocks by calling WaitForExit after
> ReadToEnd().

Well, it's a bit unfortunate that the docs is written with the exact
wording it is, because there's really no need to call WaitForExit() after
you've called ReadToEnd(). ReadToEnd() won't return until the process has
ended, which is the same condition that causes WaitForExit() to return
immediately.

What the docs really should have said is "instead of p.WaitForExit" rather
than "before p.WaitForExit".

Now, as for your actual question...

> But what if the program hangs? How can I Read To End if the program
> I am running doesn't end, if it just stops writing to Stdout, but
> doesn't exit or close the stream? How would I ever even reach the
> WaitForExit() statement?

It wouldn't. If you are concerned about the possibility of the process
hanging, you can't use ReadToEnd().

> Am I still misunderstanding something? Or is there just no way to
> implement a fully functional Shell() style timeout without resorting
> to threads?

I guess that depends on your definition of "resorting to threads". I
don't think the word "resorting" is appropriate, because that implies some
negative consequence of using another thread. But no, if you want to be
able to implement a time-out as well as redirect the standard output,
you'll have to introduce a second thread to the implementation in some way.

There are a variety of ways you could do this:

-- Just start another thread that calls ReadToEnd() while your main
thread calls WaitForExit()

-- Call ReadToEnd() from your main thread, and create a timer (for
example, Threading.Timer), providing an elapsed event handler that
terminates the process

-- Call WaitForExit() in the main thread and use the asynchronous read
methods on the StreamReader.BaseStream (using the StandardOutput
StreamReader, of course).

The latter two would not require you to explicitly create a new thread,
but they still wind up using a second thread to handle the asynchronous
execution.

Pete