Force return of the diamond operator from network socket

I am involved with a project that will use a server process written in C
and a client written in Perl. The processes will communicate over a
network socket that is opened when the Perl process starts and and remains
open throughout the life of the Perl program.

The server is queried by the Perl program and responds with short text
strings that are newline terminated. The server can return from one to
three lines depending on the query. This part is working well.

Here is where I am running into trouble. I am using the diamond
operator to read from the socket as in:

while (<$socket>) {
chomp;
print "$_\n";
}

however, the diamond operator doesn't return despite trying to send a NULL
string from the C program through the socket. Here is a snippet of my C
program:

fprintf(fout, "");
fflush(fout);

however, I'm not certain that the NULL string is even making it through
the socket to the Perl program.

Perhaps there is a better method of signalling the end of the C program's
output to the Perl program. At the moment I'm a bit stumped.

- Nate >>

--

"The optimist proclaims that we live in the best of all
possible worlds. The pessimist fears this is true."

Re: Force return of the diamond operator from network socket

>>>>> "NB" == Nate Bargmann writes:

NB> while (<$socket>) {
NB> chomp;
NB> print "$_\n";
NB> }

NB> however, the diamond operator doesn't return despite trying to
NB> send a NULL string from the C program through the socket. Here is
NB> a snippet of my C program:

NB> fprintf(fout, "");
NB> fflush(fout);

first, use write() so you don't need the fflush call. that means you
need to do an open() and not an fopen. it will run faster as well as you
avoid the stdio overhead.

a c null string is a single byte of 0. the <> operator expects to read a
line of text which a 0 byte is not. that is why you hang. you should
either send complete lines that end in \n (or cleaner for the net
\015\012) or you need to stop using <> to read from the socket. you
can't easily mix/match line oriented protocols with binary ones (and
your 0 byte forces you into a binary protocol.

NB> Perhaps there is a better method of signalling the end of the C
NB> program's output to the Perl program. At the moment I'm a bit
NB> stumped.

if this is a one and done request (like web) just close the socket and
the <> will return with an undef (eof detected). otherwise use a proper
line with some token like END or whatever you want that works within
your protocol. and yes, you have a protocol and you need to make sure it
can handle inline commands like that if you want line oriented I/O.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Architecture, Development, Training, Support, Code Review ------
----------- Search or Offer Perl Jobs ----- http://jobs.perl.org ---------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------

Re: Force return of the diamond operator from network socket

On Wed, 09 Jan 2008 21:11:09 +0000, Uri Guttman wrote:

>>>>>> "NB" == Nate Bargmann writes:
>
> NB> while (<$socket>) {
> NB> chomp;
> NB> print "$_\n";
> NB> }
>
> NB> however, the diamond operator doesn't return despite trying to
> NB> send a NULL string from the C program through the socket. Here is
> NB> a snippet of my C program:
>
> NB> fprintf(fout, "");
> NB> fflush(fout);
>
> first, use write() so you don't need the fflush call. that means you
> need to do an open() and not an fopen. it will run faster as well as you
> avoid the stdio overhead.

I think the reason the stdio functions are used is due to portability to
win32 although Linux is the primary platform. The fflush() was my addition
to try and force the NULL string through after the fprintf().

> if this is a one and done request (like web) just close the socket and
> the <> will return with an undef (eof detected). otherwise use a proper
> line with some token like END or whatever you want that works within
> your protocol. and yes, you have a protocol and you need to make sure it
> can handle inline commands like that if you want line oriented I/O.

Unlike a web browser, my intent is a persistent socket. It's possible
that the client may query the server quite frequently, every few seconds
or perhaps even more frequently. It seems to me that opening and closing
the socket that frequently would itself create a lot of overhead.

The END idea is a good one as, at this point, it doesn't conflict with the
text returned by the server. So, would the recv() function be a better
choice to read the socket? Especially since two to five lines (now
including the END token) could be available and the data length is
variable.

Thanks, Uri,

- Nate >>

--

"The optimist proclaims that we live in the best of all
possible worlds. The pessimist fears this is true."

Re: Force return of the diamond operator from network socket

On Wed, 09 Jan 2008 15:25:14 -0600 Nate Bargmann wrote:

NB> Unlike a web browser, my intent is a persistent socket. It's possible
NB> that the client may query the server quite frequently, every few seconds
NB> or perhaps even more frequently. It seems to me that opening and closing
NB> the socket that frequently would itself create a lot of overhead.

HTTP and web browsers works just fine with a persistent connection;
see http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html for example.
HTTP 1.1 makes it the default, in fact.

Unless you're doing something unusual, you may find that HTTP is a good
solution for you. Not only are there a million implementations of it
out there on both the client and the server side, but it's also easy to
script and test HTTP interactions. HTTP also allows a lot of
flexibility with the URI, so your client/server interactions can be as
rich as a URI will express (which is plenty). Binary data is well
supported in several encoding schemes, including raw or compressed raw
data if you need it. Finally, HTTP error codes allow you out-of-band
signaling back to the client about the server's state.

HTTP is not perfect, but I would at least consider it for generic
network communications nowadays.

Ted

Re: Force return of the diamond operator from network socket

>>>>> "NB" == Nate Bargmann writes:

NB> On Wed, 09 Jan 2008 21:11:09 +0000, Uri Guttman wrote:
>>>>>>> "NB" == Nate Bargmann writes:
>>
NB> while (<$socket>) {
NB> chomp;
NB> print "$_\n";
NB> }
>>
NB> however, the diamond operator doesn't return despite trying to
NB> send a NULL string from the C program through the socket. Here is
NB> a snippet of my C program:
>>
NB> fprintf(fout, "");
NB> fflush(fout);
>>
>> first, use write() so you don't need the fflush call. that means you
>> need to do an open() and not an fopen. it will run faster as well as you
>> avoid the stdio overhead.

NB> I think the reason the stdio functions are used is due to portability to
NB> win32 although Linux is the primary platform. The fflush() was my addition
NB> to try and force the NULL string through after the fprintf().

that has nothing to do with anything. writing to a socket is not
something that ever needs to use stdio. that just slows you down and
requires extra fflush calls. but this is something that is not perl
related and off topic.

>> if this is a one and done request (like web) just close the socket and
>> the <> will return with an undef (eof detected). otherwise use a proper
>> line with some token like END or whatever you want that works within
>> your protocol. and yes, you have a protocol and you need to make sure it
>> can handle inline commands like that if you want line oriented I/O.

NB> Unlike a web browser, my intent is a persistent socket. It's possible
NB> that the client may query the server quite frequently, every few seconds
NB> or perhaps even more frequently. It seems to me that opening and closing
NB> the socket that frequently would itself create a lot of overhead.

NB> The END idea is a good one as, at this point, it doesn't conflict with the
NB> text returned by the server. So, would the recv() function be a better
NB> choice to read the socket? Especially since two to five lines (now
NB> including the END token) could be available and the data length is
NB> variable.

if you are sticking with a line oriented protocol then <> is fine. it
still limits the client to being blocked on the protocol but that is ok
too if you understand that. just keep reading with <> in the loop until you see
END (or whatever token you choose) and collect the lines. then process
them and reenter the loop at the top. something like this (untested
code):

sub get_results {

my ($sock) = @_ ;

my @lines ;

while( my $line = <$sock> ) {

return @lines if $line =~ /^END$/ ;
push @lines, $line ;
}
}

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Architecture, Development, Training, Support, Code Review ------
----------- Search or Offer Perl Jobs ----- http://jobs.perl.org ---------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------

Re: Force return of the diamond operator from network socket

On Wed, 09 Jan 2008 22:17:05 +0000, Uri Guttman wrote:

> if you are sticking with a line oriented protocol then <> is fine. it
> still limits the client to being blocked on the protocol but that is ok
> too if you understand that. just keep reading with <> in the loop until you see
> END (or whatever token you choose) and collect the lines. then process
> them and reenter the loop at the top. something like this (untested
> code):
>
> sub get_results {
>
> my ($sock) = @_ ;
>
> my @lines ;
>
> while( my $line = <$sock> ) {
>
> return @lines if $line =~ /^END$/ ;
> push @lines, $line ;
> }
> }

Thanks so much, Uir! That's exactly the kind of jump-start I needed. It
works perfectly. Now I need to brush up on my poor Perl skills. :-)

- Nate >>

--

"The optimist proclaims that we live in the best of all
possible worlds. The pessimist fears this is true."

Re: Force return of the diamond operator from network socket

Nate Bargmann wrote:
> I am involved with a project that will use a server process written in C
> and a client written in Perl. The processes will communicate over a
> network socket that is opened when the Perl process starts and and
> remains open throughout the life of the Perl program.
>
> The server is queried by the Perl program and responds with short text
> strings that are newline terminated. The server can return from one to
> three lines depending on the query. This part is working well.
>
> Here is where I am running into trouble. I am using the diamond
> operator to read from the socket as in:
>
> while (<$socket>) {
> chomp;
> print "$_\n";
> }
>
> however, the diamond operator doesn't return despite trying to send a
> NULL string from the C program through the socket. Here is a snippet of
> my C program:
>
> fprintf(fout, "");
> fflush(fout);

How about sending an empty line:

fprintf(fout, "\n");
fflush(fout);

>
> however, I'm not certain that the NULL string is even making it through
> the socket to the Perl program.

How could it?

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

Re: Force return of the diamond operator from network socket

Nate Bargmann wrote:
> I am involved with a project that will use a server process written in C
> and a client written in Perl. The processes will communicate over a
> network socket that is opened when the Perl process starts and and remains
> open throughout the life of the Perl program.
>
> The server is queried by the Perl program and responds with short text
> strings that are newline terminated. The server can return from one to
> three lines depending on the query. This part is working well.
>
> Here is where I am running into trouble. I am using the diamond
> operator to read from the socket as in:
>
> while (<$socket>) {
> chomp;
> print "$_\n";
> }
>
> however, the diamond operator doesn't return despite trying to send a NULL
> string from the C program through the socket. Here is a snippet of my C
> program:
>
> fprintf(fout, "");
> fflush(fout);
>
> however, I'm not certain that the NULL string is even making it through
> the socket to the Perl program.
>
> Perhaps there is a better method of signalling the end of the C program's
> output to the Perl program. At the moment I'm a bit stumped.
>
> - Nate >>
>

By the way, you can change what <> considers its line terminator by
setting $/ ($INPUT_RECORD_SEPARATOR). Look in perlvar for the details
on this.

But that said, Ted's recommendation to use HTTP for your net
protocol is a very good recommendation. If you need some more
structure to your data, you can add YAML inside the
HTTP. A gazillion folks are doing this already with great success!

Re: Force return of the diamond operator from network socket

On 2008-01-09 21:11, Uri Guttman wrote:
>>>>>> "NB" == Nate Bargmann writes:
>
> NB> while (<$socket>) {
> NB> chomp;
> NB> print "$_\n";
> NB> }
>
> NB> however, the diamond operator doesn't return despite trying to
> NB> send a NULL string from the C program through the socket. Here is
> NB> a snippet of my C program:
>
> NB> fprintf(fout, "");
> NB> fflush(fout);
>
> first, use write() so you don't need the fflush call. that means you
> need to do an open() and not an fopen. it will run faster as well as you
> avoid the stdio overhead.
>
> a c null string is a single byte of 0.

fprintf(fout, "");

doesn't include the terminating null byte (Otherwise files written by C
programs would be riddled with 0 bytes), so it adds exactly nothing to
the output buffer.


> the <> operator expects to read a
> line of text which a 0 byte is not.

Depends on the value of $/. But since the 0 byte is never sent it cannot
be received and hence cannot have any effect.

hp