Sockets (reading)

Hi.

During a socket read, why would all the requested number of bytes not
get sent? For example, I request 1000 bytes:

$data = @socket_read ($socket, 2048, PHP_BINARY_READ);
?>

This is actually in a loop, so I can get all the data if split up. So,
for example, here's how the data split up in 3 iterations (for 1000
bytes):

650 bytes
200 bytes
150 bytes

But if I can accept up to 2048 bytes per socket read, why would it not
pull all 1000 bytes initially in 1 step? Any thoughts on this would be
greatly appreciated!

Thanks,
~Philip

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Sockets (reading)

Philip Thompson wrote:
> Hi.
>
> During a socket read, why would all the requested number of bytes not
> get sent? For example, I request 1000 bytes:
>
> > $data = @socket_read ($socket, 2048, PHP_BINARY_READ);
> ?>
>
> This is actually in a loop, so I can get all the data if split up. So,
> for example, here's how the data split up in 3 iterations (for 1000 bytes):
>
> 650 bytes
> 200 bytes
> 150 bytes
>
> But if I can accept up to 2048 bytes per socket read, why would it not
> pull all 1000 bytes initially in 1 step? Any thoughts on this would be
> greatly appreciated!
>
> Thanks,
> ~Philip
>

Is it possible that you are receiving a \r, \n, or \0 and that is
stopping the input?

Check the Parameters the second argument for Length options.
http://us3.php.net/socket_read


Here is a shortened version of a script that I use to do something similar.


error_reporting(E_ALL);
ini_set('display_errors', 'On');

// Set time limit to indefinite execution
set_time_limit(0);

// Set the ip and port we will listen on
define('LISTEN_IP', ''); // IP to listin on
define('LISTEN_PORT', ); // Port number
define('PACKET_SIZE', 512); // 512 bytes
define('SOCKET_TIMOUT', 2); // Seconds

/* Open a server socket to port 1234 on localhost */
if ( $socket = @stream_socket_server('udp://'.LISTEN_IP.':'.LISTEN_PORT,
$errno, $errstr, STREAM_SERVER_BIND) ) {
while ( true ) {
$buff = stream_socket_recvfrom($socket, PACKET_SIZE, 0, $remote_ip);
if ( !empty($buff) ) {
print('Received Data: '.PHP_EOL);
print('Do something with it...'.PHP_EOL);
} else {
print('Empty String'.PHP_EOL);
}
}
fclose($socket);
} else {
print("[{$errno}] {$error}".PHP_EOL);
}

?>


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

RE: Sockets (reading)

From: Philip Thompson
>=20
> During a socket read, why would all the requested number of bytes not

> get sent? For example, I request 1000 bytes:
>=20
> > $data =3D @socket_read ($socket, 2048, PHP_BINARY_READ);
> ?>
>=20
> This is actually in a loop, so I can get all the data if split up. So,

> for example, here's how the data split up in 3 iterations (for 1000 =20
> bytes):
>=20
> 650 bytes
> 200 bytes
> 150 bytes
>=20
> But if I can accept up to 2048 bytes per socket read, why would it not

> pull all 1000 bytes initially in 1 step? Any thoughts on this would be

> greatly appreciated!

Because that's the way TCP/IP works, by design. TCP is a stream
protocol. It guarantees all of the bytes written to one end of the pipe
will come out the other end in the same order, but not necessarily in
the same groupings. There are a number of buffers along the way that
might split them up, as well as limits on packet sizes in the various
networks it passed through. So you get what is available in the last
buffer when a timer expires, no more, and no less.

If you have serialized data that needs to be grouped in specific blocks,
your application will need to keep track of those blocks, reassembling
or splitting the streamed data as necessary. You could use UDP which
does guarantee that packets will be kept together, but that protocol
doesn't guarantee delivery.

Bob McConnell

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Sockets (reading)

On Aug 26, 2009, at 11:56 AM, Bob McConnell wrote:

> From: Philip Thompson
>>
>> During a socket read, why would all the requested number of bytes not
>> get sent? For example, I request 1000 bytes:
>>
>> >> $data = @socket_read ($socket, 2048, PHP_BINARY_READ);
>> ?>
>>
>> This is actually in a loop, so I can get all the data if split up.
>> So,
>> for example, here's how the data split up in 3 iterations (for 1000
>> bytes):
>>
>> 650 bytes
>> 200 bytes
>> 150 bytes
>>
>> But if I can accept up to 2048 bytes per socket read, why would it
>> not
>> pull all 1000 bytes initially in 1 step? Any thoughts on this would
>> be
>> greatly appreciated!
>
> Because that's the way TCP/IP works, by design. TCP is a stream
> protocol. It guarantees all of the bytes written to one end of the
> pipe
> will come out the other end in the same order, but not necessarily in
> the same groupings. There are a number of buffers along the way that
> might split them up, as well as limits on packet sizes in the various
> networks it passed through. So you get what is available in the last
> buffer when a timer expires, no more, and no less.
>
> If you have serialized data that needs to be grouped in specific
> blocks,
> your application will need to keep track of those blocks, reassembling
> or splitting the streamed data as necessary. You could use UDP which
> does guarantee that packets will be kept together, but that protocol
> doesn't guarantee delivery.
>
> Bob McConnell

Thank you for your input.

Is it guaranteed that at least 1 byte will be sent each time? For
example, if I know the data length...

$input = '';

for ($i=0; $i<$dataLength; $i++) {
// Read 1 byte at a time
if (($data = @socket_read ($socket, 1, PHP_BINARY_READ)) !==
false) {
$input .= $data;
}
}

return $input;
?>

Or is this a completely unreasonable and unnecessary way to get the
data?

Thanks,
~Philip

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Sockets (reading)

Bob McConnell wrote:
> From: Philip Thompson
>> During a socket read, why would all the requested number of bytes not
>
>> get sent? For example, I request 1000 bytes:
>>
>> >> $data = @socket_read ($socket, 2048, PHP_BINARY_READ);
>> ?>
>>
>> This is actually in a loop, so I can get all the data if split up. So,
>
>> for example, here's how the data split up in 3 iterations (for 1000
>> bytes):
>>
>> 650 bytes
>> 200 bytes
>> 150 bytes
>>
>> But if I can accept up to 2048 bytes per socket read, why would it not
>
>> pull all 1000 bytes initially in 1 step? Any thoughts on this would be
>
>> greatly appreciated!
>
> Because that's the way TCP/IP works, by design. TCP is a stream
> protocol. It guarantees all of the bytes written to one end of the pipe
> will come out the other end in the same order, but not necessarily in
> the same groupings. There are a number of buffers along the way that
> might split them up, as well as limits on packet sizes in the various
> networks it passed through. So you get what is available in the last
> buffer when a timer expires, no more, and no less.
>
> If you have serialized data that needs to be grouped in specific blocks,
> your application will need to keep track of those blocks, reassembling
> or splitting the streamed data as necessary. You could use UDP which
> does guarantee that packets will be kept together, but that protocol
> doesn't guarantee delivery.

I'm not sure this has much to do with the OP's problem, but this part is
backwards. TCP is connection oriented and tracks segments by sequence
number for each connection. This enables the stack to pass the data in
order to the higher layers. UDP is connectionless and has no way to
determine what datagram was sent before the other one, so it is up to
the higher layers to reassemble. As for IP in general, if packets need
to be fragmented along the way by a router in order to fit the MTU of a
different network, then the IP stack on the receiving end will
reassemble the fragments based upon information that the router injects
into the fragments.

--
Thanks!
-Shawn
http://www.spidean.com

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

RE: Sockets (reading)

From: Shawn McKenzie
> Bob McConnell wrote:
>> From: Philip Thompson
>>> During a socket read, why would all the requested number of bytes
not
>>=20
>>> get sent? For example, I request 1000 bytes:
>>>
>>> >>> $data =3D @socket_read ($socket, 2048, PHP_BINARY_READ);
>>> ?>
>>>
>>> This is actually in a loop, so I can get all the data if split up.
So,
>>=20
>>> for example, here's how the data split up in 3 iterations (for 1000

>>> bytes):
>>>
>>> 650 bytes
>>> 200 bytes
>>> 150 bytes
>>>
>>> But if I can accept up to 2048 bytes per socket read, why would it
not
>>=20
>>> pull all 1000 bytes initially in 1 step? Any thoughts on this would
be
>>=20
>>> greatly appreciated!
>>=20
>> Because that's the way TCP/IP works, by design. TCP is a stream
>> protocol. It guarantees all of the bytes written to one end of the
pipe
>> will come out the other end in the same order, but not necessarily in
>> the same groupings. There are a number of buffers along the way that
>> might split them up, as well as limits on packet sizes in the various
>> networks it passed through. So you get what is available in the last
>> buffer when a timer expires, no more, and no less.
>>=20
>> If you have serialized data that needs to be grouped in specific
blocks,
>> your application will need to keep track of those blocks,
reassembling
>> or splitting the streamed data as necessary. You could use UDP which
>> does guarantee that packets will be kept together, but that protocol
>> doesn't guarantee delivery.
>=20
> I'm not sure this has much to do with the OP's problem, but this part
is
> backwards. TCP is connection oriented and tracks segments by sequence
> number for each connection. This enables the stack to pass the data
in
> order to the higher layers. UDP is connectionless and has no way to
> determine what datagram was sent before the other one, so it is up to
> the higher layers to reassemble. As for IP in general, if packets
need
> to be fragmented along the way by a router in order to fit the MTU of
a
> different network, then the IP stack on the receiving end will
> reassemble the fragments based upon information that the router
injects
> into the fragments.

Shawn,

You're looking at it inside out. Yes, the individual packets are tracked
by the stack, to make sure they arrive in the correct order. But the
size and fragmentation of those packets have no relationship at all to
any data structure the application layer may imply. They simply
implement a communications stream to reliably move octets from one point
to another. If the application needs structure, it has to manage that
for itself.

For UDP, if you write a 32 byte packet, the matching read will get a 32
byte packet, if it arrived at the receiving stack. Missed data detection
and retry requests are left up to the application.

Bob McConnell

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

RE: Sockets (reading)

From: Philip Thompson
> On Aug 26, 2009, at 11:56 AM, Bob McConnell wrote:
>> From: Philip Thompson
>>>
>>> During a socket read, why would all the requested number of bytes
not
>>> get sent? For example, I request 1000 bytes:
>>>
>>> >>> $data =3D @socket_read ($socket, 2048, PHP_BINARY_READ);
>>> ?>
>>>
>>> This is actually in a loop, so I can get all the data if split up. =20
>>> So,
>>> for example, here's how the data split up in 3 iterations (for 1000
>>> bytes):
>>>
>>> 650 bytes
>>> 200 bytes
>>> 150 bytes
>>>
>>> But if I can accept up to 2048 bytes per socket read, why would it =20
>>> not
>>> pull all 1000 bytes initially in 1 step? Any thoughts on this would

>>> be
>>> greatly appreciated!
>>
>> Because that's the way TCP/IP works, by design. TCP is a stream
>> protocol. It guarantees all of the bytes written to one end of the =20
>> pipe
>> will come out the other end in the same order, but not necessarily in
>> the same groupings. There are a number of buffers along the way that
>> might split them up, as well as limits on packet sizes in the various
>> networks it passed through. So you get what is available in the last
>> buffer when a timer expires, no more, and no less.
>>
>> If you have serialized data that needs to be grouped in specific =20
>> blocks,
>> your application will need to keep track of those blocks,
reassembling
>> or splitting the streamed data as necessary. You could use UDP which
>> does guarantee that packets will be kept together, but that protocol
>> doesn't guarantee delivery.
>=20
> Thank you for your input.
>=20
> Is it guaranteed that at least 1 byte will be sent each time? For =20
> example, if I know the data length...
>=20
> > $input =3D '';
>=20
> for ($i=3D0; $i<$dataLength; $i++) {
> // Read 1 byte at a time
> if (($data =3D @socket_read ($socket, 1, PHP_BINARY_READ)) =
!== =20
> false) {
> $input .=3D $data;
> }
> }
>=20
> return $input;
> ?>
>=20
> Or is this a completely unreasonable and unnecessary way to get the =20
> data?

While I have written a lot of code to manage sockets over the years, and
coded a UDP/IP stack, I have never done it in PHP. And unfortunately, I
don't have time to experiment right now. My boss is waiting for the next
product release from me.

Getting one byte at a time is somewhat wasteful, as it requires more
system calls than necessary. That's a lot of wasted overhead.

Whether you always get one or more bytes depends on a number of factors,
including whether the calls PHP uses are blocking or non-blocking, plus
there may be ways to switch the socket back and forth.

Have you tried doing a Google search on the group of PHP functions you
expect to use. That should come up with some sample code to look at.

Bob McConnell

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Sockets (reading)

On Aug 26, 2009, at 2:47 PM, Bob McConnell wrote:

> From: Philip Thompson
>> On Aug 26, 2009, at 11:56 AM, Bob McConnell wrote:
>>> From: Philip Thompson
>>>>
>>>> During a socket read, why would all the requested number of bytes
> not
>>>> get sent? For example, I request 1000 bytes:
>>>>
>>>> >>>> $data = @socket_read ($socket, 2048, PHP_BINARY_READ);
>>>> ?>
>>>>
>>>> This is actually in a loop, so I can get all the data if split up.
>>>> So,
>>>> for example, here's how the data split up in 3 iterations (for 1000
>>>> bytes):
>>>>
>>>> 650 bytes
>>>> 200 bytes
>>>> 150 bytes
>>>>
>>>> But if I can accept up to 2048 bytes per socket read, why would it
>>>> not
>>>> pull all 1000 bytes initially in 1 step? Any thoughts on this would
>
>>>> be
>>>> greatly appreciated!
>>>
>>> Because that's the way TCP/IP works, by design. TCP is a stream
>>> protocol. It guarantees all of the bytes written to one end of the
>>> pipe
>>> will come out the other end in the same order, but not necessarily
>>> in
>>> the same groupings. There are a number of buffers along the way that
>>> might split them up, as well as limits on packet sizes in the
>>> various
>>> networks it passed through. So you get what is available in the last
>>> buffer when a timer expires, no more, and no less.
>>>
>>> If you have serialized data that needs to be grouped in specific
>>> blocks,
>>> your application will need to keep track of those blocks,
> reassembling
>>> or splitting the streamed data as necessary. You could use UDP which
>>> does guarantee that packets will be kept together, but that protocol
>>> doesn't guarantee delivery.
>>
>> Thank you for your input.
>>
>> Is it guaranteed that at least 1 byte will be sent each time? For
>> example, if I know the data length...
>>
>> >> $input = '';
>>
>> for ($i=0; $i<$dataLength; $i++) {
>> // Read 1 byte at a time
>> if (($data = @socket_read ($socket, 1, PHP_BINARY_READ)) !==
>> false) {
>> $input .= $data;
>> }
>> }
>>
>> return $input;
>> ?>
>>
>> Or is this a completely unreasonable and unnecessary way to get the
>> data?
>
> While I have written a lot of code to manage sockets over the years,
> and
> coded a UDP/IP stack, I have never done it in PHP. And
> unfortunately, I
> don't have time to experiment right now. My boss is waiting for the
> next
> product release from me.
>
> Getting one byte at a time is somewhat wasteful, as it requires more
> system calls than necessary. That's a lot of wasted overhead.
>
> Whether you always get one or more bytes depends on a number of
> factors,
> including whether the calls PHP uses are blocking or non-blocking,
> plus
> there may be ways to switch the socket back and forth.
>
> Have you tried doing a Google search on the group of PHP functions you
> expect to use. That should come up with some sample code to look at.
>
> Bob McConnell

I agree that one byte at a time is wasteful. I'm sure others haven't
implemented something along these lines, but I wrote a much more
efficient way to make sure I grab all the data of a known length...

function readSocketForDataLength ($socket, $len)
{
$offset = 0;
$socketData = '';

while ($offset < $len) {
if (($data = @socket_read ($socket, $len - $offset,
PHP_BINARY_READ)) === false) {
return false;
}

$offset += strlen ($data);
$socketData .= $data;
}

return $socketData;
}
?>

If not all the data is obtained on a read, it will loop until the
amount of data is the same as the length requested. This is working
quite well.

Thanks Bob and the others!

~Philip

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Sockets (reading)

--0016e646085633ac3004721f7bb8
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On Wed, Aug 26, 2009 at 5:36 PM, Philip Thompson wrote:

> On Aug 26, 2009, at 2:47 PM, Bob McConnell wrote:
>
> From: Philip Thompson
>>
>>> On Aug 26, 2009, at 11:56 AM, Bob McConnell wrote:
>>>
>>>> From: Philip Thompson
>>>>
>>>>>
>>>>> During a socket read, why would all the requested number of bytes
>>>>>
>>>> not
>>
>>> get sent? For example, I request 1000 bytes:
>>>>>
>>>>> >>>>> $data = @socket_read ($socket, 2048, PHP_BINARY_READ);
>>>>> ?>
>>>>>
>>>>> This is actually in a loop, so I can get all the data if split up.
>>>>> So,
>>>>> for example, here's how the data split up in 3 iterations (for 1000
>>>>> bytes):
>>>>>
>>>>> 650 bytes
>>>>> 200 bytes
>>>>> 150 bytes
>>>>>
>>>>> But if I can accept up to 2048 bytes per socket read, why would it
>>>>> not
>>>>> pull all 1000 bytes initially in 1 step? Any thoughts on this would
>>>>>
>>>>
>> be
>>>>> greatly appreciated!
>>>>>
>>>>
>>>> Because that's the way TCP/IP works, by design. TCP is a stream
>>>> protocol. It guarantees all of the bytes written to one end of the
>>>> pipe
>>>> will come out the other end in the same order, but not necessarily in
>>>> the same groupings. There are a number of buffers along the way that
>>>> might split them up, as well as limits on packet sizes in the various
>>>> networks it passed through. So you get what is available in the last
>>>> buffer when a timer expires, no more, and no less.
>>>>
>>>> If you have serialized data that needs to be grouped in specific
>>>> blocks,
>>>> your application will need to keep track of those blocks,
>>>>
>>> reassembling
>>
>>> or splitting the streamed data as necessary. You could use UDP which
>>>> does guarantee that packets will be kept together, but that protocol
>>>> doesn't guarantee delivery.
>>>>
>>>
>>> Thank you for your input.
>>>
>>> Is it guaranteed that at least 1 byte will be sent each time? For
>>> example, if I know the data length...
>>>
>>> >>> $input = '';
>>>
>>> for ($i=0; $i<$dataLength; $i++) {
>>> // Read 1 byte at a time
>>> if (($data = @socket_read ($socket, 1, PHP_BINARY_READ)) !==
>>> false) {
>>> $input .= $data;
>>> }
>>> }
>>>
>>> return $input;
>>> ?>
>>>
>>> Or is this a completely unreasonable and unnecessary way to get the
>>> data?
>>>
>>
>> While I have written a lot of code to manage sockets over the years, and
>> coded a UDP/IP stack, I have never done it in PHP. And unfortunately, I
>> don't have time to experiment right now. My boss is waiting for the next
>> product release from me.
>>
>> Getting one byte at a time is somewhat wasteful, as it requires more
>> system calls than necessary. That's a lot of wasted overhead.
>>
>> Whether you always get one or more bytes depends on a number of factors,
>> including whether the calls PHP uses are blocking or non-blocking, plus
>> there may be ways to switch the socket back and forth.
>>
>> Have you tried doing a Google search on the group of PHP functions you
>> expect to use. That should come up with some sample code to look at.
>>
>> Bob McConnell
>>
>
> I agree that one byte at a time is wasteful. I'm sure others haven't
> implemented something along these lines, but I wrote a much more efficient
> way to make sure I grab all the data of a known length...
>
> > function readSocketForDataLength ($socket, $len)
> {
> $offset = 0;
> $socketData = '';
>
> while ($offset < $len) {
> if (($data = @socket_read ($socket, $len - $offset,
> PHP_BINARY_READ)) === false) {
> return false;
> }
>
> $offset += strlen ($data);
> $socketData .= $data;
> }
>
> return $socketData;
> }
> ?>
>
> If not all the data is obtained on a read, it will loop until the amount of
> data is the same as the length requested. This is working quite well.
>
> Thanks Bob and the others!
>
> ~Philip
>
>
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>
socket related:
Does this solution work on both, blocking and non-blocking sockets ?
And what about different read method?

solution related:
Does strlen works fine with binary data?
Does this snippet work for sending/receiving multibytes strings?

--
Martin Scotta

--0016e646085633ac3004721f7bb8--