Set TCP_MAXSEG (OS buffering) on Win32? How?

For various reasons I have a requirement to have an app listening on
localhost which is proxying output from another app that is not under my
control. Currently the OS is buffering too much data (I think >XP the OS
buffers >64KB ish?) and I need to try and find a way to crank down the
OS buffers without making a system wide registry change

The solution appeared to be to use the TCP_MAXSEG option:
setsockopt($server, &Socket::IPPROTO_TCP, &Socket::TCP_MAXSEG, 2048);

However, on activestate at least this is giving me:
Your vendor has not defined Socket macro TCP_MAXSEG, used at xxx

Any ideas on how I can control OS buffering on Win32? (I need it
portable eventually, but win32 is my problem right now)

I believe that the trick on windows is to set the advertised MSS window
(as above) and this then changes the window size that the sender uses

Right now I can use SO_RCVBUF on the receiving side to turn down the
receive buffer, but I am still having problems with the sending buffer
(which is outside my control) being way too large for my requirements

Any help please?

Ed W

Re: Set TCP_MAXSEG (OS buffering) on Win32? How?

Ed W schrieb:
> For various reasons I have a requirement to have an app listening on
> localhost which is proxying output from another app that is not under my
> control. Currently the OS is buffering too much data (I think >XP the OS
> buffers >64KB ish?) and I need to try and find a way to crank down the
> OS buffers without making a system wide registry change
>
> The solution appeared to be to use the TCP_MAXSEG option:
> setsockopt($server, &Socket::IPPROTO_TCP, &Socket::TCP_MAXSEG, 2048);
>
> However, on activestate at least this is giving me:
> Your vendor has not defined Socket macro TCP_MAXSEG, used at xxx
>
> Any ideas on how I can control OS buffering on Win32? (I need it
> portable eventually, but win32 is my problem right now)
>
> I believe that the trick on windows is to set the advertised MSS window
> (as above) and this then changes the window size that the sender uses
>
> Right now I can use SO_RCVBUF on the receiving side to turn down the
> receive buffer, but I am still having problems with the sending buffer
> (which is outside my control) being way too large for my requirements

TCP segment size has _nothing at all_ to do with stream buffer sizes,
those are two limits that apply to different layers of a tcp
communication.

TCP_MAXSEG controls how big a tcp packet can be before it gets split
into consecutive segments. Trying to toy with that from within an
application is dangerous and may lead to problems that are very
hard to track down. The task of determining efficient segment
sizes is best left to the communication stack itself.

The send/receive buffering happens at a higher level. In the case of
a send buffer, it simply contains the raw data that is to be broken
into tcp segments and sent over the wire. In the case of a receive
buffer, it contains the raw data _assembled_ from one or more
consecutive tcp segments.

I can't rule out for sure that there is a (recommendable) mechanism to
change SO_SNDBUF individually for a foreign application's socket on
Win32, but I'd be surprised if there was. Even then, it could be that
the application implements its own buffering before invoking send(), and
any modification efforts would be for naught.

I'm wondering what problem you are trying to solve that appears as
"Currently the OS is buffering too much data". Your question,
however, has nothing to do with Perl and will problably get better
answers in a Win32 related newsgroup.

-Chris

Re: Set TCP_MAXSEG (OS buffering) on Win32? How?

> TCP segment size has _nothing at all_ to do with stream buffer sizes,
> those are two limits that apply to different layers of a tcp
> communication.

Actually on Win32 it appears that MSS controls the advertised receive
window. Forget the reference now, but some googling will turn it up I
think?


> TCP_MAXSEG controls how big a tcp packet can be before it gets split
> into consecutive segments.

Err? Really? I don't think it is actually? I think it's the MSS window?

Trying to toy with that from within an
> application is dangerous and may lead to problems that are very
> hard to track down. The task of determining efficient segment
> sizes is best left to the communication stack itself.

This is all over localhost!! The problem is that the operating system
is trying to open up the window to be efficient and I need to close it
down because I'm buffering to talk over a pipe which speaks 20KB per
minute!! The standard window is about a 4 *minute* buffer and this
causes problems with the apps we are trying to proxy for.

It's a specialist case for sure...

> The send/receive buffering happens at a higher level. In the case of
> a send buffer, it simply contains the raw data that is to be broken
> into tcp segments and sent over the wire. In the case of a receive
> buffer, it contains the raw data _assembled_ from one or more
> consecutive tcp segments.

I'm pretty sure that you are confusing SO_SNDBUF and SO_RCVBUF, plus the
packet size, with the receive window which is tracking the amount of
data in flight. I want to clamp down the send/receive window only - the
size of each packet is not a priority

> I can't rule out for sure that there is a (recommendable) mechanism to
> change SO_SNDBUF individually for a foreign application's socket on
> Win32,

Apparently it's to change TCP_MAXSEG, which in turn changes the
advertised receive window which affects the send window on the transmit end.

So full circle - why am I getting an error trying to set TCP_MAXSEG?


> I'm wondering what problem you are trying to solve that appears as
> "Currently the OS is buffering too much data".

I am writing a network app on a network which transmits at slightly less
then 20KB per min. This requires quite a bit of control over the data
being sent, packet size and buffer sizes. Most of this is inhand, the
problem I am having is controlling the advertised receive window on new
connections - if there is a better way to set this via perl then I am
looking for a solution - right now I believe that on win32 at least that
the TCP_MAXSEG is the setting which controls this? Problem I am having
is that current ActiveState is reporting that it's "not implemented" -
is there a workaround or some other solution?


Your question,
> however, has nothing to do with Perl

?? My app is written in perl and it's unable to set standard TCP socket
settings?! How is this NOT a perl question?

I guess I could fix this if I drop into some XS and run some direct
calls over the socket, but I am keen to get this sorted with minimum
extra modules.

Grateful for any thoughts?

Ed W

P.S. Update see http://support.microsoft.com/kb/224829/EN-US/ for a
confirmation that advertised MSS controls the transmit window under
win32 - just need to find out how to change it per connection without
screwing down the whole machine for this single application...

Re: Set TCP_MAXSEG (OS buffering) on Win32? How?

Bah, bother

I can't find a bunch of articles now which suggest that TCP_MAXSEG is a
roundabout route to accidently controlling the advertised receive
window. However, I am sure I have seen something about this

The goal anyway is to control the advertised receive window in the hope
that this will indirectly affect the actual transmit window at the
remote end (this is all on localhost so it's quite controllable).

Some tests here suggest that SO_RCVBUF actually controls the transmit
buffer to some extent on Vista, but on XP it's not happening...

Any other suggestions? I don't control the sending app and I need to
throttle it back to around <20KB per min and right now the OS buffers
are allowing it to send WAAAAY more data than is reasonable...

Cheers

Ed W

Re: Set TCP_MAXSEG (OS buffering) on Win32? How?

Quoth Ed W :
> For various reasons I have a requirement to have an app listening on
> localhost which is proxying output from another app that is not under my
> control. Currently the OS is buffering too much data (I think >XP the OS
> buffers >64KB ish?) and I need to try and find a way to crank down the
> OS buffers without making a system wide registry change
>
> The solution appeared to be to use the TCP_MAXSEG option:
> setsockopt($server, &Socket::IPPROTO_TCP, &Socket::TCP_MAXSEG, 2048);

Don't call subs with &. Not even constants.

> However, on activestate at least this is giving me:
> Your vendor has not defined Socket macro TCP_MAXSEG, used at xxx

So Socket doesn't know about TCP_MAXSEG. One way around is to find out
what its value should be (from MSDN) and specify it directly. Another
(better, but probably *much* harder on Win32) is to rebuild perl in such
a way that it can find out for itself, since Socket does know about
TCP_MAXSEG on some platforms.

Ben

Re: Set TCP_MAXSEG (OS buffering) on Win32? How?

> So Socket doesn't know about TCP_MAXSEG.

Googling around some more suggests that it's actually a read only param
on windows. I was obviously wrong about that being the backdoor to try
and control the receive window

OK, so to rephrase the question: anyone know how to influence advertised
receive window on XP+ on the localhost interface? Don't really want to
change the registry...

Cheers

Ed W

Re: Set TCP_MAXSEG (OS buffering) on Win32? How?

Ed W wrote:
> Bah, bother
>
> I can't find a bunch of articles now which suggest that TCP_MAXSEG is a
> roundabout route to accidently controlling the advertised receive
> window. However, I am sure I have seen something about this

There has been some confusion over that topic for a long
time, and I bet that a lot of people are glad now that
less misleading information shows up in google nowadays.

> The goal anyway is to control the advertised receive window in the hope
> that this will indirectly affect the actual transmit window at the
> remote end (this is all on localhost so it's quite controllable).

That would by no means be a documented and reliable behaviour
and prone to change with different OS patchlevels without
prior warning. Don't go there, it will give you quite a bit of
a headache in the future.

> Some tests here suggest that SO_RCVBUF actually controls the transmit
> buffer to some extent on Vista, but on XP it's not happening...

As the WinSock reference states, modifying SO_RCVBUF on an
established socket is only allowed on Windows Vista/Longhorn
and above (see the documentation for Winsock IOCTLs, namely
SIO_SET_COMPATIBILITY_MODE). This can work together with
tcp window autotuning, but if you want to distribute your
application to foreign systems, then you have to be aware that
many companies disable autotuning on Vista because it doesn't
work well with other IP stack implementations (especially
some firewall/vpn devices or older unices).

> Any other suggestions? I don't control the sending app and I need to
> throttle it back to around <20KB per min and right now the OS buffers
> are allowing it to send WAAAAY more data than is reasonable...

What are the symptoms of "allowing it to send WAAAAY more data..."?
Does the sending app fail with a winsock error? Does the receiving
part run out of (application) buffer? Do you get lags between
data packets? Each of those would mean that a different approach
is neccessary.

If you're getting a socket error from the downstream app, I'm not
sure if there is a clean way to control that by modfifying the
receive socket.

Would simply receiving all the data asynchronuously and filtering
out some parts of it to be passed on via the slower link not be an
option?

-Chris

Re: Set TCP_MAXSEG (OS buffering) on Win32? How?

"Ed W" wrote in message
news:iMednUXr2rEHDqnaRVnyhAA@pipex.net...
>
>
>> So Socket doesn't know about TCP_MAXSEG.
>
> Googling around some more suggests that it's actually a read only param on
> windows. I was obviously wrong about that being the backdoor to try and
> control the receive window
>
> OK, so to rephrase the question: anyone know how to influence advertised
> receive window on XP+ on the localhost interface? Don't really want to
> change the registry...
>
> Cheers
>
> Ed W

You need to do some research on the TCP protocol. You are confusing many
different things together and it is clear that you have no clue how TCP
works. I don't think you really know what you need to do because if your
app is not behaving as you expect it on localhost, changing buffers or MSS
sizes is not going to fix your problem. Since you haven't described the
problem you are trying to solve, I can only guess that you are having issues
with data written on the socket is not being sent out in a timely fashion.
You are guessing (wrongly) that this is a buffer problem, it is not. If you
are writing to a TCP socket and the data needs to be sent out immediately,
you need to set the TCP_NODELAY option.