RE: Siteminder/mod_proxy issues

We are beginning a Apache 1.3.26 reverse-proxy setup with SiteMinder. I have
not seen the double Set-Cookie strangeness. We are using SiteMinder 4.61
with the QMR4 apache webagent. I've observed some strange URL rewriting
issues involved with multi-domain sign-on and using cookie providers, but
nothing that can't be worked around.

Our setup is basically apache reverse-proxies (mod_proxy) behind F5
load-balancers. The reverse-proxies chain through intermediate firewalls
through another (forward) mod_proxy to backend DMZ servers.

I have been tracing HTTP headers and have not yet seen the behavior you
describe. Are you running SiteMinder 5.0?

-----Original Message-----
From: Martijn Schoemaker [mailto:martijn@osp.nl]
Sent: Wednesday, August 28, 2002 11:39 AM
To: modproxy-dev@apache.org
Subject: Siteminder/mod_proxy issues

Hi all,

I have the strangest results in an environment we use
at a customer.

We have an apache-server with mod_proxy on one node,
which forwards it's requests to another apache-server
with mod_proxy AND mod_sm from siteminder Netegrity.

Both apache servers are 1.3.26, pached with my previously
posted 'full reverse proxy' path, a siteminder patch (which
cannot be applied because the change seems to be in
standard 1.3.26 anyway) and a patch which removes spurious
line-feeds/carriage returns in the headers.

The strange thing is that some redirects from the application
(behind the 2nd proxy) work fine, others work only on IE6
browsers, others crash with a 'DNS error', and all netscapes
seem to display the redirect page without actually performing
the redirect.

The only strange thing I see in the sniffer logs (the redirects
are fine, no additional cr/lf's, no other strange things) is
that the siteminder cookies are set twice.

I afiak the duplication happens on the second proxy but I
cannot imagine why. Either there is a problem with the
mod_sm making it add the cookies and not replace if they
exist, or the mod_proxy duplicates the Set-Cookie headers.

I am completely baffled by this issue, and I don't have much
hair left on my head ;)

My question to y'all is :

1. Has anybody got this same config (using siteminder) ?
2. Does anybody know why the netscape browser does not
perform the redirect, except when I press 'reload' ?
3. Has anybody got a hint as to why Set-Cookie headers
might be duplicated ?

Thanks in advance,
Martijn Schoemaker

P.S.: Since this infrastucture is SSL on the front, and since
browsers nowadays don't support 'view response as-is'
anymore (let alone seeing the headers :() I do not have
the actual document 'seen' by the browser.

--
You have reached the end of the message.
Press [t] to go to the top of this message, or [c] to close it.

Re: Siteminder/mod_proxy issues

Hi,

We use SM 4.51 and not yet the QMR4 web-agent. Will install
and try this out right away. In any case, this does not seem
to be a mod_proxy problem anyway. I did some more checking
and the browser problems are probably caused by the Set-Cookie
headers which are set multiple times. Also, the Cookies them-
selves for the user that works are smaller that the ones for
the user that don't work and this prolly gives strange effects
in IE (who whould have guessed ? :))

Anyway, this seems more like a SM/Cookie/RFC issue and has no
further relation with mod_proxy.

Thanks all who replied for the input, and if insights change
y'all will hear from me :)

Greetings,
Martijn Schoemaker

P.S.: Does anyone know a browser-like test tool which handles
SSL and shows the actual data including headers ? I might
even build one myself, since debugging these issues is
now quite a pain in the *ss. I know there is an SSL ca-
pable wget, but it's pretty irritating with cookies etc.

"Foust, Adam G." wrote:

> We are beginning a Apache 1.3.26 reverse-proxy setup with SiteMinder. I have
> not seen the double Set-Cookie strangeness. We are using SiteMinder 4.61
> with the QMR4 apache webagent. I've observed some strange URL rewriting
> issues involved with multi-domain sign-on and using cookie providers, but
> nothing that can't be worked around.
>
> Our setup is basically apache reverse-proxies (mod_proxy) behind F5
> load-balancers. The reverse-proxies chain through intermediate firewalls
> through another (forward) mod_proxy to backend DMZ servers.
>
> I have been tracing HTTP headers and have not yet seen the behavior you
> describe. Are you running SiteMinder 5.0?
>

--
You have reached the end of the message.
Press [t] to go to the top of this message, or [c] to close it.

Re: Siteminder/mod_proxy issues

Be aware that it *is* possible to get SM to produce a cookie sooooo big
that it hits the 1024 (or wathever) single MIME line length limit of
apache. See

LimitRequestLine (http_core.c)
LimitRequestFieldsize (http_core.c)

and friends for more info.

Dw.

On Thu, 29 Aug 2002, Martijn Schoemaker wrote:

> Hi,
>
> We use SM 4.51 and not yet the QMR4 web-agent. Will install
> and try this out right away. In any case, this does not seem
> to be a mod_proxy problem anyway. I did some more checking
> and the browser problems are probably caused by the Set-Cookie
> headers which are set multiple times. Also, the Cookies them-
> selves for the user that works are smaller that the ones for
> the user that don't work and this prolly gives strange effects
> in IE (who whould have guessed ? :))
>
> Anyway, this seems more like a SM/Cookie/RFC issue and has no
> further relation with mod_proxy.
>
> Thanks all who replied for the input, and if insights change
> y'all will hear from me :)
>
> Greetings,
> Martijn Schoemaker
>
> P.S.: Does anyone know a browser-like test tool which handles
> SSL and shows the actual data including headers ? I might
> even build one myself, since debugging these issues is
> now quite a pain in the *ss. I know there is an SSL ca-
> pable wget, but it's pretty irritating with cookies etc.
>
> "Foust, Adam G." wrote:
>
> > We are beginning a Apache 1.3.26 reverse-proxy setup with SiteMinder. I have
> > not seen the double Set-Cookie strangeness. We are using SiteMinder 4.61
> > with the QMR4 apache webagent. I've observed some strange URL rewriting
> > issues involved with multi-domain sign-on and using cookie providers, but
> > nothing that can't be worked around.
> >
> > Our setup is basically apache reverse-proxies (mod_proxy) behind F5
> > load-balancers. The reverse-proxies chain through intermediate firewalls
> > through another (forward) mod_proxy to backend DMZ servers.
> >
> > I have been tracing HTTP headers and have not yet seen the behavior you
> > describe. Are you running SiteMinder 5.0?
> >
>
> --
> You have reached the end of the message.
> Press [t] to go to the top of this message, or [c] to close it.
>
>
>
>

Re: Siteminder/mod_proxy issues

--------------EC62B07B679E6871BD1493CB
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi,

Checked them and limits are set to 8Kb and looking at the header
sizes they don't exceed that.

I did some other testing however and I don't think we are going
to like this...

I put back an old libproxy.so from our (working) version shipped
with apache 1.3.12 into apach 1.3.26. And when I sniff the whole
kit'n kaboodle again I don't see the cookie duplication! If I put
back the version of 1.3.26, I DO see cookie duplication!

Does anybody know what might cause this ? I know siteminder depends
on a patch which inserts an : ap_overlay_tables() statement which
is already in the version of 1.3.26. Has something been changed in
relation to passing headers to other modules ? Might this statement
now causing headers to be added twice ?

Cheers,
Martijn Schoemaker

Dirk-Willem van Gulik wrote:

> Be aware that it *is* possible to get SM to produce a cookie sooooo big
> that it hits the 1024 (or wathever) single MIME line length limit of
> apache. See
>
> LimitRequestLine (http_core.c)
> LimitRequestFieldsize (http_core.c)
>
> and friends for more info.
>
> Dw.
>
> On Thu, 29 Aug 2002, Martijn Schoemaker wrote:
>
> > Hi,
> >
> > We use SM 4.51 and not yet the QMR4 web-agent. Will install
> > and try this out right away. In any case, this does not seem
> > to be a mod_proxy problem anyway. I did some more checking
> > and the browser problems are probably caused by the Set-Cookie
> > headers which are set multiple times. Also, the Cookies them-
> > selves for the user that works are smaller that the ones for
> > the user that don't work and this prolly gives strange effects
> > in IE (who whould have guessed ? :))
> >
> > Anyway, this seems more like a SM/Cookie/RFC issue and has no
> > further relation with mod_proxy.
> >
> > Thanks all who replied for the input, and if insights change
> > y'all will hear from me :)
> >
> > Greetings,
> > Martijn Schoemaker
> >
> > P.S.: Does anyone know a browser-like test tool which handles
> > SSL and shows the actual data including headers ? I might
> > even build one myself, since debugging these issues is
> > now quite a pain in the *ss. I know there is an SSL ca-
> > pable wget, but it's pretty irritating with cookies etc.
> >
> > "Foust, Adam G." wrote:
> >
> > > We are beginning a Apache 1.3.26 reverse-proxy setup with SiteMinder. I have
> > > not seen the double Set-Cookie strangeness. We are using SiteMinder 4.61
> > > with the QMR4 apache webagent. I've observed some strange URL rewriting
> > > issues involved with multi-domain sign-on and using cookie providers, but
> > > nothing that can't be worked around.
> > >
> > > Our setup is basically apache reverse-proxies (mod_proxy) behind F5
> > > load-balancers. The reverse-proxies chain through intermediate firewalls
> > > through another (forward) mod_proxy to backend DMZ servers.
> > >
> > > I have been tracing HTTP headers and have not yet seen the behavior you
> > > describe. Are you running SiteMinder 5.0?
> > >
> >
> > --
> > You have reached the end of the message.
> > Press [t] to go to the top of this message, or [c] to close it.
> >
> >
> >
> >

--
You have reached the end of the message.
Press [t] to go to the top of this message, or [c] to close it.



--------------EC62B07B679E6871BD1493CB
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit



Hi,

Checked them and limits are set to 8Kb and looking at the header

sizes they don't exceed that.

I did some other testing however and I don't think we are going

to like this...

I put back an old libproxy.so from our (working) version shipped

with apache 1.3.12 into apach 1.3.26. And when I sniff the whole

kit'n kaboodle again I don't see the cookie duplication! If I
put

back the version of 1.3.26, I DO see cookie duplication!

Does anybody know what might cause this ? I know siteminder depends

on a patch which inserts an : ap_overlay_tables() statement which

is already in the version of 1.3.26. Has something been changed in

relation to passing headers to other modules ? Might this statement

now causing headers to be added twice ?

Cheers,

Martijn Schoemaker

Dirk-Willem van Gulik wrote:

Be aware that it *is* possible to get SM to produce
a cookie sooooo big

that it hits the 1024 (or wathever) single MIME line length limit of

apache. See

        LimitRequestLine (http_core.c)

        LimitRequestFieldsize (http_core.c)

and friends for more info.

Dw.

On Thu, 29 Aug 2002, Martijn Schoemaker wrote:

> Hi,

>

> We use SM 4.51 and not yet the QMR4 web-agent. Will install

> and try this out right away. In any case, this does not seem

> to be a mod_proxy problem anyway. I did some more checking

> and the browser problems are probably caused by the Set-Cookie

> headers which are set multiple times. Also, the Cookies them-

> selves for the user that works are smaller that the ones for

> the user that don't work and this prolly gives strange effects

> in IE (who whould have guessed ? :))

>

> Anyway, this seems more like a SM/Cookie/RFC issue and has no

> further relation with mod_proxy.

>

> Thanks all who replied for the input, and if insights change

> y'all will hear from me :)

>

> Greetings,

> Martijn Schoemaker

>

> P.S.: Does anyone know a browser-like test tool which handles

>       SSL and shows the actual data
including headers ? I might

>       even build one myself, since
debugging these issues is

>       now quite a pain in the *ss.
I know there is an SSL ca-

>       pable wget, but it's pretty irritating
with cookies etc.

>

> "Foust, Adam G." wrote:

>

> > We are beginning a Apache 1.3.26 reverse-proxy setup with SiteMinder.
I have

> > not seen the double Set-Cookie strangeness. We are using SiteMinder
4.61

> > with the QMR4 apache webagent. I've observed some strange URL rewriting

> > issues involved with multi-domain sign-on and using cookie providers,
but

> > nothing that can't be worked around.

> >

> > Our setup is basically apache reverse-proxies (mod_proxy) behind
F5

> > load-balancers. The reverse-proxies chain through intermediate
firewalls

> > through another (forward) mod_proxy to backend DMZ servers.

> >

> > I have been tracing HTTP headers and have not yet seen the behavior
you

> > describe. Are you running SiteMinder 5.0?

> >

>

> --

> You have reached the end of the message.

> Press [t] to go to the top of this message, or [c] to close it.

>

>

>

>

--

You have reached the end of the message. 

Press [t] to go to the top of this message, or [c] to close it.

Re: Siteminder/mod_proxy issues

You may want to move this to dev@httpd.apache.org - this sure sounds like
something in the core being the issue.

DW.

On Fri, 30 Aug 2002, Martijn Schoemaker wrote:

> Hi,
>
> Checked them and limits are set to 8Kb and looking at the header
> sizes they don't exceed that.
>
> I did some other testing however and I don't think we are going
> to like this...
>
> I put back an old libproxy.so from our (working) version shipped
> with apache 1.3.12 into apach 1.3.26. And when I sniff the whole
> kit'n kaboodle again I don't see the cookie duplication! If I put
> back the version of 1.3.26, I DO see cookie duplication!
>
> Does anybody know what might cause this ? I know siteminder depends
> on a patch which inserts an : ap_overlay_tables() statement which
> is already in the version of 1.3.26. Has something been changed in
> relation to passing headers to other modules ? Might this statement
> now causing headers to be added twice ?
>
> Cheers,
> Martijn Schoemaker
>
> Dirk-Willem van Gulik wrote:
>
> > Be aware that it *is* possible to get SM to produce a cookie sooooo big
> > that it hits the 1024 (or wathever) single MIME line length limit of
> > apache. See
> >
> > LimitRequestLine (http_core.c)
> > LimitRequestFieldsize (http_core.c)
> >
> > and friends for more info.
> >
> > Dw.
> >
> > On Thu, 29 Aug 2002, Martijn Schoemaker wrote:
> >
> > > Hi,
> > >
> > > We use SM 4.51 and not yet the QMR4 web-agent. Will install
> > > and try this out right away. In any case, this does not seem
> > > to be a mod_proxy problem anyway. I did some more checking
> > > and the browser problems are probably caused by the Set-Cookie
> > > headers which are set multiple times. Also, the Cookies them-
> > > selves for the user that works are smaller that the ones for
> > > the user that don't work and this prolly gives strange effects
> > > in IE (who whould have guessed ? :))
> > >
> > > Anyway, this seems more like a SM/Cookie/RFC issue and has no
> > > further relation with mod_proxy.
> > >
> > > Thanks all who replied for the input, and if insights change
> > > y'all will hear from me :)
> > >
> > > Greetings,
> > > Martijn Schoemaker
> > >
> > > P.S.: Does anyone know a browser-like test tool which handles
> > > SSL and shows the actual data including headers ? I might
> > > even build one myself, since debugging these issues is
> > > now quite a pain in the *ss. I know there is an SSL ca-
> > > pable wget, but it's pretty irritating with cookies etc.
> > >
> > > "Foust, Adam G." wrote:
> > >
> > > > We are beginning a Apache 1.3.26 reverse-proxy setup with SiteMinder. I have
> > > > not seen the double Set-Cookie strangeness. We are using SiteMinder 4.61
> > > > with the QMR4 apache webagent. I've observed some strange URL rewriting
> > > > issues involved with multi-domain sign-on and using cookie providers, but
> > > > nothing that can't be worked around.
> > > >
> > > > Our setup is basically apache reverse-proxies (mod_proxy) behind F5
> > > > load-balancers. The reverse-proxies chain through intermediate firewalls
> > > > through another (forward) mod_proxy to backend DMZ servers.
> > > >
> > > > I have been tracing HTTP headers and have not yet seen the behavior you
> > > > describe. Are you running SiteMinder 5.0?
> > > >
> > >
> > > --
> > > You have reached the end of the message.
> > > Press [t] to go to the top of this message, or [c] to close it.
> > >
> > >
> > >
> > >
>
> --
> You have reached the end of the message.
> Press [t] to go to the top of this message, or [c] to close it.
>
>
>

Re: Siteminder/mod_proxy issues

--------------75B845151DFDC26A1D0A9FFC
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi,

Well, if it is, it's also a mod_proxy issue since the patch is
shipped with siteminder (smproxy.patch) and which now seems to
be in the mod_proxy standard distribution seems to cause all
headers set by the siteminder module to be added twice which
does not seem right anyway. This might even cause headers set
by all other modules to be set twice though I have not tested
this yet.

To avoid confusion, I have done all tests with the official
QMR4 version of the siteminder webagent, the one officialy released
for apache 1.3.26. So using that webagent does not solve the problem.

However, when I comment out the lines which should be added by the
smproxy.patch (but are now in the standard mod_proxy) all problems
seem to be solved.

The lines are :

proxy_http.c
+ /* Now add out bound headers set by other modules */
+ resp_hdrs = ap_overlay_tables(r->pool, r->err_headers_out, resp_hdrs);

Now, the reason I bumped into this might have to do with the
fact that the siteminder headers are huge, and duplicating them
overflows some boundary somewhere in some cases in some browsers
in some universes (but definitely ours).

So can we verify that 'we' (as in the mod_proxy developers)
are doing things wrong, or 'they' (as in the apache core developers)
are wrong ? ;)

Cheers,
Martijn Schoemaker


Dirk-Willem van Gulik wrote:

> You may want to move this to dev@httpd.apache.org - this sure sounds like
> something in the core being the issue.
>

--
You have reached the end of the message.
Press [t] to go to the top of this message, or [c] to close it.



--------------75B845151DFDC26A1D0A9FFC
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit



Hi,

Well, if it is, it's also a mod_proxy issue since the patch is

shipped with siteminder (smproxy.patch) and which now seems to

be in the mod_proxy standard distribution seems to cause all

headers set by the siteminder module to be added twice which

does not seem right anyway. This might even cause headers set

by all other modules to be set twice though I have not tested

this yet.

To avoid confusion, I have done all tests with the official

QMR4 version of the siteminder webagent, the one officialy released

for apache 1.3.26. So using that webagent does not solve the problem.

However, when I comment out the lines which should be added by the

smproxy.patch (but are now in the standard mod_proxy) all problems

seem to be solved.

The lines are :

proxy_http.c

+       /* Now add out bound headers
set by other modules */

+       resp_hdrs = ap_overlay_tables(r->pool,
r->err_headers_out, resp_hdrs);

Now, the reason I bumped into this might have to do with the

fact that the siteminder headers are huge, and duplicating them

overflows some boundary somewhere in some cases in some browsers

in some universes (but definitely ours).

So can we verify that 'we' (as in the mod_proxy developers)

are doing things wrong, or 'they' (as in the apache core developers)

are wrong ? ;)

Cheers,

Martijn Schoemaker

 

Dirk-Willem van Gulik wrote:

You may want to move this to dev@httpd.apache.org
- this sure sounds like

something in the core being the issue.

 

--

You have reached the end of the message. 

Press [t] to go to the top of this message, or [c] to close it.

Re: Siteminder/mod_proxy issues

Martijn Schoemaker wrote:

> I put back an old libproxy.so from our (working) version shipped
> with apache 1.3.12 into apach 1.3.26. And when I sniff the whole
> kit'n kaboodle again I don't see the cookie duplication! If I put
> back the version of 1.3.26, I DO see cookie duplication!

If you can sniff the connection both sides (turn off SSL for testing, or
use something like openssl s_client and speak HTTP directly to the
server) then we can see what is happening on either side.

Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm
"There's a moon
over Bourbon Street
tonight..."

Re: Siteminder/mod_proxy issues

--------------DDF63724F37CF970156668E2
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Ok guys, I think I found the problem.

This is really a mod_proxy issue since the sending of headers
changed with the http 1.1 patch which was created by Graham.

In the 'old' version (1.3.12) the mod_proxy uses ap_table_do()
to send out all headers directly. Since the siteminder headers
were in err_headers_out, they needed to be concatenated to the
output response headers otherwise they would get lost. For this
there was a patch, provided by siteminder, which added an
ap_overlay_tables() to concatenate the err_headers to the response
headers which were sent directly. This patch was also incor-
porated in the mod_proxy source. (dunno from which version)

Now, in the new mod_proxy, ap_table_do() is no longer used to
send out the headers, but ap_send_http_headers() is used. And
in this method it does another concatenate of the err_headers
to the output response headers. Since the siteminder headers
were still in here, they were concatenated twice. Once by the
previously added ap_overlay_tables() (originally put in to
do what ap_send_http_headers() already does already) and once
in the ap_send_http_headers().

Now I think this is enough proof to say that the 'old' manual
ap_overlay_tables() line (539 in my proxy_http.c file) can be
(ehm MUST be) removed from the code since is now causes dup-
lication of headers which again triggers a lot of other pro-
blems ....

Do y'all concur in this one ?

Cheers,
Martijn Schoemaker

--
You have reached the end of the message.
Press [t] to go to the top of this message, or [c] to close it.



--------------DDF63724F37CF970156668E2
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit



Ok guys, I think I found the problem.

This is really a mod_proxy issue since the sending of headers

changed with the http 1.1 patch which was created by Graham.

In the 'old' version (1.3.12) the mod_proxy uses ap_table_do()

to send out all headers directly. Since the siteminder headers

were in err_headers_out, they needed to be concatenated to the

output response headers otherwise they would get lost. For this

there was a patch, provided by siteminder, which added an

ap_overlay_tables() to concatenate the err_headers to the response

headers which were sent directly. This patch was also incor-

porated in the mod_proxy source. (dunno from which version)

Now, in the new mod_proxy, ap_table_do() is no longer used to

send out the headers, but ap_send_http_headers() is used. And

in this method it does another concatenate of the err_headers

to the output response headers. Since the siteminder headers

were still in here, they were concatenated twice. Once by the

previously added ap_overlay_tables() (originally put in to

do what ap_send_http_headers() already does already) and once

in the ap_send_http_headers().

Now I think this is enough proof to say that the 'old' manual

ap_overlay_tables() line (539 in my proxy_http.c file) can be

(ehm MUST be) removed from the code since is now causes dup-

lication of headers which again triggers a lot of other pro-

blems ....

Do y'all concur in this one ?

Cheers,

Martijn Schoemaker

--

You have reached the end of the message. 

Press [t] to go to the top of this message, or [c] to close it.

Re: Siteminder/mod_proxy issues

Martijn Schoemaker wrote:

> Ok guys, I think I found the problem.

> Now I think this is enough proof to say that the 'old' manual
> ap_overlay_tables() line (539 in my proxy_http.c file) can be
> (ehm MUST be) removed from the code since is now causes dup-
> lication of headers which again triggers a lot of other pro-
> blems ....
>
> Do y'all concur in this one ?

Good catch... I scratched my head for ages trying to figure out what
gives on this one.

Will commit a fix...

Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm
"There's a moon
over Bourbon Street
tonight..."