input filters & mod_proxy

[Sorry if this is an FAQ, but I haven't seen it mentioned in either the
proxy status doc or the Apache 2.0 status doc.]

I'm writing a module that utilizes both input and output filters and
needs to work in conjunction with mod_proxy. More specifically, my
module runs in the context of a 2.0 server that is being used only as a
proxy (using both ProxyPass and ProxyRequests). Until recently, both
filters have been read-only, but now I'm starting to modify the request,
initially just to add a header in the input filter.

When I tested this, I found that the header I was adding was not making
it through to the destination. Stepping through, I see that
ap_proxy_http_request() (in proxy_http.c) is getting called *before* my
input filter. Since ap_proxy_http_request() performs the actual request
sending, my call to apr_table_set(r->headers_in, ...) from within my
input filter is effectively ignored. Based on this, I'm assuming that
I'll have the same problem when I start modifying the request body.

I'm currently working around this problem by hacking into proxy_http.c
to call back into my code to do the r->headers_in manipulation before
the proxy sends the request out, but obviously I don't want to do it
that way.

So, two questions. 1: Is this a known issues? 2: If so, what's the
plan, if any, to fix it?

Thanks in advance.

- Dave

--
Dave Seidel, Founder
Mindreef, LLC

RE: input filters & mod_proxy

Dave,

I think I know what the problem is, but I have a few questions.

1) What type of input filter are you using?
2) When is it getting inserted?

The problem is how input filters have been designed. There are really
two types of input filters, those that operate on just the request body,
and those that operate on the body and the headers. The type is defined
by whether the filter is inserted for all requests, or by location.

The problem that it sounds like you are having, is that you are
filtering the body data, and trying to modify the headers, but by the
time the body has been read, the headers have most likely already been
sent to the back-end client. There are a couple of fixes for this.

1) Modify the headers for the incoming request in a different request
processing phase, such as the post_read_request.

2) Make sure that your filter is available during header processing.

I recommend option 1, because filtering headers is a PITA, and it sounds
like it would be overkill for your solution. If you want to modify a
header, it would work, but you are adding one, so you should be okay.

The final option is that people keep talking about adding a phase in the
proxy that would be specifically for adding a header to the back-end
connection, but that hasn't happened yet, and I am trying to solve your
problem without making you modify the code.

You shouldn't have a problem with the body data, because you are
modifying it as the proxy is reading it. If you try it and it doesn't
work, I would be more than happy to help you debug and fix the problem.

Ryan

----------------------------------------------
Ryan Bloom rbb@covalent.net
645 Howard St. rbb@apache.org
San Francisco, CA

> -----Original Message-----
> From: Dave Seidel [mailto:dave@mindreef.com]
> Sent: Thursday, March 28, 2002 7:57 AM
> To: Modproxy-Dev; apache-modules
> Subject: input filters & mod_proxy
>
> [Sorry if this is an FAQ, but I haven't seen it mentioned in either
the
> proxy status doc or the Apache 2.0 status doc.]
>
> I'm writing a module that utilizes both input and output filters and
> needs to work in conjunction with mod_proxy. More specifically, my
> module runs in the context of a 2.0 server that is being used only as
a
> proxy (using both ProxyPass and ProxyRequests). Until recently, both
> filters have been read-only, but now I'm starting to modify the
request,
> initially just to add a header in the input filter.
>
> When I tested this, I found that the header I was adding was not
making
> it through to the destination. Stepping through, I see that
> ap_proxy_http_request() (in proxy_http.c) is getting called *before*
my
> input filter. Since ap_proxy_http_request() performs the actual
request
> sending, my call to apr_table_set(r->headers_in, ...) from within my
> input filter is effectively ignored. Based on this, I'm assuming that
> I'll have the same problem when I start modifying the request body.
>
> I'm currently working around this problem by hacking into proxy_http.c
> to call back into my code to do the r->headers_in manipulation before
> the proxy sends the request out, but obviously I don't want to do it
> that way.
>
> So, two questions. 1: Is this a known issues? 2: If so, what's the
> plan, if any, to fix it?
>
> Thanks in advance.
>
> - Dave
>
> --
> Dave Seidel, Founder
> Mindreef, LLC
>
>
>

RE: input filters & mod_proxy

Ryan,

I'm using AP_FTYPE_NETWORK for my input filter, but I'm still not
completely clear on the properties of the different filter types. Your
solution 1 sounds fine to me at this point. I'm not ready yet to try
partying on the body, but when I do, I'll let you know if there are
problems. Thanks for the reply.

- Dave

-----Original Message-----
From: Ryan Bloom [mailto:rbb@covalent.net]
Sent: Thursday, March 28, 2002 11:19 AM
To: modproxy-dev@apache.org; dave@mindreef.com; 'apache-modules'
Subject: RE: input filters & mod_proxy

Dave,

I think I know what the problem is, but I have a few questions.

1) What type of input filter are you using?
2) When is it getting inserted?

The problem is how input filters have been designed. There are really
two types of input filters, those that operate on just the request body,
and those that operate on the body and the headers. The type is defined
by whether the filter is inserted for all requests, or by location.

The problem that it sounds like you are having, is that you are
filtering the body data, and trying to modify the headers, but by the
time the body has been read, the headers have most likely already been
sent to the back-end client. There are a couple of fixes for this.

1) Modify the headers for the incoming request in a different request
processing phase, such as the post_read_request.

2) Make sure that your filter is available during header processing.

I recommend option 1, because filtering headers is a PITA, and it sounds
like it would be overkill for your solution. If you want to modify a
header, it would work, but you are adding one, so you should be okay.

The final option is that people keep talking about adding a phase in the
proxy that would be specifically for adding a header to the back-end
connection, but that hasn't happened yet, and I am trying to solve your
problem without making you modify the code.

You shouldn't have a problem with the body data, because you are
modifying it as the proxy is reading it. If you try it and it doesn't
work, I would be more than happy to help you debug and fix the problem.

Ryan

----------------------------------------------
Ryan Bloom rbb@covalent.net
645 Howard St. rbb@apache.org
San Francisco, CA

> -----Original Message-----
> From: Dave Seidel [mailto:dave@mindreef.com]
> Sent: Thursday, March 28, 2002 7:57 AM
> To: Modproxy-Dev; apache-modules
> Subject: input filters & mod_proxy
>
> [Sorry if this is an FAQ, but I haven't seen it mentioned in either
the
> proxy status doc or the Apache 2.0 status doc.]
>
> I'm writing a module that utilizes both input and output filters and
> needs to work in conjunction with mod_proxy. More specifically, my
> module runs in the context of a 2.0 server that is being used only as
a
> proxy (using both ProxyPass and ProxyRequests). Until recently, both
> filters have been read-only, but now I'm starting to modify the
request,
> initially just to add a header in the input filter.
>
> When I tested this, I found that the header I was adding was not
making
> it through to the destination. Stepping through, I see that
> ap_proxy_http_request() (in proxy_http.c) is getting called *before*
my
> input filter. Since ap_proxy_http_request() performs the actual
request
> sending, my call to apr_table_set(r->headers_in, ...) from within my
> input filter is effectively ignored. Based on this, I'm assuming that
> I'll have the same problem when I start modifying the request body.
>
> I'm currently working around this problem by hacking into proxy_http.c
> to call back into my code to do the r->headers_in manipulation before
> the proxy sends the request out, but obviously I don't want to do it
> that way.
>
> So, two questions. 1: Is this a known issues? 2: If so, what's the
> plan, if any, to fix it?
>
> Thanks in advance.
>
> - Dave
>
> --
> Dave Seidel, Founder
> Mindreef, LLC
>
>
>