generic question on input filters

Hi.

I want to write a mod_perl input filter, which essentially would replace
some characters from form-posted input on the fly, before letting the
request go to the real application.
The requests are POST, with content encoded in one long string, URL-encoded.
I am thinking of writing this as a FilterRequestHandler filter, grabbing
the content, splitting it on "&", URL-decoding the so-splitted query
parameters individually, then in the value performing the character
substitutions, then re-URL-encoding the query parameters, re-joining
them with "&", and essentially et voilà.

Basically, the type shown here :
http://perl.apache.org/docs/2.0/user/handlers/filters.html#t oc_Stream_oriented_Input_Filters

Is the above a realistic view, or am I missing something essential ?

Thanks for comments.
André

Re: generic question on input filters

On Wed 08 Apr 2009, André Warnier wrote:
> I want to write a mod_perl input filter, which essentially would
> replace some characters from form-posted input on the fly, before
> letting the request go to the real application.
> The requests are POST, with content encoded in one long string,
> URL-encoded. I am thinking of writing this as a FilterRequestHandler
> filter, grabbing the content, splitting it on "&", URL-decoding the
> so-splitted query parameters individually, then in the value
> performing the character substitutions, then re-URL-encoding the
> query parameters, re-joining them with "&", and essentially et voilà.
>
> Basically, the type shown here :
> http://perl.apache.org/docs/2.0/user/handlers/filters.html#t oc_Stream
>_oriented_Input_Filters
>
> Is the above a realistic view, or am I missing something essential ?

Remember the request can consist of multiple brigades. Each filter=20
invocation gets one brigade. That means you have to be prepared to=20
store some context (store the /[^&;]*$/ part in $f->ctx). Don't try to=20
read in the whole request. Operate on each part of it.

=46or example try the following PerlInputFilterHandler:

sub handler {
my ($f)=3D@_;

$f->print("\n###########\n");
my $buf;
while($f->read($buf, 1024)) {
$f->print("\n>>>$buf<<<\n");
undef $buf;
}
$f->print("\n:::::::::::\n");
return Apache2::Const::OK;
}

with this PerlResponseHandler:

sub handler {
my ($r)=3D@_;

$r->content_type('text/plain');
my $buf;
while($r->read($buf, 15000)) {
$r->print($buf);
undef $buf;
}

return Apache2::Const::OK;
}

and feed it with more than 8000 bytes:

perl -e 'print "x"x10000' | curl -qs -d @- http://localhost/iftest

You'll see multiple blocks wrapped in >>> ... <<< from the filter loop.=20
But you'll also see 2 blocks of ######### ... :::::::::: that mark the=20
filter invocations.

BTW, the buffer size in the response handler is that large because is=20
must be able to store a whole brigade's content due to a modperl bug,=20
see "XXX:" comment in modperl_request_read(). So, don't $f->print much=20
in one filter invocation.

Torsten

=2D-=20
Need professional mod_perl support?
Just hire me: torsten.foertsch@gmx.net

Re: generic question on input filters

Torsten Foertsch wrote:
[...]
Thanks for the previous answer and tips.
I have written and tried the filter, and now I have a problem (at the
end). Anyone knows what that means ?

Suspicion : I cannot read the content, and print nothing when I am
called. Instead of accumulating the content and process it all at once
at the end, should I process it chunk by chunk ?
Is it OK then when the Content-Length header is wrong (since I am
changing the size of the data) ?



main server error log:
[Wed Apr 08 22:42:37 2009] [notice] Apache/2.2.3 (Linux/SUSE)
mod_jk/1.2.28-dev-727724 mod_perl/2.0.2 Perl/v5.8.8 configured --
resuming normal operations

VirtualHost error log (logging by the filter, a few runs that seem to
work fine, then suddenly the error. Sorry for the clutter, that's the
html form sending stuff):

[Wed Apr 08 22:43:22 2009] [warn] [client 10.13.16.93] FSWB: 1st call,
referer: http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:43:22 2009] [warn] [client 10.13.16.93] FSWB: buffer:
[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&_ _hiddenstyle=A&__dirtyFlag=undefined&__action=216&Near=&Sele ctedIndexTerms=__05.12.1971%3B%3B%3A%3A&SelectedIndexTerms=_ _05.1970%3B%3B%3A%3A&SelectedIndexTerms=__06+SIERPNIA+2008+R .%3B%3B%3A%3A&SelectedIndexTerms=__06+SIERPNIA+2008+R.+-+SKR %D3T%3B%3B%3A%3A&SelectedIndexTerms=__06.02.1972%3B%3B%3A%3A &SelectedIndexTerms=__06.04.1969%3B%3B%3A%3A&SelectedIndexTe rms=__06.05.1968%3B%3B%3A%3A&SelectedIndexTerms=__06.06.1971 %3B%3B%3A%3A&SelectedIndexTerms=__06.10.1965%3B%3B%3A%3A&Sel ectedIndexTerms=__06.11.1970%3B%3B%3A%3A&SelectedIndexTerms= __06.12.1968+ROK%3B%3B%3A%3A&SelectedIndexTerms=__07+SIERPIE %D1+2008%3B%3B%3A%3A&SelectedIndexTerms=__07+SIERPNIA+2008+R .%3B%3B%3A%3A&SelectedIndexTerms=__07+ZG%A3O%3F+SI%CA%3B%3B% 3A%3A&SelectedIndexT
erms=__07+ZG%A3O%3F+SI%CA%3B%3B%3A%3A&__ReportId=histIndeksu %5E0],
referer: http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:43:22 2009] [warn] [client 10.13.16.93] FSWB: splitted
params: __sessionNumber=67 || __pageid=daneIndeksu || __a1= || __a2= ||
__a3= || __hiddenstyle=A || __dirtyFlag=undefined || __action=216 ||
Near= || SelectedIndexTerms=__05.12.1971%3B%3B%3A%3A ||
SelectedIndexTerms=__05.1970%3B%3B%3A%3A ||
SelectedIndexTerms=__06+SIERPNIA+2008+R.%3B%3B%3A%3A ||
SelectedIndexTerms=__06+SIERPNIA+2008+R.+-+SKR%D3T%3B%3B%3A% 3A ||
SelectedIndexTerms=__06.02.1972%3B%3B%3A%3A ||
SelectedIndexTerms=__06.04.1969%3B%3B%3A%3A ||
SelectedIndexTerms=__06.05.1968%3B%3B%3A%3A ||
SelectedIndexTerms=__06.06.1971%3B%3B%3A%3A ||
SelectedIndexTerms=__06.10.1965%3B%3B%3A%3A ||
SelectedIndexTerms=__06.11.1970%3B%3B%3A%3A ||
SelectedIndexTerms=__06.12.1968+ROK%3B%3B%3A%3A ||
SelectedIndexTerms=__07+SIERPIE%D1+2008%3B%3B%3A%3A ||
SelectedIndexTerms=__07+SIERPNIA+2008+R.%3B%3B%3A%3A ||
SelectedIndexTerms=__07+ZG%A3O%3F+SI%CA%3B%3B%3A%3A ||
SelectedIndexTerms=__07+ZG%A3O%3F+SI%CA%3B%3B%3A%3A ||
__ReportId=histIndeksu%5E0, referer:
http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:43:22 2009] [warn] [client 10.13.16.93] FSWB: new
content:
[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&_ _hiddenstyle=A&__dirtyFlag=undefined&__action=216&Near=&Sele ctedIndexTerms%3D__05.12.1971%3B%3B%3A%3A&SelectedIndexTerms %3D__05.1970%3B%3B%3A%3A&SelectedIndexTerms%3D__06%2BSIERPNI A%2B2008%2BR.%3B%3B%3A%3A&SelectedIndexTerms%3D__06%2BSIERPN IA%2B2008%2BR.%2B-%2BSKROT%3B%3B%3A%3A&SelectedIndexTerms%3D __06.02.1972%3B%3B%3A%3A&SelectedIndexTerms%3D__06.04.1969%3 B%3B%3A%3A&SelectedIndexTerms%3D__06.05.1968%3B%3B%3A%3A&Sel ectedIndexTerms%3D__06.06.1971%3B%3B%3A%3A&SelectedIndexTerm s%3D__06.10.1965%3B%3B%3A%3A&SelectedIndexTerms%3D__06.11.19 70%3B%3B%3A%3A&SelectedIndexTerms%3D__06.12.1968%2BROK%3B%3B %3A%3A&SelectedIndexTerms%3D__07%2BSIERPIEN%2B2008%3B%3B%3A% 3A&SelectedIndexTerms%3D__07%2BSIERPNIA%2B2008%2BR.%3B%3B%3A %3A&SelectedIndexTer
ms%3D__07%2BZGLO%3F%2BSIE%3B%3B%3A%3A&SelectedIndexTerms%3D_ _07%2BZGLO%3F%2BSIE%3B%3B%3A%3A&__ReportId%3DhistIndeksu%5E0 ],
referer: http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:45:35 2009] [warn] [client 10.13.16.93] FSWB: 1st call,
referer: http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:45:35 2009] [warn] [client 10.13.16.93] FSWB: buffer:
[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&_ _hiddenstyle=A&__dirtyFlag=undefined&__action=213&Near=ab&Se lectedIndexTerms=__07.01.1968%3B%3B%3A%3A&SelectedIndexTerms =__07.01.1973%3B%3B%3A%3A&SelectedIndexTerms=__07.02.1959+NR +9%3B%3B%3A%3A&SelectedIndexTerms=__07.02.1967%3B%3B%3A%3A&S electedIndexTerms=__07.03.1971%3B%3B%3A%3A&SelectedIndexTerm s=__07.04.1968%3B%3B%3A%3A&SelectedIndexTerms=__07.05.1972%3 B%3B%3A%3A&SelectedIndexTerms=__07.07.1968%3B%3B%3A%3A&Selec tedIndexTerms=__07.08.1972%3B%3B%3A%3A&SelectedIndexTerms=__ 07.09.1969%3B%3B%3A%3A&SelectedIndexTerms=__07.09.1970%3B%3B %3A%3A&SelectedIndexTerms=__07.11.1971%3B%3B%3A%3A&SelectedI ndexTerms=__08.03.1970%3B%3B%3A%3A&SelectedIndexTerms=__08.0 4.1962%3B%3B%3A%3A&SelectedIndexTerms=__08.04.1973%3B%3B%3A% 3A&__ReportId=histIn
deksu%5E0],
referer: http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:45:35 2009] [warn] [client 10.13.16.93] FSWB: splitted
params: __sessionNumber=67 || __pageid=daneIndeksu || __a1= || __a2= ||
__a3= || __hiddenstyle=A || __dirtyFlag=undefined || __action=213 ||
Near=ab || SelectedIndexTerms=__07.01.1968%3B%3B%3A%3A ||
SelectedIndexTerms=__07.01.1973%3B%3B%3A%3A ||
SelectedIndexTerms=__07.02.1959+NR+9%3B%3B%3A%3A ||
SelectedIndexTerms=__07.02.1967%3B%3B%3A%3A ||
SelectedIndexTerms=__07.03.1971%3B%3B%3A%3A ||
SelectedIndexTerms=__07.04.1968%3B%3B%3A%3A ||
SelectedIndexTerms=__07.05.1972%3B%3B%3A%3A ||
SelectedIndexTerms=__07.07.1968%3B%3B%3A%3A ||
SelectedIndexTerms=__07.08.1972%3B%3B%3A%3A ||
SelectedIndexTerms=__07.09.1969%3B%3B%3A%3A ||
SelectedIndexTerms=__07.09.1970%3B%3B%3A%3A ||
SelectedIndexTerms=__07.11.1971%3B%3B%3A%3A ||
SelectedIndexTerms=__08.03.1970%3B%3B%3A%3A ||
SelectedIndexTerms=__08.04.1962%3B%3B%3A%3A ||
SelectedIndexTerms=__08.04.1973%3B%3B%3A%3A ||
__ReportId=histIndeksu%5E0, referer:
http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:45:35 2009] [warn] [client 10.13.16.93] FSWB: new
content:
[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&_ _hiddenstyle=A&__dirtyFlag=undefined&__action=213&Near=ab&Se lectedIndexTerms%3D__07.01.1968%3B%3B%3A%3A&SelectedIndexTer ms%3D__07.01.1973%3B%3B%3A%3A&SelectedIndexTerms%3D__07.02.1 959%2BNR%2B9%3B%3B%3A%3A&SelectedIndexTerms%3D__07.02.1967%3 B%3B%3A%3A&SelectedIndexTerms%3D__07.03.1971%3B%3B%3A%3A&Sel ectedIndexTerms%3D__07.04.1968%3B%3B%3A%3A&SelectedIndexTerm s%3D__07.05.1972%3B%3B%3A%3A&SelectedIndexTerms%3D__07.07.19 68%3B%3B%3A%3A&SelectedIndexTerms%3D__07.08.1972%3B%3B%3A%3A &SelectedIndexTerms%3D__07.09.1969%3B%3B%3A%3A&SelectedIndex Terms%3D__07.09.1970%3B%3B%3A%3A&SelectedIndexTerms%3D__07.1 1.1971%3B%3B%3A%3A&SelectedIndexTerms%3D__08.03.1970%3B%3B%3 A%3A&SelectedIndexTerms%3D__08.04.1962%3B%3B%3A%3A&SelectedI ndexTerms%3D__08.04.
1973%3B%3B%3A%3A&__ReportId%3DhistIndeksu%5E0],
referer: http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:45:48 2009] [warn] [client 10.13.16.93] FSWB: 1st call,
referer: http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:45:48 2009] [warn] [client 10.13.16.93] FSWB: buffer:
[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&_ _hiddenstyle=A&__dirtyFlag=undefined&__action=216&Near=ab&Se lectedIndexTerms=__AB+OVO%3B%3B%3A%3A&SelectedIndexTerms=__A B+URBE+CONDITA%3B%3B%3A%3A&SelectedIndexTerms=__ABADDON%3B%3 B%3A%3A&SelectedIndexTerms=__ABAKANOWICZ+W+POZNANIU%3B%3B%3A %3A&SelectedIndexTerms=__ABAKANOWICZ+W+STARMACH+GALLERY%3B%3 B%3A%3A&SelectedIndexTerms=__ABAKANY%3B%3B%3A%3A&SelectedInd exTerms=__ABAKANY+CZY+STEREOFORMY%3B%3B%3A%3A&SelectedIndexT erms=__ABAKANY+W+KRAKOWIE+%7CA+00%3A14%3A31%3A15+%7CB+00%3A1 7%3A49%3A15+%7CC+3+%7CD+18%3B%3B%3A%3A&SelectedIndexTerms=__ ABANDON%3B%3B%3A%3A&SelectedIndexTerms=__ABBA%3B%3B%3A%3A&Se lectedIndexTerms=__ABBA%3B%3B%3A%3A&SelectedIndexTerms=__ABB A+-+MONEY%3B%3B%3A%3A&SelectedIndexTerms=__ABBA+EBAN+AT+UN%3 B%3B%3A%3A&SelectedI
ndexTerms=__ABBA+PATER%3B%3B%3A%3A&SelectedIndexTerms=__ABBA +W+STUDIO+2%3B%3B%3A%3A&__ReportId=histIndeksu%5E0],
referer: http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:45:48 2009] [warn] [client 10.13.16.93] FSWB: splitted
params: __sessionNumber=67 || __pageid=daneIndeksu || __a1= || __a2= ||
__a3= || __hiddenstyle=A || __dirtyFlag=undefined || __action=216 ||
Near=ab || SelectedIndexTerms=__AB+OVO%3B%3B%3A%3A ||
SelectedIndexTerms=__AB+URBE+CONDITA%3B%3B%3A%3A ||
SelectedIndexTerms=__ABADDON%3B%3B%3A%3A ||
SelectedIndexTerms=__ABAKANOWICZ+W+POZNANIU%3B%3B%3A%3A ||
SelectedIndexTerms=__ABAKANOWICZ+W+STARMACH+GALLERY%3B%3B%3A %3A ||
SelectedIndexTerms=__ABAKANY%3B%3B%3A%3A ||
SelectedIndexTerms=__ABAKANY+CZY+STEREOFORMY%3B%3B%3A%3A ||
SelectedIndexTerms=__ABAKANY+W+KRAKOWIE+%7CA+00%3A14%3A31%3A 15+%7CB+00%3A17%3A49%3A15+%7CC+3+%7CD+18%3B%3B%3A%3A
|| SelectedIndexTerms=__ABANDON%3B%3B%3A%3A ||
SelectedIndexTerms=__ABBA%3B%3B%3A%3A ||
SelectedIndexTerms=__ABBA%3B%3B%3A%3A ||
SelectedIndexTerms=__ABBA+-+MONEY%3B%3B%3A%3A ||
SelectedIndexTerms=__ABBA+EBAN+AT+UN%3B%3B%3A%3A ||
SelectedIndexTerms=__ABBA+PATER%3B%3B%3A%3A ||
SelectedIndexTerms=__ABBA+W+STUDIO+2%3B%3B%3A%3A ||
__ReportId=histIndeksu%5E0, referer:
http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:45:48 2009] [warn] [client 10.13.16.93] FSWB: new
content:
[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&_ _hiddenstyle=A&__dirtyFlag=undefined&__action=216&Near=ab&Se lectedIndexTerms%3D__AB%2BOVO%3B%3B%3A%3A&SelectedIndexTerms %3D__AB%2BURBE%2BCONDITA%3B%3B%3A%3A&SelectedIndexTerms%3D__ ABADDON%3B%3B%3A%3A&SelectedIndexTerms%3D__ABAKANOWICZ%2BW%2 BPOZNANIU%3B%3B%3A%3A&SelectedIndexTerms%3D__ABAKANOWICZ%2BW %2BSTARMACH%2BGALLERY%3B%3B%3A%3A&SelectedIndexTerms%3D__ABA KANY%3B%3B%3A%3A&SelectedIndexTerms%3D__ABAKANY%2BCZY%2BSTER EOFORMY%3B%3B%3A%3A&SelectedIndexTerms%3D__ABAKANY%2BW%2BKRA KOWIE%2B%7CA%2B00%3A14%3A31%3A15%2B%7CB%2B00%3A17%3A49%3A15% 2B%7CC%2B3%2B%7CD%2B18%3B%3B%3A%3A&SelectedIndexTerms%3D__AB ANDON%3B%3B%3A%3A&SelectedIndexTerms%3D__ABBA%3B%3B%3A%3A&Se lectedIndexTerms%3D__ABBA%3B%3B%3A%3A&SelectedIndexTerms%3D_ _ABBA%2B-%2BMONEY%3B
%3B%3A%3A&SelectedIndexTerms%3D__ABBA%2BEBAN%2BAT%2BUN%3B%3B %3A%3A&SelectedIndexTerms%3D__ABBA%2BPATER%3B%3B%3A%3A&Selec tedIndexTerms%3D__ABBA%2BW%2BSTUDIO%2B2%3B%3B%3A%3A&__Report Id%3Dhist
Indeksu%5E0],
referer: http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:46:22 2009] [warn] [client 10.13.16.93] FSWB: 1st call,
referer: http://host.customer.com/starweb/wideoteka/servlet.starweb
[Wed Apr 08 22:46:22 2009] [warn] [client 10.13.16.93] FSWB: buffer:
[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&_ _hiddenstyle=A&__dirtyFlag=undefined&__action=213&Near=ABBAD ON+-+ANIO%A3+ZAG%A3ADY&SelectedIndexTerms=__ABBA%3A+THE+MAMM A+MIA+STORY%3B%3B%3A%3A&SelectedIndexTerms=__ABBADO+IN+BERLI N+-+THE+FIRST+YEAR%3B%3B%3A%3A&SelectedIndexTerms=__ABBADON+ -+ANIO%A3+ZAG%A3ADY%3B%3B%3A%3A&SelectedIndexTerms=__ABBAS+K IAROSTANI%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONAST ERIES+%2F+ARMENIAN+MONASTERY+OF+SRPOTS+ISRAEL%3B%3B%3A%3A&Se lectedIndexTerms=__ABBEYS+%26+MONASTERIES+%2F+CARMELITE+MONK S+%26+NUNS%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONAS TERIES+%2F+COPTIC+MONASTERRIES+ANBA+BISHOY%3B%3B%3A%3A&Selec tedIndexTerms=__ABBEYS+%26+MONASTERIES+%2F+ETHIOPIAN-ORTODOX +JERUSALEM%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONAS TERIES+%2F+FRANCISKA
NES+-+CONVENTUELS+ASSISI%3B%3B%3A%3A&SelectedIndexTerms=__AB BEYS+%26+MONASTERIES+%2F+GREEK-ORTHODOX+MONASTERY+OF+ST.+JOH N%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONASTERIES+%2 F+GREEK-O
RTHODOX+OF+ZERBISTA%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+ %26+MONASTERIES+%2F+MONASTERIO+DI+CAMALDOLI+ITALY%3B%3B%3A%3 A&SelectedIndexTerms=__ABBEYS+%26+MONASTERIES+%2F+ORTHODOX+M ONASTERY+HOLY+TRINI],
referer: http://host.customer.com/starweb/wideoteka/servlet.starweb

main server error log :
[Wed Apr 08 22:46:22 2009] [crit] [Wed Apr 08 22:46:22 2009] file
http_filters.c, line 1324, assertion "!(((&(bb)->list))->next == (struct
apr_bucket *)((char *)((&(bb)->list)) - ((long) (((char *) (&(((struct
apr_bucket*)((void *)0))->link))) - ((char *) ((void *)0))))))" failed
[Wed Apr 08 22:46:22 2009] [notice] child pid 5518 exit signal Aborted (6)


Here is the filter code, non-essential parts omitted :

sub handler : FilterRequestHandler {
my $f = shift;
my $r = $f->r;
my $logger = $r->log;
my $ctx = $f->ctx;
unless (defined $ctx) {
# first invocation
$logger->warn("FSWB: 1st call") if DEBUG;
# remove the request Content-Length header (NOT!)
#$r->headers_in->unset('Content-Length');
$f->ctx({ 'post' => ''});
$ctx = $f->ctx;
}
my $post = $ctx->{'post'}; # empty first time around, but not later

while ($f->read(my $buffer, BUFLEN)) {
# read from brigade and accumulate
$logger->warn("FSWB: buffer: [$buffer]") if DEBUG;
$post .= $buffer;
}
# this brigade exhausted, but maybe there's more
unless ($f->seen_eos) {
# there is more then
$ctx->{'post'} = $post; # save accumulated data in context
return OK;
}
# have seen eos, so we can get to work
my $filtered = '';
my @qparams = split('&',$post);
$logger->warn("FSWB: splitted params: " . join(' || ',@qparams)) if DEBUG;
foreach (@qparams) {
# if the parameter does not contain any % sign at all, take a shortcut
unless ($_ =~ /%/) {
$filtered .= '&' . $_;
next;
}
my $unpar = uri_unescape($_);
#my @chars = split('',$cleanpar);
$unpar =~ s/([\x80-\xFF])/$CVTTBL->{$1} || $1/ge;
$filtered .= '&' . uri_escape($unpar);
}
$filtered =~ s/^\&//;
$logger->warn("FSWB: new content: [$filtered]") if DEBUG;
# re-add a Content-size header ?
my $size = length($filtered);
$r->headers_in->set('Content-Length',"$size");
# and forward content
$f->print($filtered);

return OK;
}

Re: generic question on input filters

On Thu 09 Apr 2009, André Warnier wrote:
> Suspicion : I cannot read the content, and print nothing when I am
> called.  Instead of accumulating the content and process it all at
> once at the end, should I process it chunk by chunk ?

yes, as I said in my previous mail, do it chunkwise.

> Is it OK then when the Content-Length header is wrong (since I am
> changing the size of the data) ?

As long as your handler does not rely on it you can ignore the=20
Content-Length header. In fact, a request can also be sumbitted=20
in "Transfer-Encoding: chunked" which means there is no CL header.

The end of request body is determined by the HTTP_IN filter=20
(ap_http_filter() in httpd/modules/http/http_filters.c) which is a=20
protocol filter. That means it comes before your filter.

Torsten

=2D-=20
Need professional mod_perl support?
Just hire me: torsten.foertsch@gmx.net

Re: generic question on input filters

Torsten Foertsch wrote:
> On Thu 09 Apr 2009, André Warnier wrote:
>> Suspicion : I cannot read the content, and print nothing when I am
>> called. Instead of accumulating the content and process it all at
>> once at the end, should I process it chunk by chunk ?
>
> yes, as I said in my previous mail, do it chunkwise.
>
Ok, I will change my filter to do it chunk by chunk.
It is quite a pain though, as I have to look for '&' boundaries between
query parameters, to make sure I don't cut in the middle of a %-escaped
sequence.

I have the impression that I am getting this Apache error :

[Wed Apr 08 22:46:22 2009] [crit] [Wed Apr 08 22:46:22 2009] file
http_filters.c, line 1324, assertion "!(((&(bb)->list))->next == (struct
apr_bucket *)((char *)((&(bb)->list)) - ((long) (((char *) (&(((struct
apr_bucket*)((void *)0))->link))) - ((char *) ((void *)0))))))" failed
[Wed Apr 08 22:46:22 2009] [notice] child pid 5518 exit signal Aborted (6)

when I exceed some size in what I save with $f->ctx($data).
(say with $data around 1100 bytes ?).
Is that so ?

Re: generic question on input filters

On Thu 09 Apr 2009, André Warnier wrote:
> I have the impression that I am getting this Apache error :
>
> [Wed Apr 08 22:46:22 2009] [crit] [Wed Apr 08 22:46:22 2009] file
> http_filters.c, line 1324, assertion "!(((&(bb)->list))->next ==
> (struct apr_bucket *)((char *)((&(bb)->list)) - ((long) (((char *)
> (&(((struct apr_bucket*)((void *)0))->link))) - ((char *) ((void
> *)0))))))" failed [Wed Apr 08 22:46:22 2009] [notice] child pid 5518
> exit signal Aborted (6)

Perhaps it's easier to implement a brigade based filter since this comes
from here:

rv =3D ap_get_brigade(r->input_filters, bb, AP_MODE_READBYTES,
APR_BLOCK_READ, bufsiz);

..

/* If this fails, it means that a filter is written incorrectly and that
* it needs to learn how to properly handle APR_BLOCK_READ requests by
* returning data when requested.
*/
AP_DEBUG_ASSERT(!APR_BRIGADE_EMPTY(bb));

Meaning your filter hasn't written anything.

Torsten

=2D-=20
Need professional mod_perl support?
Just hire me: torsten.foertsch@gmx.net

Re: generic question on input filters

Torsten Foertsch wrote:
> On Thu 09 Apr 2009, André Warnier wrote:
>> I have the impression that I am getting this Apache error :
>>
>> [Wed Apr 08 22:46:22 2009] [crit] [Wed Apr 08 22:46:22 2009] file
>> http_filters.c, line 1324, assertion "!(((&(bb)->list))->next ==
>> (struct apr_bucket *)((char *)((&(bb)->list)) - ((long) (((char *)
>> (&(((struct apr_bucket*)((void *)0))->link))) - ((char *) ((void
>> *)0))))))" failed [Wed Apr 08 22:46:22 2009] [notice] child pid 5518
>> exit signal Aborted (6)
>
> Perhaps it's easier to implement a brigade based filter since this comes
> from here:
>
> rv = ap_get_brigade(r->input_filters, bb, AP_MODE_READBYTES,
> APR_BLOCK_READ, bufsiz);
>
> ...
>
> /* If this fails, it means that a filter is written incorrectly and that
> * it needs to learn how to properly handle APR_BLOCK_READ requests by
> * returning data when requested.
> */
> AP_DEBUG_ASSERT(!APR_BRIGADE_EMPTY(bb));
>
> Meaning your filter hasn't written anything.
>
Hmmm.
So I cannot return "nothing" at any filter invocation.

Is there some hack allowing me to return "something" that will satisfy
Apache, without introducing input data I don't want ?
(sending like a "null string" brigade, but brigade anyway ?).
Can I for instance do a $f->print('') ?

On the input side, I get a URI-escaped query string, like
key1=FOO%D3D+I+SALUT+ARMATNI%3B%3B%3A%3A&key2=BAR%D3D+I+SALU T+ARMATNI%3B%3B%3A%3A
(but much longer).
I want to unescape this data, replace some of the characters, then
re-escape the data and forward it to the application.
Presumably, the input could be split over 2 bucket brigades, the first
one ending at some "%" or "%3", leaving one trailing escape sequence
incomplete.
I was trying to avoid that, by only processing "complete" query
parameters "key=value" (which is also nicer for logging), but that of
course can lead to the case where I have a brigade like
&key1=FOO%D3D+I+SALUT+ARMATNI%3
(with a yet-incomplete parameter, but nothing before it that I can process).
In that case, I was saving the string for later in $f->ctx, and
forwarding nothing this time around.

Looks like I'll have to drop the idea of processing complete query
parameters, and switch to processing complete escape sequences instead.
Or else, do a print of some dummy parameter which I know the application
will ignore. But that is a really application-specific hack.