mod_perl Output Filters and mod_deflate

mod_perl Output Filters and mod_deflate

am 06.08.2009 00:31:06 von James Smith

Has anyone had experience of using mod_perl OutputFilters with
mod_deflate, I've been banging my head against a brick wall today....
I've learnt a lot about bucket brigades - but for every two steps
forward it's one step back...

Scenario:

static page - being wrapped with an output filter - works with or
without mod_deflate

dynamic page - either Perl or PHP, without mod_deflate the output of
the page gets re-processed and the correct HTML is produced, but with
mod_deflate the content returned is the unwrapped output (i.e. the
output before the output filter has been actioned?)

any ideas or ways to resolve this...

below is my code....

package Sanger::Web::DecoratePage;

use strict;
use warnings;
no warnings qw(uninitialized);

use base qw(Apache2::Filter);

use Apache2::RequestRec ();

use APR::Table ();
use APR::Bucket ();
use APR::Brigade ();
use Apache2::Connection ();

use Apache2::Const -compile => qw(OK DECLINED CONN_KEEPALIVE);
use APR::Const -compile => ':common';

use constant LENGTH => 2048;
sub decorate_page {
my $content = shift;
my $title = $content =~ /(.*)<\/title>/s ? $1 : "untitled<br /> document";<br /> my $body = $content =~ /<body>(.*)<\/body>/s ? $1 : $content;<br /> return qq(<html><br /> <head><br /> <title>Wellcome Trust Sanger Institute: $title

Wellcome Trust Sanger Institute:



$body

Footer content....


);
}

sub handler : FilterRequestHandler {
my ($filter, $bb) = @_;
my $bb_ctx = APR::Brigade->new($filter->c->pool,
$filter->c->bucket_alloc);

my $t = $filter->r->headers_in();
return Apache2::Const::DECLINED unless $filter->r->status == 200;
warn "\n";
warn "\n";
warn ".. handler [[\n";
foreach my $key (keys %$t) {
warn sprintf " %40s = %s\n", $key, $t->{$key};
}
warn "]]\n";
my $ctx = context( $filter );
# pass through unmodified
return Apache2::Const::DECLINED if $ctx->{state};
my $data = exists $ctx->{data} ? $ctx->{data} : '';

$ctx->{invoked}++;
my( $bdata, $seen_eos, $beos ) = flatten_bb($bb);

$data .= $bdata if $bdata;
if ($seen_eos) {
$data = decorate_page( $data );
my $len = length $data;
$filter->r->headers_out->set('Content-Length', $len);
$filter->r->headers_out->set('Filter-Actioned',
'Sanger::Web::DecoratePage' );
$filter->r->content_type( "text/html" );

if( $data ) {
while( $data ) {
my $x = substr($data,0,LENGTH);
substr($data,0,LENGTH) = '';
warn sprintf "inserting bucket .... %4d :
%10s...%10s",length($x),substr( $x,0,10),substr($x,-10,10);
my $b = APR::Bucket->new( $bb->bucket_alloc, " $x" );
$bb_ctx->insert_tail($b);
}
}
$bb_ctx->insert_tail( $beos );

warn "
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
$data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (length $len)
\n" if 0;
$ctx->{state}++
} else {
warn "... store ...";
# store context for all but the last invocation
$ctx->{data} = $data;
$filter->ctx($ctx);
}
$t = $filter->r->headers_out();
warn "\n\n.. [[\n";
foreach my $key (keys %$t) {
warn sprintf " %40s = %s\n", $key, $t->{$key};
}
warn "]]\n";
warn "\n";
warn "end of handler....\n";
warn "\n";
my $rv = $filter->next->pass_brigade($bb_ctx);
return $rv unless $rv == APR::Const::SUCCESS;

return Apache2::Const::OK;
}

sub flatten_bb {
my $bb = shift;
my $seen_eos = 0;
my @data;
my $b = $bb->first;
while( $b ) {
$b->remove;
if($b->is_eos) {
$seen_eos++;
last
}
$b->read(my $bdata);
push @data, $bdata;
$b = $bb->next( $b );
}
return (join('', @data), $seen_eos, $b );
}

sub context {
my ($f) = shift;
my $ctx = $f->ctx;
warn "... $ctx ...";
warn $f->c->keepalive,"; ",Apache2::Const::CONN_KEEPALIVE,";
",$f->c->keepalives;
unless ($ctx) {
$ctx = {
state => 0,
keepalives => $f->c->keepalives,
};
use Data::Dumper;
warn Data::Dumper::Dumper( $ctx );
$f->ctx($ctx);
return $ctx;
}
my $c = $f->c;
if(
$c->keepalive == Apache2::Const::CONN_KEEPALIVE &&
$ctx->{state} &&
$c->keepalives > $ctx->{keepalives}
) {
$ctx->{state} = 0;
$ctx->{keepalives} = $c->keepalives;
}
return $ctx;
}

1;


--
The Wellcome Trust Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.

Re: mod_perl Output Filters and mod_deflate

am 06.08.2009 11:14:39 von aw

... and sorry again for sending directly to you. I keep forgetting this
list doesn't set this automatically.

James Smith wrote:
> Has anyone had experience of using mod_perl OutputFilters with
> mod_deflate, I've been banging my head against a brick wall today....
> I've learnt a lot about bucket brigades - but for every two steps
> forward it's one step back...
>
> Scenario:
>
> static page - being wrapped with an output filter - works with or
> without mod_deflate
>
> dynamic page - either Perl or PHP, without mod_deflate the output of
> the page gets re-processed and the correct HTML is produced, but with
> mod_deflate the content returned is the unwrapped output (i.e. the
> output before the output filter has been actioned?)
>
> any ideas or ways to resolve this...
>
> below is my code....
>
> package Sanger::Web::DecoratePage;
>
...
Sorry, I did not go through your code.
I assume however that this code has some way of detecting whether it
should do something to this output or not.

If mod_deflate sees the output first however, it will compress it.
Your filter, if it comes second, may not recognise this compressed
output anymore, and thus do nothing but let it through. Same for PHP.

Alternatively, mod_deflate is being a bad boy, and disables any filters
that may come after it. In a way this would be justified, because it
really needs to be at the end of the chain.

Unfortunately, I am rich in guesses but poor in solutions.

Re: mod_perl Output Filters and mod_deflate

am 06.08.2009 14:39:44 von James Smith

André Warnier wrote:
> .. and sorry again for sending directly to you. I keep forgetting
> this list doesn't set this automatically.
>
> James Smith wrote:
>> Has anyone had experience of using mod_perl OutputFilters with
>> mod_deflate, I've been banging my head against a brick wall today....
>> I've learnt a lot about bucket brigades - but for every two steps
>> forward it's one step back...
>>
>> Scenario:
>>
>> static page - being wrapped with an output filter - works with or
>> without mod_deflate
>>
>> dynamic page - either Perl or PHP, without mod_deflate the output of
>> the page gets re-processed and the correct HTML is produced, but with
>> mod_deflate the content returned is the unwrapped output (i.e. the
>> output before the output filter has been actioned?)
>>
>> any ideas or ways to resolve this...
>>
>> below is my code....
>>
>> package Sanger::Web::DecoratePage;
>>
> ..
> Sorry, I did not go through your code.
> I assume however that this code has some way of detecting whether it
> should do something to this output or not.
>
> If mod_deflate sees the output first however, it will compress it.
> Your filter, if it comes second, may not recognise this compressed
> output anymore, and thus do nothing but let it through. Same for PHP.
>
> Alternatively, mod_deflate is being a bad boy, and disables any filters
> that may come after it. In a way this would be justified, because it
> really needs to be at the end of the chain.
>
> Unfortunately, I am rich in guesses but poor in solutions.
>
>

According to the documentation - mod_deflate should be being run after
my filter as mod_deflate is a connection filter where mine is request
filter!

My filter is being run - and getting the right input content - observed as I was dumping the HTML coming in and going out!




--
The Wellcome Trust Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.

Re: mod_perl Output Filters and mod_deflate

am 06.08.2009 15:14:05 von aw

James Smith wrote:
>
> According to the documentation - mod_deflate should be being run after
> my filter as mod_deflate is a connection filter where mine is request
> filter!
>
> My filter is being run - and getting the right input content - observed
> as I was dumping the HTML coming in and going out!
>
Allright. I still don't feel like diving in your code, but for the sake
of someone else maybe :
- when mod_deflate is not in the picture, the output is the result of
the original output, modified by your filter, uncompressed.
- when mod_deflate is configured, then the output is compressed, but it
appears to be the unmodified original output, as if your filter was no
longer in the picture.
- however, in both cases, you can see that your filter runs and modifies
the original output data.

So it almost looks as if both your filter and mod_deflate get hold of
the original output data, and whatever your filter produces as output
just disappears into perl heaven..

Although I'm not still diving into your code, I seem to see somewhere in
it that it is also adding a response header. Does that response header
also disappear, or is it still in the mod_deflate output ?