strange mod_deflate behaviour on binary output

Hi,

I noticed something interesting with mod_deflate when serving binary
content from a mason component:

<%init>
$m->clear_buffer();
$r->content_type("video/mp4");
$r->headers_out->add('Content-length' => -s $file);
$r->sendfile($file);
$m->abort(200);


mod_deflate is enabled with the default:

AddOutputFilterByType DEFLATE text/html text/plain text/xml

and .md mason components have:

AddType text/html .md

But when I change the content-type from within the component mod_deflate
doesn't seem to notice and keeps gzip'ing the video/mp4 output, which
FlowPlayer doesn't appreciate.

I certainly could disable mod_deflate and be on my merry way.

But I thought I'd ask the modperl list (already posted to mason-users)
whether this mod_deflate behaviour is expected and/or normal?

Thanks,

Re: strange mod_deflate behaviour on binary output

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig0B5775D4E6EB56752AD7AD2F
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 22/3/09 15:25, Louis-David Mitterrand wrote:
> Hi,
>=20
> I noticed something interesting with mod_deflate when serving binary
> content from a mason component:
>=20
> <%init>
> $m->clear_buffer();
> $r->content_type("video/mp4");
> $r->headers_out->add('Content-length' =3D> -s $file);
> $r->sendfile($file);
> $m->abort(200);
>
>=20
> mod_deflate is enabled with the default:
>=20
> AddOutputFilterByType DEFLATE text/html text/plain text/xml
>=20
> and .md mason components have:
>=20
> AddType text/html .md
>=20
> But when I change the content-type from within the component mod_deflat=
e
> doesn't seem to notice and keeps gzip'ing the video/mp4 output, which
> FlowPlayer doesn't appreciate.
>=20
> I certainly could disable mod_deflate and be on my merry way.=20
>=20
> But I thought I'd ask the modperl list (already posted to mason-users)
> whether this mod_deflate behaviour is expected and/or normal?

Sounds like Mason runs exclusively in the PerlResponse phase, at which
point, AddOutputFilterByType has already decided if it should apply the
filter or not, so it's too late to change that.

One easy way to disable mod_deflate from that point in time, from mason,
would be to do:

$r->subprocess_env('no-gzip' =3D> 1);

Since from mod_deflate.c:

/* some browsers might have problems, so set no-gzip
* (with browsermatch) for them
*/
if (apr_table_get(r->subprocess_env, "no-gzip")) {
ap_remove_output_filter(f);
return ap_pass_brigade(f->next, bb);
}

Give it a try, this is untested!

--=20
Philippe M. Chiasson GPG: F9BFE0C2480E7680 1AE53631CB32A107 88C3A5A5
http://gozer.ectoplasm.org/ m/gozer\@(apache|cpan|ectoplasm)\.org/


--------------enig0B5775D4E6EB56752AD7AD2F
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)

iD8DBQFJxuSlyzKhB4jDpaURAjPpAJ9jRbOH1CFVzCegpLtQKI35bKQBOACf U/Rq
Esk5YeEEWOusYbqTeMmyl9w=
=hXcE
-----END PGP SIGNATURE-----

--------------enig0B5775D4E6EB56752AD7AD2F--

Re: strange mod_deflate behaviour on binary output

Philippe M. Chiasson wrote:
> On 22/3/09 15:25, Louis-David Mitterrand wrote:
[...]
Hi.
There was a thread here some time in the past (late 2008 ?) between
Torsten Foetsch and myself, talking about overriding this Content-Type
header.
As I recall, this particular header is somewhat special, because the
response's content-type is part of a separate Apache structure related
to the request, which Apache uses when outputting the request.
The practical result is that even when you explicitly set the
content-type in your request-processing code (like you do in your
original post), Apache will later on overwrite this header again, using
information from that other structure.
That's what gives the impression that your original code does not work.
It does, but what it sets gets overwritten later on.

The solution is to use an output filter (which runs after the request
processing has taken place). Sample working code below.
In my case, I basically wanted to always override the "charset"
attribute set by an application, but it should work in other cases. Be
sure to check carefully where you apply it though, since you proably do
not want to override all your Content-Type headers.

Configuration in httpd.conf :

PerlOutputFilterHandler AM::FixStarWebCharset



package AM::FixStarWebCharset;
# File : AM/FixStarWebCharset.pm
# Fixes servlet-generated Content-Type header
# Idea from Torsten Foetsch.

$AM::FixStarWebCharset::VERSION = '0.01';

use strict;
use warnings FATAL => 'all';

use mod_perl2 2.000001;

use base qw(Apache2::Filter);
use Apache2::Const ();

use constant DEBUG => 0;

sub handler : FilterRequestHandler {
my ($f, $bb) = @_;
my $r = $f->r;
$r->content_type("text/html; charset=ISO-8859-2");
# should we also remove ourselves ?
return Apache2::Const::DECLINED;
}
1;

Re: strange mod_deflate behaviour on binary output

On Sun, Mar 22, 2009 at 09:23:49PM -0400, Philippe M. Chiasson wrote:
> On 22/3/09 15:25, Louis-David Mitterrand wrote:
> > Hi,
> >
> > I noticed something interesting with mod_deflate when serving binary
> > content from a mason component:
> >
> > <%init>
> > $m->clear_buffer();
> > $r->content_type("video/mp4");
> > $r->headers_out->add('Content-length' => -s $file);
> > $r->sendfile($file);
> > $m->abort(200);
> >
> >
> > mod_deflate is enabled with the default:
> >
> > AddOutputFilterByType DEFLATE text/html text/plain text/xml
> >
> > and .md mason components have:
> >
> > AddType text/html .md
> >
> > But when I change the content-type from within the component mod_deflate
> > doesn't seem to notice and keeps gzip'ing the video/mp4 output, which
> > FlowPlayer doesn't appreciate.
> >
> > I certainly could disable mod_deflate and be on my merry way.
> >
> > But I thought I'd ask the modperl list (already posted to mason-users)
> > whether this mod_deflate behaviour is expected and/or normal?
>
> Sounds like Mason runs exclusively in the PerlResponse phase, at which
> point, AddOutputFilterByType has already decided if it should apply the
> filter or not, so it's too late to change that.
>
> One easy way to disable mod_deflate from that point in time, from mason,
> would be to do:
>
> $r->subprocess_env('no-gzip' => 1);
>
> Since from mod_deflate.c:
>
> /* some browsers might have problems, so set no-gzip
> * (with browsermatch) for them
> */
> if (apr_table_get(r->subprocess_env, "no-gzip")) {
> ap_remove_output_filter(f);
> return ap_pass_brigade(f->next, bb);
> }
>
> Give it a try, this is untested!

And this works fine! Thank you.

Another possible (less robust) solution is to add this to deflate.conf:

SetEnvIfNoCase Request_URI \.mp4$ no-gzip dont-vary

This only works if the $r->uri ends with .mp4 which in my case does
because I use an explicit $r->path_info:

/video.md/directory/file.mp4

"/directory/file.mp4" points to a directory hierarchy outside of the
ServerRoot.

Cheers,

--
http://www.critikart.net