Capturing Apache response

Capturing Apache response

am 08.02.2009 22:59:24 von Solutio at Gmail

This is a multi-part message in MIME format.

------=_NextPart_000_00D8_01C98A06.367C4340
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

We are trying to implement a solution that would allow us to capture =
Apache (v. 2.0.63) responses sent to clients, each in a separate file on =
the server. I added a bucket brigade based output filter that mimics an =
example found in =
http://perl.apache.org/docs/2.0/user/handlers/filters.html#B ucket_Brigade=
_based_Output_Filters, as follows:

____

package httpd::FilterCaptureResponse;

use strict;
use warnings FATAL =3D> 'all';

use base qw(Apache2::Filter);

use Apache2::RequestRec ();

use APR::Table ();
use APR::Bucket ();
use APR::Brigade ();

use Apache2::Const -compile =3D> qw(OK);
use APR::Const -compile =3D> ':common';

sub handler {
my ($filter, $bb) =3D @_;
my $ctx =3D $filter->ctx;

my $data =3D exists $ctx->{data} ? $ctx->{data} : '';
$ctx->{invoked}++;
my ($bdata, $seen_eos) =3D flatten_bb($bb);
# $bdata =3D~ s/-//g;
$data .=3D $bdata if $bdata;

if ($seen_eos) {
my $r_headers =3D $filter->r->headers_out;
use Data::Dumper; # DEBUG
warn "Dumping Apache response headers:\n"; # DEBUG
warn Dumper($r_headers); # DEBUG
if ($data) {
if (open(CAPTION, ">/tmp/ApacheResponse.txt")) {
print CAPTION $data;
close CAPTION;
}
else { warn "Unable to open caption file: $!\n" }
$filter->print($data);
}
}
else {
# store context for all but the last invocation
$ctx->{data} =3D $data;
$filter->ctx($ctx);
}

return Apache2::Const::OK;
}

sub flatten_bb {
my ($bb) =3D shift;
my $seen_eos =3D 0;
my @data;
for (my $b =3D $bb->first; $b; $b =3D $bb->next($b)) {
$seen_eos++, last if $b->is_eos;
$b->read(my $bdata);
push @data, $bdata;
}
return (join('', @data), $seen_eos);
}

1;

-----

It works fine for the response body, but doesn't get the headers. The =
document I mentioned says that headers can only be seen by connection =
filters, so apparently the log contains the empty hash

$VAR1 =3D bless( {}, 'APR::Table' );

I wonder if there is a workaround for this without adding a connection =
filter?

Another question is what the best way would be to pass on arguments to =
the filter so that we could tell it where to store the response?

Thank you in advance.


------=_NextPart_000_00D8_01C98A06.367C4340
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable



content=3Dtext/html;charset=3Diso-8859-1>

style=3D"PADDING-RIGHT: 10px; PADDING-LEFT: 10px; PADDING-TOP: 15px" =
leftMargin=3D0=20
topMargin=3D0 CanvasTabStop=3D"true" name=3D"Compose message area">

We are trying to implement a =
solution that=20
would allow us to capture Apache (v. 2.0.63) responses sent to clients, =
each in=20
a separate file on the server. I added a bucket brigade =
based output filter=20
that mimics an example found in
title=3D"http://perl.apache.org/docs/2.0/user/handlers/filte rs.html#Bucke=
t_Brigade_based_Output_Filters CTRL + Click to follow link"=20
href=3D"http://perl.apache.org/docs/2.0/user/handlers/filter s.html#Bucket=
_Brigade_based_Output_Filters"> title=3D"http://perl.apache.org/docs/2.0/user/handlers/filte rs.html#Bucke=
t_Brigade_based_Output_Filters CTRL + Click to follow link"=20
face=3DCourier>http://perl.apache.org/docs/2.0/user/handlers /filters.html=
#Bucket_Brigade_based_Output_Filters
face=3DCourier>, as follows:

 

____

 

package=20
httpd::FilterCaptureResponse;

 

use strict;
use warnings FATAL =
=3D>=20
'all';

 

use base =
qw(Apache2::Filter);

 

use Apache2::RequestRec =
();

 

use APR::Table ();
use APR::Bucket =
();
use=20
APR::Brigade ();

 

use Apache2::Const -compile =3D> =
qw(OK);
use=20
APR::Const     -compile =3D> =
':common';

 

sub handler {
    =
my ($filter,=20
$bb) =3D @_;
    my $ctx =3D =
$filter->ctx;

 

    my $data =3D =
exists=20
$ctx->{data} ? $ctx->{data} : '';
   =20
$ctx->{invoked}++;
    my ($bdata, $seen_eos) =
flatten_bb($bb);
#    $bdata =3D~ =
s/-//g;
   =20
$data .=3D $bdata if $bdata;

 

    if ($seen_eos)=20
{
        my $r_headers =
$filter->r->headers_out;
      &nb=
sp; use=20
Data::Dumper; # DEBUG
        warn =

"Dumping Apache response headers:\n"; #=20
DEBUG
        warn =
Dumper($r_headers); #=20
DEBUG
        if ($data)=20
{
          if =
(open(CAPTION,=20
">/tmp/ApacheResponse.txt"))=20
{
            =
print=20
CAPTION=20
$data;
          &nb=
sp;=20
close CAPTION;
          =

}
          else { warn =
"Unable=20
to open caption file: $!\n"=20
}
         =20
$filter->print($data);
       =20
}
    }
    else=20
{
        # store context for all =
but the=20
last invocation
        =
$ctx->{data} =
$data;
       =20
$filter->ctx($ctx);
    }

 

    return=20
Apache2::Const::OK;
}

 

sub flatten_bb =
{
    my ($bb) =
shift;
    my $seen_eos =3D 0;
    =
my=20
@data;
    for (my $b =3D $bb->first; $b; $b =
$bb->next($b)) {
        =
$seen_eos++,=20
last if $b->is_eos;
       =20
$b->read(my $bdata);
        =
push=20
@data, $bdata;
    }
    return =
(join('',=20
@data), $seen_eos);
}

 

1;

 

-----

 

It works fine for the response =
body, but=20
doesn't get the headers. The document I mentioned says that headers =
can=20
only be seen by connection filters, so apparently the log contains the =
empty=20
hash

 

 $VAR1 =3D =
bless( {},=20
'APR::Table' );

 

I wonder if there is a workaround for =
this=20
without adding a connection filter?

 

Another question is what the best way =
would be to=20
pass on arguments to the filter so that we could tell it where to store =
the=20
response?

 

Thank you in advance.

 

 


------=_NextPart_000_00D8_01C98A06.367C4340--

Re: Capturing Apache response

am 09.02.2009 19:51:04 von Mark Hedges

On Sun, 8 Feb 2009, Solutio at Gmail wrote:

> I wonder if there is a workaround for this without adding
> a connection filter?

Try using something like the all-in-one FilterSnoop handler
on the same page at
http://perl.apache.org/docs/2.0/user/handlers/filters.html#A ll_in_One_Filter
- I had good results with this. You do have to use it as a
connection output filter but so what. Works for me.

> Another question is what the best way would be to pass on
> arguments to the filter so that we could tell it where to
> store the response?

Probably you want to implement config directives with
Apache2::Module. Some good examples are in
Apache2::Controller::Directives (as a separate package) or
Apache2::AuthenOpenID (built into a handler package). Also
see
http://perl.apache.org/docs/2.0/user/config/custom.html#Crea ting_and_Using_Custom_Configuration_Directives
..

Mark

Re: Capturing Apache response

am 10.02.2009 06:52:58 von Solutio at Gmail

--0016369fa251680c0704628a1910
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I believe parsing custom headers in the filter should not be a problem with
the 'do' and 'get' methods. But how to add them in a CGI environment? As a
quick, dirty and inefficient work-around I consider writing the dynamic
paramenters into a file to be picked up by the filter.


On Sun, Feb 8, 2009 at 7:32 PM, Andr=E9 Warnier wrote:

> Solutio at Gmail wrote:
>
>> Thanks, your version works like a charm...but doesn't quite meets our
>> needs. I apologize for being too vague regarding passing parameters on t=
o
>> the filter. Basically, we would like to somehow link the location and na=
me
>> of the response dump with the request data, like the date/time of the
>> request, session ID, username, URL. We log that information in the audit
>> data repository. In other words, the file handle for the captured data n=
eeds
>> to point to a dynamically created file name.
>>
>>
> Well, I did understand that that was what you were really aiming at, I ju=
st
> couldn't think of a way. These filters get invoked with a fixed set of
> parameters, directly by Apache, so you can't do much there.
> If your application (the response generator) was also made with mod_perl,
> it would be different, because there is a structure in memory that is
> available to the content handler, and can be retrieved by a filter.
> But I can think of one way : if at the level where you generate a respons=
e,
> you can generate an additional HTTP response header containing the necess=
ary
> information, then you could read this header in the filter, and use it to
> construct your trace file name.
> In the code below, the $r->headers_out is a structure out of which you ca=
n
> retrieve a specific HTTP response header by name (and its corresponding
> value).
> The filter could even remove it after usage.
>
> So think of a way of generating such a response header, and when you do g=
et
> back to me. mod_headers may be helpful.
> Will be tomorrow though, it's 2 AM here.
>
>
>
>
> --------------------------------------------------
>> From: "Andr=E9 Warnier"
>> Sent: Sunday, February 08, 2009 5:26 PM
>> To: "Solutio at Gmail"
>> Subject: Re: Capturing Apache response
>>
>> Hi.
>>> Sorry to top-post, but I think in this case it will be clearer.
>>> Not being the ultimate guru, but being probably the only guy available =
at
>>> this day and time, I'll try my hand, at the risk of getting contradicte=
d by
>>> the real gurus later.
>>> So start with a backup of your current filter.
>>>
>>> I don't have a certain answer about how to pass a parameter.
>>> I try to show a way below, I'm not sure it works.
>>> It is based on having an Apache directive like this :
>>> PerlSetvar filter_output_dir "/tmp"
>>>
>>>
>>>
>>> I would then modify the filter as follows :
>>>
>>> package httpd::FilterCaptureResponse;
>>>
>>> use strict;
>>> use warnings FATAL =3D> 'all';
>>>
>>> use base qw(Apache2::Filter);
>>>
>>> use Apache2::RequestRec ();
>>>
>>> use APR::Table ();
>>> use APR::Bucket ();
>>> use APR::Brigade ();
>>>
>>> use Apache2::Const -compile =3D> qw(OK DECLINED);
>>> use APR::Const -compile =3D> ':common';
>>>
>>> # 1) not really essential, but considering it's a compile-time thing :
>>> use Data::Dumper; # DEBUG
>>>
>>> sub handler {
>>> my ($filter, $bb) =3D @_;
>>> my $ctx =3D $filter->ctx;
>>> my $fh;
>>>
>>> unless ($ctx->{invoked}) {
>>> # you probably want to do this the first time you're called,
>>> # not after the data has all gone through
>>> my $r =3D $filter->r;
>>> my $outdir =3D $r->dir_config('filter_output_dir');
>>> unless (open($fh,'>',$outdir/ApacheResponse.txt")) {
>>> warn "filter: could not open trace file : $!"; return DECLINED; # don=
't
>>> filter, let output through
>>> }
>>> $ctx->{fh} =3D $fh; # save the filehandle
>>> my $r_headers =3D $r->headers_out;
>>> # the following goes to the Apache error log
>>> warn "Dumping Apache response headers"; # DEBUG
>>> # but this goes to the trace file
>>> print $fh Dumper($r_headers),"\n"; # DEBUG
>>> $ctx->{invoked} =3D 0; # initialise
>>> }
>>> $ctx->{invoked}++;
>>> $fh =3D $ctx->{fh}; # retrieve stored filehandle
>>> my ($bdata, $seen_eos) =3D flatten_bb($bb);
>>> if ($seen_eos) {
>>> print $fh $bdata if length($bdata !=3D 0);
>>> close $fh;
>>> return Apache2::Const::OK;
>>> }
>>> print $fh $bdata if length($bdata !=3D 0);
>>> # store context for all but the last invocation
>>> $filter->ctx($ctx);
>>> return Apache2::Const::OK;
>>> }
>>>
>>> sub flatten_bb {
>>> my ($bb) =3D shift;
>>> my $seen_eos =3D 0;
>>> my @data;
>>> for (my $b =3D $bb->first; $b; $b =3D $bb->next($b)) {
>>> $seen_eos++, last if $b->is_eos;
>>> $b->read(my $bdata);
>>> push @data, $bdata;
>>> }
>>> return (join('', @data), $seen_eos);
>>> }
>>>
>>> 1;
>>>
>>>
>>> Solutio at Gmail wrote:
>>>
>>>> We are trying to implement a solution that would allow us to capture
>>>> Apache (v. 2.0.63) responses sent to clients, each in a separate file =
on the
>>>> server. I added a bucket brigade based output filter that mimics an ex=
ample
>>>> found in
>>>> http://perl.apache.org/docs/2.0/user/handlers/filters.html#B ucket_Brig=
ade_based_Output_Filters,
>>>> as follows:
>>>>
>>>> ____
>>>>
>>>> package httpd::FilterCaptureResponse;
>>>>
>>>> use strict;
>>>> use warnings FATAL =3D> 'all';
>>>>
>>>> use base qw(Apache2::Filter);
>>>>
>>>> use Apache2::RequestRec ();
>>>>
>>>> use APR::Table ();
>>>> use APR::Bucket ();
>>>> use APR::Brigade ();
>>>>
>>>> use Apache2::Const -compile =3D> qw(OK);
>>>> use APR::Const -compile =3D> ':common';
>>>>
>>>> sub handler {
>>>> my ($filter, $bb) =3D @_;
>>>> my $ctx =3D $filter->ctx;
>>>>
>>>> my $data =3D exists $ctx->{data} ? $ctx->{data} : '';
>>>> $ctx->{invoked}++;
>>>> my ($bdata, $seen_eos) =3D flatten_bb($bb);
>>>> # $bdata =3D~ s/-//g;
>>>> $data .=3D $bdata if $bdata;
>>>>
>>>> if ($seen_eos) {
>>>> my $r_headers =3D $filter->r->headers_out;
>>>> use Data::Dumper; # DEBUG
>>>> warn "Dumping Apache response headers:\n"; # DEBUG
>>>> warn Dumper($r_headers); # DEBUG
>>>> if ($data) {
>>>> if (open(CAPTION, ">/tmp/ApacheResponse.txt")) {
>>>> print CAPTION $data;
>>>> close CAPTION;
>>>> }
>>>> else { warn "Unable to open caption file: $!\n" }
>>>> $filter->print($data);
>>>> }
>>>> }
>>>> else {
>>>> # store context for all but the last invocation
>>>> $ctx->{data} =3D $data;
>>>> $filter->ctx($ctx);
>>>> }
>>>>
>>>> return Apache2::Const::OK;
>>>> }
>>>>
>>>> sub flatten_bb {
>>>> my ($bb) =3D shift;
>>>> my $seen_eos =3D 0;
>>>> my @data;
>>>> for (my $b =3D $bb->first; $b; $b =3D $bb->next($b)) {
>>>> $seen_eos++, last if $b->is_eos;
>>>> $b->read(my $bdata);
>>>> push @data, $bdata;
>>>> }
>>>> return (join('', @data), $seen_eos);
>>>> }
>>>>
>>>> 1;
>>>>
>>>> -----
>>>>
>>>> It works fine for the response body, but doesn't get the headers. The
>>>> document I mentioned says that headers can only be seen by connection
>>>> filters, so apparently the log contains the empty hash
>>>>
>>>> $VAR1 =3D bless( {}, 'APR::Table' );
>>>>
>>>> I wonder if there is a workaround for this without adding a connection
>>>> filter?
>>>>
>>>> Another question is what the best way would be to pass on arguments to
>>>> the filter so that we could tell it where to store the response?
>>>>
>>>> Thank you in advance.
>>>>
>>>>
>>>>
>>>
>>>
>

--0016369fa251680c0704628a1910
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I bel=
ieve parsing custom headers in the filter should not be a problem with=
the=20
'do' and 'get' methods. But how to add them in a CGI enviro=
nment? As a quick,=20
dirty and inefficient work-around I consider writing the dynamic param=
enters=20
into a file to be picked up by the filter.


uote">On Sun, Feb 8, 2009 at 7:32 PM, Andr=E9 Warnier <=
;>
wrote:
lockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, 20=
4, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Solutio at Gmail wrote:

204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Thanks, your version works like a charm...but doesn't quite meets our n=
eeds. I apologize for being too vague regarding passing parameters on to th=
e filter. Basically, we would like to somehow link the location and name of=
the response dump with the request data, like the date/time of the request=
, session ID, username, URL. We log that information in the audit data repo=
sitory. In other words, the file handle for the captured data needs to poin=
t to a dynamically created file name.







Well, I did understand that that was what you were really aiming at, I just=
couldn't think of a way. These filters get invoked with a fixed set of=
parameters, directly by Apache, so you can't do much there.

If your application (the response generator) was also made with mod_perl, i=
t would be different, because there is a structure in memory that is availa=
ble to the content handler, and can be retrieved by a filter.

But I can think of one way : if at the level where you generate a response,=
you can generate an additional HTTP response header containing the necessa=
ry information, then you could read this header in the filter, and use it t=
o construct your trace file name.


In the code below, the $r->headers_out is a structure out of which you c=
an retrieve a specific HTTP response header by name (and its corresponding =
value).

The filter could even remove it after usage.



So think of a way of generating such a response header, and when you do get=
back to me.  mod_headers may be helpful.

Will be tomorrow though, it's 2 AM here.
j3C7c">







204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
--------------------------------------------------

From: "Andr=E9 Warnier" < et=3D"_blank">aw@ice-sa.com>

Sent: Sunday, February 08, 2009 5:26 PM

To: "Solutio at Gmail" < arget=3D"_blank">solutio@gmail.com>

Subject: Re: Capturing Apache response



204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi.

Sorry to top-post, but I think in this case it will be clearer.

Not being the ultimate guru, but being probably the only guy available at t=
his day and time, I'll try my hand, at the risk of getting contradicted=
by the real gurus later.

So start with a backup of your current filter.



I don't have a certain answer about how to pass a parameter.

I try to show a way below, I'm not sure it works.

It is based on having an Apache directive like this :

 PerlSetvar filter_output_dir  "/tmp"







I would then modify the filter as follows :



package httpd::FilterCaptureResponse;



use strict;

use warnings FATAL =3D> 'all';



use base qw(Apache2::Filter);



use Apache2::RequestRec ();



use APR::Table ();

use APR::Bucket ();

use APR::Brigade ();



use Apache2::Const -compile =3D> qw(OK DECLINED);

use APR::Const     -compile =3D> ':common';



# 1) not really essential, but considering it's a compile-time thing :<=
br>
use Data::Dumper; # DEBUG



sub handler {

   my ($filter, $bb) =3D @_;

   my $ctx =3D $filter->ctx;

   my $fh;



   unless ($ctx->{invoked}) {

# you probably want to do this the first time you're called,

# not after the data has all gone through

       my $r =3D $filter->r;

       my $outdir =3D $r->dir_config('filter_ou=
tput_dir');

       unless (open($fh,'>',$outdir/ApacheR=
esponse.txt")) {

  warn "filter: could not open trace file : $!"; return DEC=
LINED; # don't filter, let output through

       }

$ctx->{fh} =3D $fh; # save the filehandle

       my $r_headers =3D $r->headers_out;

# the following goes to the Apache error log

       warn "Dumping Apache response headers"=
;; # DEBUG

# but this goes to the trace file

       print $fh Dumper($r_headers),"\n"; # =
DEBUG

$ctx->{invoked} =3D 0; # initialise

   }

   $ctx->{invoked}++;

   $fh =3D $ctx->{fh}; # retrieve stored filehandle

   my ($bdata, $seen_eos) =3D flatten_bb($bb);

   if ($seen_eos) {

     print $fh $bdata if length($bdata !=3D 0);

     close $fh;

     return Apache2::Const::OK;

   }

   print $fh $bdata if length($bdata !=3D 0);

   # store context for all but the last invocation

   $filter->ctx($ctx);

   return Apache2::Const::OK;

}



sub flatten_bb {

   my ($bb) =3D shift;

   my $seen_eos =3D 0;

   my @data;

   for (my $b =3D $bb->first; $b; $b =3D $bb->next($b)) {<=
br>
       $seen_eos++, last if $b->is_eos;

       $b->read(my $bdata);

       push @data, $bdata;

   }

   return (join('', @data), $seen_eos);

}



1;





Solutio at Gmail wrote:

204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
We are trying to implement a solution that would allow us to capture Apache=
(v. 2.0.63) responses sent to clients, each in a separate file on the serv=
er. I added a bucket brigade based output filter that mimics an example fou=
nd in #Bucket_Brigade_based_Output_Filters" target=3D"_blank">http://perl.apache.=
org/docs/2.0/user/handlers/filters.html#Bucket_Brigade_based _Output_Filters=
, as follows:




____



package httpd::FilterCaptureResponse;



use strict;

use warnings FATAL =3D> 'all';



use base qw(Apache2::Filter);



use Apache2::RequestRec ();



use APR::Table ();

use APR::Bucket ();

use APR::Brigade ();



use Apache2::Const -compile =3D> qw(OK);

use APR::Const     -compile =3D> ':common';



sub handler {

   my ($filter, $bb) =3D @_;

   my $ctx =3D $filter->ctx;



   my $data =3D exists $ctx->{data} ? $ctx->{data} : '=
';

   $ctx->{invoked}++;

   my ($bdata, $seen_eos) =3D flatten_bb($bb);

#    $bdata =3D~ s/-//g;

   $data .=3D $bdata if $bdata;



   if ($seen_eos) {

       my $r_headers =3D $filter->r->headers_out=
;

       use Data::Dumper; # DEBUG

       warn "Dumping Apache response headers:\n&q=
uot;; # DEBUG

       warn Dumper($r_headers); # DEBUG

       if ($data) {

         if (open(CAPTION, ">/tmp/ApacheR=
esponse.txt")) {

           print CAPTION $data;

           close CAPTION;

         }

         else { warn "Unable to open caption=
file: $!\n" }

         $filter->print($data);

       }

   }

   else {

       # store context for all but the last invocation=


       $ctx->{data} =3D $data;

       $filter->ctx($ctx);

   }



   return Apache2::Const::OK;

}



sub flatten_bb {

   my ($bb) =3D shift;

   my $seen_eos =3D 0;

   my @data;

   for (my $b =3D $bb->first; $b; $b =3D $bb->next($b)) {<=
br>
       $seen_eos++, last if $b->is_eos;

       $b->read(my $bdata);

       push @data, $bdata;

   }

   return (join('', @data), $seen_eos);

}



1;



-----



It works fine for the response body, but doesn't get the headers. The d=
ocument I mentioned says that headers can only be seen by connection filter=
s, so apparently the log contains the empty hash



 $VAR1 =3D bless( {}, 'APR::Table' );



I wonder if there is a workaround for this without adding a connection filt=
er?



Another question is what the best way would be to pass on arguments to the =
filter so that we could tell it where to store the response?



Thank you in advance.
















--0016369fa251680c0704628a1910--

Re: Capturing Apache response

am 10.02.2009 21:37:08 von Mark Hedges

Solutio at Gmail wrote:
> Thanks, your version works like a charm...but doesn't
> quite meets our needs. I apologize for being too vague
> regarding passing parameters on to the filter. Basically,
> we would like to somehow link the location and name of the
> response dump with the request data, like the date/time of
> the request, session ID, username, URL. We log that
> information in the audit data repository. In other words,
> the file handle for the captured data needs to point to a
> dynamically created file name.

How about a PerlInitHandler that saves the request information in
the connection object pnotes?

use Apache2::Connection ();
use Apache2::ConnectionUtil ();
use Apache2::RequestRec ();

sub handler {
my $r = shift;
my $c = $r->connection;
# parse cookies, get session id, whatever
my $data = {
protocol => $r->protocol,
session_id => $session_id,
# etc.
};
$c->pnotes(request_data => $data);
}

Then the connection output filter can read it from there.
A similar post-request PerlCleanupHandler or PerlLogHandler
could save any info needed after the request was processed.

You shouldn't overload headers to save this kind of info.
That's a messy way to write code and that kind of
undisciplined coding is the reason some people hate perl.

Is that syntax for $c->pnotes correct from Apache2::ConnectionUtil?
It's not like how $r->pnotes seems to work these days, I would
have expected

$c->pnotes->{request_data} = $data;

or something like that.

Mark

Re: Capturing Apache response

am 10.02.2009 23:04:12 von aw

Mark Hedges wrote:
[...]

> Then the connection output filter can read it from there.

Remove "connection" and I agree, that's much cleaner than putting it in
a header.

> A similar post-request PerlCleanupHandler or PerlLogHandler
> could save any info needed after the request was processed.

But not what the OP is trying to do, which is to save the headers *and*
the body of the responses (plus some data about the request).
I think the PerlCleanupHandler or PerlLogHandler are a bit late for that.

>
> You shouldn't overload headers to save this kind of info.

I basically agree, but this was not done in a vacuum.
It's the OP's first mod_perl venture, unless I'm confusing him with
someone else. So I did not want to push it by introducing yet another
perl module. I assumed that adding a response header was the easiest
way to go, in whatever response handler is being used now (and which is
not a perl handler).

> That's a messy way to write code and that kind of
> undisciplined coding is the reason some people hate perl.

That's hard. I love perl and mod_perl.
And I'm sure the OP will love it also, since it gives him a way to
achieve what he cannot otherwise.

>
> Is that syntax for $c->pnotes correct from Apache2::ConnectionUtil?
> It's not like how $r->pnotes seems to work these days, I would
> have expected
>
I believe the OP's filter is not a connection filter, but a Request
output filter (even if he chose the difficult way and is using the
buckets version).
He is right too, because saving each request data separately would be a
lot harder to do in a Connection filter.


So, Dear Original Poster, do you feel ready to add another perl handler
*before* your request gets processed, to save some information about the
request, in a location where the output filter (running *after* the
request gets processed) can find it ?
This would allow you to decide, at request time, the filename for the
saved response, and have the output filter place it there.
Both parts can co-exist in the same code module.
And there is also an example on the mod_perl site.

Re: Capturing Apache response

am 11.02.2009 00:14:32 von Solutio at Gmail

First of all, I thank you both for your expert opinion on the topic. I have
never had to fiddle with this sort of Apache customization, so I'm learning
in the process...

As for the way to communicate the file name to the filter, sure, we would
like to keep it as simple as possible and avoid pitfalls. I agree that Perl
is a powerful tool that can easily screw many things up if applied
carelessly.

I guess if we properly implement the custom header option, it shouldn't
become a problem. We'll give it a try.

--------------------------------------------------
From: "André Warnier"
Sent: Tuesday, February 10, 2009 4:04 PM
To:
Subject: Re: Capturing Apache response

> Mark Hedges wrote:
> [...]
>
>> Then the connection output filter can read it from there.
>
> Remove "connection" and I agree, that's much cleaner than putting it in a
> header.
>
>> A similar post-request PerlCleanupHandler or PerlLogHandler
>> could save any info needed after the request was processed.
>
> But not what the OP is trying to do, which is to save the headers *and*
> the body of the responses (plus some data about the request).
> I think the PerlCleanupHandler or PerlLogHandler are a bit late for that.
>
>>
>> You shouldn't overload headers to save this kind of info.
>
> I basically agree, but this was not done in a vacuum.
> It's the OP's first mod_perl venture, unless I'm confusing him with
> someone else. So I did not want to push it by introducing yet another
> perl module. I assumed that adding a response header was the easiest way
> to go, in whatever response handler is being used now (and which is not a
> perl handler).
>
>> That's a messy way to write code and that kind of
>> undisciplined coding is the reason some people hate perl.
>
> That's hard. I love perl and mod_perl.
> And I'm sure the OP will love it also, since it gives him a way to achieve
> what he cannot otherwise.
>
>>
>> Is that syntax for $c->pnotes correct from Apache2::ConnectionUtil?
>> It's not like how $r->pnotes seems to work these days, I would
>> have expected
>>
> I believe the OP's filter is not a connection filter, but a Request output
> filter (even if he chose the difficult way and is using the buckets
> version).
> He is right too, because saving each request data separately would be a
> lot harder to do in a Connection filter.
>
>
> So, Dear Original Poster, do you feel ready to add another perl handler
> *before* your request gets processed, to save some information about the
> request, in a location where the output filter (running *after* the
> request gets processed) can find it ?
> This would allow you to decide, at request time, the filename for the
> saved response, and have the output filter place it there.
> Both parts can co-exist in the same code module.
> And there is also an example on the mod_perl site.
>
>
>

Re: Capturing Apache response

am 13.02.2009 19:07:22 von Mark Hedges

On Tue, 10 Feb 2009, Solutio at Gmail wrote:

> First of all, I thank you both for your expert opinion on the topic. I have
> never had to fiddle with this sort of Apache customization, so I'm learning in
> the process...
>
> As for the way to communicate the file name to the filter, sure, we would like
> to keep it as simple as possible and avoid pitfalls. I agree that Perl is a
> powerful tool that can easily screw many things up if applied carelessly.
>
> I guess if we properly implement the custom header option, it shouldn't become
> a problem. We'll give it a try.

To recap, you shouldn't use the custom header, you should
probably use $r->pnotes or maybe $c->pnotes to store the
information from the beginning to be accessed at the end.
This is what the pnotes tables are for. Don't mess with the
headers. Headers are for communicating with the client.

Mark

Re: Capturing Apache response

am 13.02.2009 19:36:56 von Solutio at Gmail

I understand your point, and we'll strip the custom headers off before
sending the response to the client. We are also looking at another way to
dump the response (in the application's parent template class), which would
help us avoid messing with Apache altogether.

--------------------------------------------------
From: "Mark Hedges"
Sent: Friday, February 13, 2009 12:07 PM
To: "Solutio at Gmail"
Cc: "André Warnier" ;
Subject: Re: Capturing Apache response

>
>
>
> On Tue, 10 Feb 2009, Solutio at Gmail wrote:
>
>> First of all, I thank you both for your expert opinion on the topic. I
>> have
>> never had to fiddle with this sort of Apache customization, so I'm
>> learning in
>> the process...
>>
>> As for the way to communicate the file name to the filter, sure, we would
>> like
>> to keep it as simple as possible and avoid pitfalls. I agree that Perl is
>> a
>> powerful tool that can easily screw many things up if applied carelessly.
>>
>> I guess if we properly implement the custom header option, it shouldn't
>> become
>> a problem. We'll give it a try.
>
> To recap, you shouldn't use the custom header, you should
> probably use $r->pnotes or maybe $c->pnotes to store the
> information from the beginning to be accessed at the end.
> This is what the pnotes tables are for. Don't mess with the
> headers. Headers are for communicating with the client.
>
> Mark
>

Re: Capturing Apache response

am 13.02.2009 20:11:09 von Mark Hedges

This message is in MIME format. The first part should be readable text,
while the remaining parts are likely unreadable without MIME-aware tools.

--373334887-2105706774-1234552269=:12818
Content-Type: TEXT/PLAIN; charset=iso-8859-1
Content-Transfer-Encoding: QUOTED-PRINTABLE


I'm trying to say that you shouldn't use custom headers to
save this information, when you should be using the correct
facility, which is pnotes. Why do you want to use the wrong
thing when the right thing is available?

Mark

On Fri, 13 Feb 2009, Solutio at Gmail wrote:

> I understand your point, and we'll strip the custom headers off before se=
nding
> the response to the client. We are also looking at another way to dump th=
e
> response (in the application's parent template class), which would help u=
s
> avoid messing with Apache altogether.
>
> --------------------------------------------------
> From: "Mark Hedges"
> Sent: Friday, February 13, 2009 12:07 PM
> To: "Solutio at Gmail"
> Cc: "Andr=E9 Warnier" ;
> Subject: Re: Capturing Apache response
>
> >
> >
> >
> > On Tue, 10 Feb 2009, Solutio at Gmail wrote:
> >
> > > First of all, I thank you both for your expert opinion on the topic. =
I
> > > have
> > > never had to fiddle with this sort of Apache customization, so I'm
> > > learning in
> > > the process...
> > >
> > > As for the way to communicate the file name to the filter, sure, we w=
ould
> > > like
> > > to keep it as simple as possible and avoid pitfalls. I agree that Per=
l is
> > > a
> > > powerful tool that can easily screw many things up if applied careles=
sly.
> > >
> > > I guess if we properly implement the custom header option, it shouldn=
't
> > > become
> > > a problem. We'll give it a try.
> >
> > To recap, you shouldn't use the custom header, you should
> > probably use $r->pnotes or maybe $c->pnotes to store the
> > information from the beginning to be accessed at the end.
> > This is what the pnotes tables are for. Don't mess with the
> > headers. Headers are for communicating with the client.
> >
> > Mark
> >
>
--373334887-2105706774-1234552269=:12818--

Re: Capturing Apache response

am 13.02.2009 20:42:04 von Solutio at Gmail

We'll look at pnotes too. Thanks again.

--------------------------------------------------
From: "Mark Hedges"
Sent: Friday, February 13, 2009 1:11 PM
To: "Solutio at Gmail"
Cc: "André Warnier" ;
Subject: Re: Capturing Apache response

>
> I'm trying to say that you shouldn't use custom headers to
> save this information, when you should be using the correct
> facility, which is pnotes. Why do you want to use the wrong
> thing when the right thing is available?
>
> Mark
>
> On Fri, 13 Feb 2009, Solutio at Gmail wrote:
>
>> I understand your point, and we'll strip the custom headers off before
>> sending
>> the response to the client. We are also looking at another way to dump
>> the
>> response (in the application's parent template class), which would help
>> us
>> avoid messing with Apache altogether.
>>
>> --------------------------------------------------
>> From: "Mark Hedges"
>> Sent: Friday, February 13, 2009 12:07 PM
>> To: "Solutio at Gmail"
>> Cc: "André Warnier" ;
>> Subject: Re: Capturing Apache response
>>
>> >
>> >
>> >
>> > On Tue, 10 Feb 2009, Solutio at Gmail wrote:
>> >
>> > > First of all, I thank you both for your expert opinion on the topic.
>> > > I
>> > > have
>> > > never had to fiddle with this sort of Apache customization, so I'm
>> > > learning in
>> > > the process...
>> > >
>> > > As for the way to communicate the file name to the filter, sure, we
>> > > would
>> > > like
>> > > to keep it as simple as possible and avoid pitfalls. I agree that
>> > > Perl is
>> > > a
>> > > powerful tool that can easily screw many things up if applied
>> > > carelessly.
>> > >
>> > > I guess if we properly implement the custom header option, it
>> > > shouldn't
>> > > become
>> > > a problem. We'll give it a try.
>> >
>> > To recap, you shouldn't use the custom header, you should
>> > probably use $r->pnotes or maybe $c->pnotes to store the
>> > information from the beginning to be accessed at the end.
>> > This is what the pnotes tables are for. Don't mess with the
>> > headers. Headers are for communicating with the client.
>> >
>> > Mark
>> >
>>