Apache2::ModLogConfig help needed

Hi,

I am in the process of developing an interface to mod_log_config. It can do=
2=20
things:

* make CustomLog call perl handlers
* work as a log drain or log file

Log Drain
=========3D

CustomLog "@perl: My::Handler" "%I%O"

Assuming that My::Handler is defined it will receive 4 parameters here, the=
=20
request object, the input bytes according to mod_logio (the %I format), the=
=20
output bytes according to mod_logio (%O) and a newline string. Other=20
information can of course be specified in the format string.

Make it call perl handlers
==================== =====3D=
=3D

CustomLog log/file.log "%r %{My::Fmt}^ ..."

It allocates the ^ format character and then calls My::Fmt with the request=
=20
object as the only parameter. The <> format modifiers can be used to specif=
y=20
if the initial request is meant or the final one.

My::Fmt is expected to return a string that is then inserted in place of th=
e=20
format specification.


Now, I have found a bug in apache that has been fixed somewhere between 2.2=
9=20
and 2.2.15. I have also figured out a workaround. But I expect it to lead t=
o=20
segfaults when the server is restarted by means of SIGHUP or SIGUSR1 and in=
=20
the old config BufferedLogs were on while in the new one they are off.

The reason for this mail is that I ask for help to figure out which apache=
=20
version has fixed the bug. So, if you have a httpd between 2.2.9 and 2.2.15=
=20
could you please try out the module.

perl Makefile.PL
make test

If the test hangs in starting up the web server and the only lines in t/log=
s=20
error_log look similar to these:

$ cat t/logs/error_log=20
V=3D2002009.
V=3D2002009.
buggy.

where V=3D... is the apache version then the bug is present.

If you like you can then go to line 169 in ModLogConfig.xs and change the=20
2002009 to your version+1 and retry.

The current state can be downloaded under:

http://foertsch.name/Apache2-ModLogConfig-0.01.tar.gz


BTW, if you feel urge to comment the idea or the implementation please don'=
t=20
hesitate.

Thanks,
Torsten Förtsch

=2D-=20
Need professional modperl support? Hire me! (http://foertsch.name)

Like fantasy? http://kabatinte.net

Re: Apache2::ModLogConfig help needed (feature suggestion)

> Hi,
>
> I am in the process of developing an interface to mod_log_config. It can do 2
> things:
>
> * make CustomLog call perl handlers
> * work as a log drain or log file
[sNip]
> BTW, if you feel urge to comment the idea or the implementation please don't
> hesitate.

What I think would be a very nice addition to that list of two
things is to be able to specify a customized filename for the log
files. I'm unsure if this is even possible though.

I currently pipe through "cronolog" to include the YEAR and MONTH
numbers in the filenames. If this could be handled without piping
out to another process, then it would reduce a lot of pipe process
overhead (for each VirtualHost there are two "cronolog" processes,
one for the CustomLog, and the other for the ErrorLog).

An example of the filenames for log files for this month are
"access.2011-03.log" and "errors.2011-03.log."

If this feature is possible, then this module could also help to
make ModPerl 2 a little bit more popular if other administrators are
interested in reducing the number of piped processes.

Randolf Richardson - randolf@inter-corporate.com
Inter-Corporate Computer & Network Services, Inc.
Vancouver, British Columbia, Canada
http://www.inter-corporate.com

Re: Apache2::ModLogConfig help needed (feature suggestion)

On Thursday, March 03, 2011 07:43:54 Randolf Richardson wrote:
> > I am in the process of developing an interface to mod_log_config. It can
> > do 2=20
> >
> > things:
> >=20
> >
> > * make CustomLog call perl handlers
> > * work as a log drain or log file
>=20
> [sNip]
>=20
> > BTW, if you feel urge to comment the idea or the implementation please
> > don't hesitate.
>=20
> What I think would be a very nice addition to that list of two=20
> things is to be able to specify a customized filename for the log=20
> files. I'm unsure if this is even possible though.

Not taking into account the security implications this would be possible:


package My::LogWriter;
use Apache2::RequestRec ();
use Apache2::Const -compile=3D>'OK';

sub handler {
my ($r, @string)=3D@_;

my $filehandle=3Dcompute_filehandle_based_on_request_time($r->r equest_tim=
e);
print $filehandle join '', @string;

return Apache2::Const::OK;
}

PerlLoadModule Apache2::ModLogConfig
CustomLog "@perl: My::LogWriter" "format spec"

You even can use anonymous subs here. Thus, the handler can be made more=20
general:

CustomLog "@perl: sub {My::LogWriter::handler(q/%YYYY-%MM-%DD.log/, @_)}" \
"format spec"

> I currently pipe through "cronolog" to include the YEAR and MONTH=
=20
> numbers in the filenames. If this could be handled without piping=20
> out to another process, then it would reduce a lot of pipe process=20
> overhead (for each VirtualHost there are two "cronolog" processes,=20
> one for the CustomLog, and the other for the ErrorLog).
>=20
> An example of the filenames for log files for this month are=20
> "access.2011-03.log" and "errors.2011-03.log."
>=20
> If this feature is possible, then this module could also help to=
=20
> make ModPerl 2 a little bit more popular if other administrators are=20
> interested in reducing the number of piped processes.

I think it depends on the number of workers if this approach is really bett=
er.

Why don't you log an VHost ID as well and and use exactly 1 reader? Don't k=
now=20
if cronolog can do that but generally you can use the additional ID to spli=
t=20
the log into parts for each VHost. That way the opening and file name guess=
ing=20
is done in one place.

Anyway, the reason for me to implement this was to get hands on values that=
=20
are otherwise really hard to get. mod_logio measures the number of bytes th=
at=20
come in an go out on the wire for each request. For the input it installs a=
=20
network level filter. Modperl implements only connection and request level=
=20
filters. But the output counting is done by the core output filter and=20
reported to mod_logio by means of an optional function. So, measuring these=
=20
values in perl is impossible.

Now with this module I can do (provided mod_perl, mod_log_config and mod_lo=
gio=20
are loaded)


package My::IO;
use Apache2::RequestRec ();
use Apache2::Const -compile=3D>'OK';

sub handler {
my ($r, $inbytes, $outbytes)=3D@_;

$r->pnotes->{inout}=3D[$inbytes, $outbytes];

return Apache2::Const::OK;
}

PerlLoadModule Apache2::ModLogConfig
CustomLog "@perl: My::IO" "%I%O"

A request pool cleanup handler for example would then be able to use the=20
values.

Torsten Förtsch

=2D-=20
Need professional modperl support? Hire me! (http://foertsch.name)

Like fantasy? http://kabatinte.net

Re: Apache2::ModLogConfig help needed

On Wednesday, March 02, 2011 22:02:21 Torsten Förtsch wrote:
> Now, I have found a bug in apache that has been fixed somewhere between
> 2.2.9 and 2.2.15.

Well, it's probably not a bug per se. The behavior simply differs when=20
mod_log_config is compiled in compared to a shared module.

Torsten Förtsch

=2D-=20
Need professional modperl support? Hire me! (http://foertsch.name)

Like fantasy? http://kabatinte.net