Re: Possible extension for DBI error handler (fwd)

From DBI's pod:

This only works for methods which return a single value
and is hard to make reliable (avoiding infinite loops,
for example) and so isn't recommended for general use!
If you find a good use for it then please let me know.

The use of it would be returning a defined-but-false
value (probably '') on the way through. The use would
be:

my $err_handler
= sub
{
# boilerplate
...

if( $err =~ /$regex_for_connection_errors/o )
{
log_error 'Failed database connection:', $err;

$dbh->connect_cached( @{ $meta{ $dbh } } );

$_[-1] = '';

# short circut the handler logic and let the
# caller deal with the false return value.

1
}
else
{
# nothing I can do about it... let the exception
# take its course...

0
}
};


this allows a polling daemon to use code like:

POLL:
for(;;)
{
if( my $now = $get_dbtime_t->() )
{
sleep $poll_int - ( $now % $poll_int );

my $rowz
= $dbh->selectall_arrayref( $sth, undef, @valuz ) )
or next POLL;

...
}

# at this point the connection error was rendered non-fatal
# by the handler. and the database handle [hopefully] re-
# connected to the database. in any case, we will find out
# about it on the next polling pass...
}

Thing about this approach is that I don't need an explicit
check for the error in my main code.

What'd really be sexy -- and seems doable from what I've
seen inside DBI -- is to use tristate logic to allow
restarting the failed operation:

sub some_dbi_method
{
my $result =
do
{
if( my $handler = $dbh->{meta}{err_handler} )
{
$handler->( $errstr, $dbh, $ret );
}
else
{
undef
}
};

if( $result )
{
# caller dealt with it.

return $ret;
}
elsif( defined $result )
{
# retry the failed operation by jumping back into the
# current subroutine.

goto &foo;
}
else
{
# error handler set result to undef, which means
# to let RaiseError, etc, follow its normal course.

$dbh->_raise_error;
}
}

The user would have to manage their own timeouts in a place
accessable to the error handler (kwikhak, without boilerplate):

my %meta = ();

my $handle_error
= sub
{
my( $err, $dbh, $cruft ) = @_;

if( $err =~ /$regex/o )
{
my( $time, $trys, $argz )
= @{ $meta{ $dbh } }{ qw(timeout, retry_count, connect_args)
};

for( 1..$trys )
{
# carp would help the poor slob debugging
# this figure out WHICH dbh is causing
# the pain.

carp "Retrying connection ($_)...";

$_[0] = DBI->connect( @$argz )
and last;

sleep $time;
}

# return false-but-defined if the connection was
# remade, otherwise give up.

$_[0] ? '' : undef;
}
else
{
undef
}
};

# override the DBI constrctors with one that
# saves the arguments for future use and then
# redispatches them to DBI.

for( qw(connect connect_cached connect_whatever) )
{
my $dbisub = DBI->can( $name );

my $ref = qualify_to_ref $_, __PACKAGE__;

*$ref
= sub
{
my $proto = shift;

$_[ -1 ]->{ HandleError } ||= $handle_error;

my $argz = [ @_ ];

unshift @_, DBI;

my $dbh = &$dbisub;

$meta{ $dbh } = $argz;
};
}

This allows a derived class to reconnect and retry the
operation without having to wrap the DBI class at all.
This also minimized the overhead since it doees not
require an AUTOLOAD or re-dispatching every call: only
the constructors are overloaded and they all pass the
result off to DBI as-is.

I could dodge overloading the DBI constructors by calling
a separate metadata storage routine. The only way to
avoid wrapping all of the DBI calls in their own eval's
seems to be some way for HandleError to retry the failed
operation if it thinks the underlying issue has been
resolved.

enjoi

--
Steven Lembark 85-09 90th Street
Workhorse Computing Woodhaven, NY 11421
lembark@wrkhors.com 1 888 359 3508

Re: Possible extension for DBI error handler (fwd)

On Mon, Jan 23, 2006 at 02:37:24PM -0500, Steven Lembark wrote:
>
> >From DBI's pod:
>
> This only works for methods which return a single value
> and is hard to make reliable (avoiding infinite loops,
> for example) and so isn't recommended for general use!
> If you find a good use for it then please let me know.
>
> The use of it would be returning a defined-but-false
> value (probably '') on the way through. The use would
> be:
>
> my $err_handler
> = sub
> {
> # boilerplate
> ...
>
> if( $err =~ /$regex_for_connection_errors/o )
> {
> log_error 'Failed database connection:', $err;
>
> $dbh->connect_cached( @{ $meta{ $dbh } } );

If the connection has failed then connect_cached() will return a new,
different, $dbh which would be lost with the code above.

To change the $dbh in the application to be this new $dbh you'd need to
use swap_internal_handle(). If I was mad enough to try this I'd probably
do:
$new_dbh = $dbh->clone;
$dbh->swap_internal_handle($new_dbh);

Tim.

> $_[-1] = '';
>
> # short circut the handler logic and let the
> # caller deal with the false return value.
>
> 1
> }
> else
> {
> # nothing I can do about it... let the exception
> # take its course...
>
> 0
> }
> };
>
>
> this allows a polling daemon to use code like:
>
> POLL:
> for(;;)
> {
> if( my $now = $get_dbtime_t->() )
> {
> sleep $poll_int - ( $now % $poll_int );
>
> my $rowz
> = $dbh->selectall_arrayref( $sth, undef, @valuz ) )
> or next POLL;
>
> ...
> }
>
> # at this point the connection error was rendered non-fatal
> # by the handler. and the database handle [hopefully] re-
> # connected to the database. in any case, we will find out
> # about it on the next polling pass...
> }
>
> Thing about this approach is that I don't need an explicit
> check for the error in my main code.
>
> What'd really be sexy -- and seems doable from what I've
> seen inside DBI -- is to use tristate logic to allow
> restarting the failed operation:
>
> sub some_dbi_method
> {
> my $result =
> do
> {
> if( my $handler = $dbh->{meta}{err_handler} )
> {
> $handler->( $errstr, $dbh, $ret );
> }
> else
> {
> undef
> }
> };
>
> if( $result )
> {
> # caller dealt with it.
>
> return $ret;
> }
> elsif( defined $result )
> {
> # retry the failed operation by jumping back into the
> # current subroutine.
>
> goto &foo;
> }
> else
> {
> # error handler set result to undef, which means
> # to let RaiseError, etc, follow its normal course.
>
> $dbh->_raise_error;
> }
> }
>
> The user would have to manage their own timeouts in a place
> accessable to the error handler (kwikhak, without boilerplate):
>
> my %meta = ();
>
> my $handle_error
> = sub
> {
> my( $err, $dbh, $cruft ) = @_;
>
> if( $err =~ /$regex/o )
> {
> my( $time, $trys, $argz )
> = @{ $meta{ $dbh } }{ qw(timeout, retry_count, connect_args)
> };
>
> for( 1..$trys )
> {
> # carp would help the poor slob debugging
> # this figure out WHICH dbh is causing
> # the pain.
>
> carp "Retrying connection ($_)...";
>
> $_[0] = DBI->connect( @$argz )
> and last;
>
> sleep $time;
> }
>
> # return false-but-defined if the connection was
> # remade, otherwise give up.
>
> $_[0] ? '' : undef;
> }
> else
> {
> undef
> }
> };
>
> # override the DBI constrctors with one that
> # saves the arguments for future use and then
> # redispatches them to DBI.
>
> for( qw(connect connect_cached connect_whatever) )
> {
> my $dbisub = DBI->can( $name );
>
> my $ref = qualify_to_ref $_, __PACKAGE__;
>
> *$ref
> = sub
> {
> my $proto = shift;
>
> $_[ -1 ]->{ HandleError } ||= $handle_error;
>
> my $argz = [ @_ ];
>
> unshift @_, DBI;
>
> my $dbh = &$dbisub;
>
> $meta{ $dbh } = $argz;
> };
> }
>
> This allows a derived class to reconnect and retry the
> operation without having to wrap the DBI class at all.
> This also minimized the overhead since it doees not
> require an AUTOLOAD or re-dispatching every call: only
> the constructors are overloaded and they all pass the
> result off to DBI as-is.
>
> I could dodge overloading the DBI constructors by calling
> a separate metadata storage routine. The only way to
> avoid wrapping all of the DBI calls in their own eval's
> seems to be some way for HandleError to retry the failed
> operation if it thinks the underlying issue has been
> resolved.
>
> enjoi
>
> --
> Steven Lembark 85-09 90th Street
> Workhorse Computing Woodhaven, NY 11421
> lembark@wrkhors.com 1 888 359 3508

Re: Possible extension for DBI error handler (fwd)

-- Tim Bunce

>> $dbh->connect_cached( @{ $meta{ $dbh } } );
>
> If the connection has failed then connect_cached() will return a new,
> different, $dbh which would be lost with the code above.
>
> To change the $dbh in the application to be this new $dbh you'd need to
> use swap_internal_handle(). If I was mad enough to try this I'd probably
> do:
> $new_dbh = $dbh->clone;
> $dbh->swap_internal_handle($new_dbh);

So much for hacking error handlers in my mail editor :-)

Why "mad enough"? The point is that if I have a long
running (months at a time) daemon then it'd be nice
not to have to restart it each time the databas is
restarted: just let HandleError deal with it and keep
on trukin...

--
Steven Lembark 85-09 90th Street
Workhorse Computing Woodhaven, NY 11421
lembark@wrkhors.com 1 888 359 3508

Re: Possible extension for DBI error handler (fwd)

On Wed, Jan 25, 2006 at 02:51:14PM -0500, Steven Lembark wrote:
>
>
> -- Tim Bunce
>
> >> $dbh->connect_cached( @{ $meta{ $dbh } } );
> >
> > If the connection has failed then connect_cached() will return a new,
> > different, $dbh which would be lost with the code above.
> >
> > To change the $dbh in the application to be this new $dbh you'd need to
> > use swap_internal_handle(). If I was mad enough to try this I'd probably
> > do:
> > $new_dbh = $dbh->clone;
> > $dbh->swap_internal_handle($new_dbh);
>
> So much for hacking error handlers in my mail editor :-)
>
> Why "mad enough"? The point is that if I have a long
> running (months at a time) daemon then it'd be nice
> not to have to restart it each time the databas is
> restarted: just let HandleError deal with it and keep
> on trukin...

Sure, I can certainly understand the motivation. It's just that I
suspect there will be plenty of worms to deal with in this can before
you arrive at a reasonably robust solution.

Tim.

Re: Possible extension for DBI error handler (fwd)

Tim Bunce wrote:
> On Wed, Jan 25, 2006 at 02:51:14PM -0500, Steven Lembark wrote:
>
>>
>>-- Tim Bunce
>>
>>>> $dbh->connect_cached( @{ $meta{ $dbh } } );
>>>
>>>If the connection has failed then connect_cached() will return a new,
>>>different, $dbh which would be lost with the code above.
>>>
>>>To change the $dbh in the application to be this new $dbh you'd need to
>>>use swap_internal_handle(). If I was mad enough to try this I'd probably
>>>do:
>>> $new_dbh = $dbh->clone;
>>> $dbh->swap_internal_handle($new_dbh);
>>
>>So much for hacking error handlers in my mail editor :-)
>>
>>Why "mad enough"? The point is that if I have a long
>>running (months at a time) daemon then it'd be nice
>>not to have to restart it each time the databas is
>>restarted: just let HandleError deal with it and keep
>>on trukin...
>
>
> Sure, I can certainly understand the motivation. It's just that I
> suspect there will be plenty of worms to deal with in this can before
> you arrive at a reasonably robust solution.
>
> Tim.


That's what I did with DBIx::HA, which was the reason for my requesting
swap_inner_handle() in the first place. :-)
Very obscure worms still existed as of DBI 1.45. I haven't checked
since, but the problems were extremely hard to reproduce (of course).

H