Mem leak upon recovering after a disconnect

am 10.01.2003 03:40:05 von John Heitmann

Hello,

In our mod_perl app we came across this rather nasty memory leak. I
assume it is from DBD::mysql judging from the source, but if it's a DBI
thing I can take it to that list.

This loop leaks at least a page of memory per iteration:

for my $i (1..1000) {
my $dbh = DBI->connect(...);
$dbh->disconnect();
$dbh->do(something);
sleep(1);
}

A small 4kb per iteration isn't the limit to how bad this leak is, we
saw multiple megabytes per iteration leak in our full production setup.
It is easy to workaround once we found the problem, we just check the
Active flag before we do anything. Finding the problem took quite
awhile though :-(

I understand that the above construct is dubious. There is a good
explanation I won't get into as to how it came about. I'm not
necessarily asking for the leak to get fixed, but if it doesn't get
fixed I think the above should error out or give a warning.

I'll be happy to work up a patch to fix the leak or add a warning if
someone could point me at the right places in the source to dive in, I
didn't have much luck finding the right spot to start my attack.

Thanks,

John

------------------------------------------------------------ ---------
Please check "http://www.mysql.com/Manual_chapter/manual_toc.html" before
posting. To request this thread, e-mail msql-mysql-modules-thread1998@lists.mysql.com

To unsubscribe, send a message to the address shown in the
List-Unsubscribe header of this message. If you cannot see it,
e-mail msql-mysql-modules-unsubscribe@lists.mysql.com instead.

Re: Mem leak upon recovering after a disconnect

am 10.01.2003 11:07:34 von Jochen Wiedmann

Zitiere John Heitmann :

> for my $i (1..1000) {
> my $dbh = DBI->connect(...);
> $dbh->disconnect();
> $dbh->do(something);
> sleep(1);
> }

I do get you right, that the problem is caused because you
attempt to use a closed connection?

> It is easy to workaround once we found the problem, we just check the
> Active flag before we do anything. Finding the problem took quite
> awhile though :-(

By that you mean that you omit using the closed connection?

> I understand that the above construct is dubious.

It is not what I would have expected the users to do. :-)
Question: What happens, if you set the RaiseError flag and trap
the exception in the loop? (Using RaiseError is definitely
recommended.) Does that help?

Regards,

Jochen

------------------------------------------------------------ ---------
Please check "http://www.mysql.com/Manual_chapter/manual_toc.html" before
posting. To request this thread, e-mail msql-mysql-modules-thread1999@lists.mysql.com

To unsubscribe, send a message to the address shown in the
List-Unsubscribe header of this message. If you cannot see it,
e-mail msql-mysql-modules-unsubscribe@lists.mysql.com instead.

Re: Mem leak upon recovering after a disconnect

am 10.01.2003 11:07:34 von Jochen Wiedmann

Re: Mem leak upon recovering after a disconnect

am 10.01.2003 19:42:35 von John Heitmann

On Friday, January 10, 2003, at 05:07 AM, Jochen Wiedmann wrote:

>
> Zitiere John Heitmann :
>
>> for my $i (1..1000) {
>> my $dbh = DBI->connect(...);
>> $dbh->disconnect();
>> $dbh->do(something);
>> sleep(1);
>> }
>
> I do get you right, that the problem is caused because you
> attempt to use a closed connection?

Exactly.

>
>> It is easy to workaround once we found the problem, we just check the
>> Active flag before we do anything. Finding the problem took quite
>> awhile though :-(
>
> By that you mean that you omit using the closed connection?

Right, so here's our current fix (since in our full example we can
never be sure if 3rd party code has closed the connection):

for my $i (1..1000) {
my $dbh = DBI->connect(...);
$dbh->disconnect();
unless ($dbh->{Active}) {
$dbh = DBI->connect(...);
}
$dbh->do(something);
sleep(1);
}

>
>> I understand that the above construct is dubious.
>
> It is not what I would have expected the users to do. :-)

:-)

> Question: What happens, if you set the RaiseError flag and trap
> the exception in the loop? (Using RaiseError is definitely
> recommended.) Does that help?

We run with RaiseError =>1 and PrintError => 0 exclusively. Changing
those the other way doesn't make a difference either. It is silent.
This was originally found on a redhat linux system, I just verified it
happens on Mac OS X too (no errors reported, mem leaks). Both running
Perl 5.8.0.

Thanks,

John

------------------------------------------------------------ ---------
Please check "http://www.mysql.com/Manual_chapter/manual_toc.html" before
posting. To request this thread, e-mail msql-mysql-modules-thread2000@lists.mysql.com

To unsubscribe, send a message to the address shown in the
List-Unsubscribe header of this message. If you cannot see it,
e-mail msql-mysql-modules-unsubscribe@lists.mysql.com instead.

Re: Mem leak upon recovering after a disconnect

am 10.01.2003 19:42:35 von John Heitmann