Where"s my memory going?!

Hi all.

I have a script that opens a socket, creates a persistent mysql
connection, and loops to receive data. When the amount of specified
data has been received, it calls a class which processes the data and
inserts it into the database. Each iteration, I unset/destruct that
class I call. However, the script keeps going up in memory and
eventually runs out, causing a fatal error. Any thoughts on where to
start to see where I'm losing my memory?

Thanks in advance,
~Philip

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Where"s my memory going?!

Philip Thompson wrote:
> Hi all.
>
> I have a script that opens a socket, creates a persistent mysql
> connection, and loops to receive data. When the amount of specified data
> has been received, it calls a class which processes the data and inserts
> it into the database. Each iteration, I unset/destruct that class I
> call. However, the script keeps going up in memory and eventually runs
> out, causing a fatal error. Any thoughts on where to start to see where
> I'm losing my memory?
>
> Thanks in advance,
> ~Philip
>

We cannot tell you anything without see an example of what your script is doing.

Let us see some code!

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Where"s my memory going?!

------=_NextPart_000_00A9_01CA4093.4EA84060
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

well this sound clearly to me like you are not freeing resultsets you =
are not going to use anymore. In long scripts you have to take care of =
this. on short scripts you can be a bit weak on that, because the =
resultsets are closed and freed on script ending.

assumed u r using MySQL are u using mysql_free_result($result)

goog luck

ralph_deffke@yahoo.de


"Philip Thompson" wrote in message =
news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com...
> Hi all.
>=20
> I have a script that opens a socket, creates a persistent mysql =20
> connection, and loops to receive data. When the amount of specified =20
> data has been received, it calls a class which processes the data and =

> inserts it into the database. Each iteration, I unset/destruct that =20
> class I call. However, the script keeps going up in memory and =20
> eventually runs out, causing a fatal error. Any thoughts on where to =20
> start to see where I'm losing my memory?
>=20
> Thanks in advance,
> ~Philip
------=_NextPart_000_00A9_01CA4093.4EA84060--

Re: Where"s my memory going?!

On Sep 28, 2009, at 4:18 PM, Jim Lucas wrote:

> Philip Thompson wrote:
>> Hi all.
>>
>> I have a script that opens a socket, creates a persistent mysql
>> connection, and loops to receive data. When the amount of specified
>> data
>> has been received, it calls a class which processes the data and
>> inserts
>> it into the database. Each iteration, I unset/destruct that class I
>> call. However, the script keeps going up in memory and eventually
>> runs
>> out, causing a fatal error. Any thoughts on where to start to see
>> where
>> I'm losing my memory?
>>
>> Thanks in advance,
>> ~Philip
>>
>
> We cannot tell you anything without see an example of what your
> script is doing.
>
> Let us see some code!

I was wondering if you were gonna ask that! Umm... I don't know if I
can show you the code b/c it's not as trivial as I explained it above.
I'll tell you what, I give you something to chew on and you let me
know if it's enough.

class SocketListener extends Socket
{
public function __construct ()
{
$this->core = new coreFunctions (false, 'interface');
parent::__construct ($this->core, $host, $port, $clients);
$this->readFunction = 'processData';
$this->startPersistentListener();
}

public function processData ($data)
{
if ($this->filetype == 'demographics') {
$demoImporter = new Demographics ();
$ret = $demoImporter->importFromDataStream ($data);
$demoImporter->__destruct();
unset ($demoImporter);
echo "Memory usage: " . number_format (memory_get_usage
()) . "\n";
return $ret;
}
}
}

class Socket
{
public function startPersistentListener ()
{
// Create a persistent listening socket connection
$this->create();
$this->setOption(SOL_SOCKET, SO_KEEPALIVE, 1);
$this->bind();
$this->listen();
$this->clients = array($this->sock);

while ($this->isAlive) {
// Create the list of connected clients
$this->createClients();

if ($this->select() < 1) {
// No clients - go to the next iteration
continue;
}

$this->accept();
$this->read();
}
}
}

set_time_limit (0);
new SocketListener ();
?>

I stripped out a lot of code, but this is the basics. I have a
listener class that extends the socket class. When the socket class
receives some data, it calls the processData() method - which creates
a new instance of the Demographics class. It destroys the instance
after it's finished. The line where I echo the memory usage indicates
that the memory continues to grow until it just explodes.

There's a lot more going on behind the scenes that *could* cause the
memory leak, but hopefully this will give you somewhere to go.

Thanks!
~Philip

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Re: Where"s my memory going?!

On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:

> well this sound clearly to me like you are not freeing resultsets
> you are not going to use anymore. In long scripts you have to take
> care of this. on short scripts you can be a bit weak on that,
> because the resultsets are closed and freed on script ending.
>
> assumed u r using MySQL are u using mysql_free_result($result)
>
> goog luck
>
> ralph_deffke@yahoo.de
>
>
> "Philip Thompson" wrote in message news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com
> ...
>> Hi all.
>>
>> I have a script that opens a socket, creates a persistent mysql
>> connection, and loops to receive data. When the amount of specified
>> data has been received, it calls a class which processes the data and
>> inserts it into the database. Each iteration, I unset/destruct that
>> class I call. However, the script keeps going up in memory and
>> eventually runs out, causing a fatal error. Any thoughts on where to
>> start to see where I'm losing my memory?
>>
>> Thanks in advance,
>> ~Philip

I am not using mysql_free_result(). Is that highly recommended by all?

Thanks,
~Philip

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Re: Where"s my memory going?!

Yes, that's the best way to clean up after yourself. And you really
should use that on anything you have sitting around daemon like.

Jeff

Philip Thompson wrote:
> On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:
>
>> well this sound clearly to me like you are not freeing resultsets you
>> are not going to use anymore. In long scripts you have to take care of
>> this. on short scripts you can be a bit weak on that, because the
>> resultsets are closed and freed on script ending.
>>
>> assumed u r using MySQL are u using mysql_free_result($result)
>>
>> goog luck
>>
>> ralph_deffke@yahoo.de
>>
>>
>> "Philip Thompson" wrote in message
>> news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com...
>>> Hi all.
>>>
>>> I have a script that opens a socket, creates a persistent mysql
>>> connection, and loops to receive data. When the amount of specified
>>> data has been received, it calls a class which processes the data and
>>> inserts it into the database. Each iteration, I unset/destruct that
>>> class I call. However, the script keeps going up in memory and
>>> eventually runs out, causing a fatal error. Any thoughts on where to
>>> start to see where I'm losing my memory?
>>>
>>> Thanks in advance,
>>> ~Philip
>
> I am not using mysql_free_result(). Is that highly recommended by all?
>
> Thanks,
> ~Philip

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Where"s my memory going?!

On Sep 28, 2009, at 4:40 PM, jeff brown wrote:

> Yes, that's the best way to clean up after yourself. And you really
> should use that on anything you have sitting around daemon like.
>
> Jeff
>
> Philip Thompson wrote:
>> On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:
>>> well this sound clearly to me like you are not freeing resultsets
>>> you are not going to use anymore. In long scripts you have to take
>>> care of this. on short scripts you can be a bit weak on that,
>>> because the resultsets are closed and freed on script ending.
>>>
>>> assumed u r using MySQL are u using mysql_free_result($result)
>>>
>>> goog luck
>>>
>>> ralph_deffke@yahoo.de
>>>
>>>
>>> "Philip Thompson" wrote in message news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com
>>> ...
>>>> Hi all.
>>>>
>>>> I have a script that opens a socket, creates a persistent mysql
>>>> connection, and loops to receive data. When the amount of specified
>>>> data has been received, it calls a class which processes the data
>>>> and
>>>> inserts it into the database. Each iteration, I unset/destruct that
>>>> class I call. However, the script keeps going up in memory and
>>>> eventually runs out, causing a fatal error. Any thoughts on where
>>>> to
>>>> start to see where I'm losing my memory?
>>>>
>>>> Thanks in advance,
>>>> ~Philip
>> I am not using mysql_free_result(). Is that highly recommended by
>> all?
>> Thanks,
>> ~Philip

I took your suggestions and made sure to clean up after myself. I'm
running into something that *appears* to be a bug with
mysql_free_result(). Here's a snippet of my db class.

class db {
function fetch ($sql, $assoc=false)
{
echo "\nMemory usage before query: " . number_format
(memory_get_usage ()) . "\n";
$resultSet = $this->query($sql);
echo "Memory usage after query: " . number_format
(memory_get_usage ()) . "\n";

if (!$assoc) { $result = $this->fetch_row($resultSet); }
else {
$result = $this->fetch_array($resultSet);
echo "Memory usage after fetch: " . number_format
(memory_get_usage ()) . "\n";
}

$this->freeResult($resultSet);
echo "Memory usage after free: " . number_format
(memory_get_usage ()) . "\n";

return $result;
}

function freeResult ($result)
{
if (is_resource ($result)) {
if (!mysql_free_result ($result)) { echo "Memory could
not be freed\n"; }
}
unset ($result); // For good measure
}

function fetch_row ($set) {
return mysql_fetch_row ($set);
}

function fetch_array ($set) {
return mysql_fetch_array ($set, MYSQL_ASSOC);
}
}

// I seem to be losing memory when I call this
$db->fetch($sql);
?>

The result I get with this is...

Memory usage before query: 6,406,548
Memory usage after query: 6,406,548
Memory usage after fetch: 6,406,548
Memory usage after free: 6,406,572

As you may notice, the memory actually goes UP after the *freeing* of
memory. Why is this happening?! What have I done wrong? Is this a bug?
Any thoughts would be appreciated.

Thanks,
~Philip


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Where"s my memory going?!

Philip Thompson wrote:
> On Sep 28, 2009, at 4:40 PM, jeff brown wrote:
>
>> Yes, that's the best way to clean up after yourself. And you really
>> should use that on anything you have sitting around daemon like.
>>
>> Jeff
>>
>> Philip Thompson wrote:
>>> On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:
>>>> well this sound clearly to me like you are not freeing resultsets
>>>> you are not going to use anymore. In long scripts you have to take
>>>> care of this. on short scripts you can be a bit weak on that,
>>>> because the resultsets are closed and freed on script ending.
>>>>
>>>> assumed u r using MySQL are u using mysql_free_result($result)
>>>>
>>>> goog luck
>>>>
>>>> ralph_deffke@yahoo.de
>>>>
>>>>
>>>> "Philip Thompson" wrote in message
>>>> news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com...
>>>>> Hi all.
>>>>>
>>>>> I have a script that opens a socket, creates a persistent mysql
>>>>> connection, and loops to receive data. When the amount of specified
>>>>> data has been received, it calls a class which processes the data and
>>>>> inserts it into the database. Each iteration, I unset/destruct that
>>>>> class I call. However, the script keeps going up in memory and
>>>>> eventually runs out, causing a fatal error. Any thoughts on where to
>>>>> start to see where I'm losing my memory?
>>>>>
>>>>> Thanks in advance,
>>>>> ~Philip
>>> I am not using mysql_free_result(). Is that highly recommended by all?
>>> Thanks,
>>> ~Philip
>
> I took your suggestions and made sure to clean up after myself. I'm
> running into something that *appears* to be a bug with
> mysql_free_result(). Here's a snippet of my db class.
>
> > class db {
> function fetch ($sql, $assoc=false)
> {
> echo "\nMemory usage before query: " . number_format
> (memory_get_usage ()) . "\n";
> $resultSet = $this->query($sql);
> echo "Memory usage after query: " . number_format
> (memory_get_usage ()) . "\n";
>
> if (!$assoc) { $result = $this->fetch_row($resultSet); }
> else {
> $result = $this->fetch_array($resultSet);
> echo "Memory usage after fetch: " . number_format
> (memory_get_usage ()) . "\n";
> }
>
/*

> $this->freeResult($resultSet);
*/
mysql_free_result($resultSet);

> echo "Memory usage after free: " . number_format
> (memory_get_usage ()) . "\n";
>
> return $result;
> }
>
> function freeResult ($result)
> {
> if (is_resource ($result)) {
> if (!mysql_free_result ($result)) { echo "Memory could not
> be freed\n"; }
> }
> unset ($result); // For good measure
> }
>
> function fetch_row ($set) {
> return mysql_fetch_row ($set);
> }
>
> function fetch_array ($set) {
> return mysql_fetch_array ($set, MYSQL_ASSOC);
> }
> }
>
> // I seem to be losing memory when I call this
> $db->fetch($sql);
> ?>
>
> The result I get with this is...
>
> Memory usage before query: 6,406,548
> Memory usage after query: 6,406,548
> Memory usage after fetch: 6,406,548
> Memory usage after free: 6,406,572
>
> As you may notice, the memory actually goes UP after the *freeing* of
> memory. Why is this happening?! What have I done wrong? Is this a bug?
> Any thoughts would be appreciated.
>
> Thanks,
> ~Philip
>

try the above...

jeff

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Where"s my memory going?!

On Sep 29, 2009, at 3:07 PM, jeff brown wrote:

> Philip Thompson wrote:
>> On Sep 28, 2009, at 4:40 PM, jeff brown wrote:
>>> Yes, that's the best way to clean up after yourself. And you
>>> really should use that on anything you have sitting around daemon
>>> like.
>>>
>>> Jeff
>>>
>>> Philip Thompson wrote:
>>>> On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:
>>>>> well this sound clearly to me like you are not freeing
>>>>> resultsets you are not going to use anymore. In long scripts you
>>>>> have to take care of this. on short scripts you can be a bit
>>>>> weak on that, because the resultsets are closed and freed on
>>>>> script ending.
>>>>>
>>>>> assumed u r using MySQL are u using mysql_free_result($result)
>>>>>
>>>>> goog luck
>>>>>
>>>>> ralph_deffke@yahoo.de
>>>>>
>>>>>
>>>>> "Philip Thompson" wrote in message news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com
>>>>> ...
>>>>>> Hi all.
>>>>>>
>>>>>> I have a script that opens a socket, creates a persistent mysql
>>>>>> connection, and loops to receive data. When the amount of
>>>>>> specified
>>>>>> data has been received, it calls a class which processes the
>>>>>> data and
>>>>>> inserts it into the database. Each iteration, I unset/destruct
>>>>>> that
>>>>>> class I call. However, the script keeps going up in memory and
>>>>>> eventually runs out, causing a fatal error. Any thoughts on
>>>>>> where to
>>>>>> start to see where I'm losing my memory?
>>>>>>
>>>>>> Thanks in advance,
>>>>>> ~Philip
>>>> I am not using mysql_free_result(). Is that highly recommended by
>>>> all?
>>>> Thanks,
>>>> ~Philip
>> I took your suggestions and made sure to clean up after myself. I'm
>> running into something that *appears* to be a bug with
>> mysql_free_result(). Here's a snippet of my db class.
>> >> class db {
>> function fetch ($sql, $assoc=false)
>> {
>> echo "\nMemory usage before query: " . number_format
>> (memory_get_usage ()) . "\n";
>> $resultSet = $this->query($sql);
>> echo "Memory usage after query: " . number_format
>> (memory_get_usage ()) . "\n";
>> if (!$assoc) { $result = $this->fetch_row($resultSet); }
>> else {
>> $result = $this->fetch_array($resultSet);
>> echo "Memory usage after fetch: " . number_format
>> (memory_get_usage ()) . "\n";
>> }
>>
> /*
>
>> $this->freeResult($resultSet);
> */
> mysql_free_result($resultSet);
>
>> echo "Memory usage after free: " . number_format
>> (memory_get_usage ()) . "\n";
>> return $result;
>> }
>> function freeResult ($result)
>> {
>> if (is_resource ($result)) {
>> if (!mysql_free_result ($result)) { echo "Memory could
>> not be freed\n"; }
>> }
>> unset ($result); // For good measure
>> }
>> function fetch_row ($set) {
>> return mysql_fetch_row ($set);
>> }
>> function fetch_array ($set) {
>> return mysql_fetch_array ($set, MYSQL_ASSOC);
>> }
>> }
>> // I seem to be losing memory when I call this
>> $db->fetch($sql);
>> ?>
>> The result I get with this is...
>> Memory usage before query: 6,406,548
>> Memory usage after query: 6,406,548
>> Memory usage after fetch: 6,406,548
>> Memory usage after free: 6,406,572
>> As you may notice, the memory actually goes UP after the *freeing*
>> of memory. Why is this happening?! What have I done wrong? Is this
>> a bug? Any thoughts would be appreciated.
>> Thanks,
>> ~Philip
>
> try the above...
>
> jeff

Unfortunately, the same result:

Memory usage before query: 6,524,676
Memory usage after query: 6,524,676
Memory usage after fetch: 6,524,676
Memory usage after free: 6,524,700

Memory usage before query: 6,524,792
Memory usage after query: 6,524,792
Memory usage after fetch: 6,524,792
Memory usage after free: 6,524,816

Memory usage before query: 6,524,932
Memory usage after query: 6,524,932
Memory usage after fetch: 6,524,932
Memory usage after free: 6,524,956

Each iteration shows a 24 byte difference between fetching the array
and freeing the result. Thoughts? This is baffling to me.

Thanks,
~Philip


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Where"s my memory going?!

On Sep 29, 2009, at 3:23 PM, Philip Thompson wrote:

> On Sep 29, 2009, at 3:07 PM, jeff brown wrote:
>
>> Philip Thompson wrote:
>>> On Sep 28, 2009, at 4:40 PM, jeff brown wrote:
>>>> Yes, that's the best way to clean up after yourself. And you
>>>> really should use that on anything you have sitting around daemon
>>>> like.
>>>>
>>>> Jeff
>>>>
>>>> Philip Thompson wrote:
>>>>> On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:
>>>>>> well this sound clearly to me like you are not freeing
>>>>>> resultsets you are not going to use anymore. In long scripts
>>>>>> you have to take care of this. on short scripts you can be a
>>>>>> bit weak on that, because the resultsets are closed and freed
>>>>>> on script ending.
>>>>>>
>>>>>> assumed u r using MySQL are u using mysql_free_result($result)
>>>>>>
>>>>>> goog luck
>>>>>>
>>>>>> ralph_deffke@yahoo.de
>>>>>>
>>>>>>
>>>>>> "Philip Thompson" wrote in message news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com
>>>>>> ...
>>>>>>> Hi all.
>>>>>>>
>>>>>>> I have a script that opens a socket, creates a persistent mysql
>>>>>>> connection, and loops to receive data. When the amount of
>>>>>>> specified
>>>>>>> data has been received, it calls a class which processes the
>>>>>>> data and
>>>>>>> inserts it into the database. Each iteration, I unset/destruct
>>>>>>> that
>>>>>>> class I call. However, the script keeps going up in memory and
>>>>>>> eventually runs out, causing a fatal error. Any thoughts on
>>>>>>> where to
>>>>>>> start to see where I'm losing my memory?
>>>>>>>
>>>>>>> Thanks in advance,
>>>>>>> ~Philip
>>>>> I am not using mysql_free_result(). Is that highly recommended
>>>>> by all?
>>>>> Thanks,
>>>>> ~Philip
>>> I took your suggestions and made sure to clean up after myself.
>>> I'm running into something that *appears* to be a bug with
>>> mysql_free_result(). Here's a snippet of my db class.
>>> >>> class db {
>>> function fetch ($sql, $assoc=false)
>>> {
>>> echo "\nMemory usage before query: " . number_format
>>> (memory_get_usage ()) . "\n";
>>> $resultSet = $this->query($sql);
>>> echo "Memory usage after query: " . number_format
>>> (memory_get_usage ()) . "\n";
>>> if (!$assoc) { $result = $this->fetch_row($resultSet); }
>>> else {
>>> $result = $this->fetch_array($resultSet);
>>> echo "Memory usage after fetch: " . number_format
>>> (memory_get_usage ()) . "\n";
>>> }
>>>
>> /*
>>
>>> $this->freeResult($resultSet);
>> */
>> mysql_free_result($resultSet);
>>
>>> echo "Memory usage after free: " . number_format
>>> (memory_get_usage ()) . "\n";
>>> return $result;
>>> }
>>> function freeResult ($result)
>>> {
>>> if (is_resource ($result)) {
>>> if (!mysql_free_result ($result)) { echo "Memory could
>>> not be freed\n"; }
>>> }
>>> unset ($result); // For good measure
>>> }
>>> function fetch_row ($set) {
>>> return mysql_fetch_row ($set);
>>> }
>>> function fetch_array ($set) {
>>> return mysql_fetch_array ($set, MYSQL_ASSOC);
>>> }
>>> }
>>> // I seem to be losing memory when I call this
>>> $db->fetch($sql);
>>> ?>
>>> The result I get with this is...
>>> Memory usage before query: 6,406,548
>>> Memory usage after query: 6,406,548
>>> Memory usage after fetch: 6,406,548
>>> Memory usage after free: 6,406,572
>>> As you may notice, the memory actually goes UP after the *freeing*
>>> of memory. Why is this happening?! What have I done wrong? Is this
>>> a bug? Any thoughts would be appreciated.
>>> Thanks,
>>> ~Philip
>>
>> try the above...
>>
>> jeff
>
> Unfortunately, the same result:
>
> Memory usage before query: 6,524,676
> Memory usage after query: 6,524,676
> Memory usage after fetch: 6,524,676
> Memory usage after free: 6,524,700
>
> Memory usage before query: 6,524,792
> Memory usage after query: 6,524,792
> Memory usage after fetch: 6,524,792
> Memory usage after free: 6,524,816
>
> Memory usage before query: 6,524,932
> Memory usage after query: 6,524,932
> Memory usage after fetch: 6,524,932
> Memory usage after free: 6,524,956
>
> Each iteration shows a 24 byte difference between fetching the array
> and freeing the result. Thoughts? This is baffling to me.
>
> Thanks,
> ~Philip

I did a little searching and found this article:

http://www.ibm.com/developerworks/opensource/library/os-php- v521/

He gives the same example I do with mysql_free_result() not
appropriately freeing the memory. He concludes with "we can assume
that mysql_query() is allocating memory incorrectly." However, he
never explains his thoughts on why or anything else. Thanks, Tracy.

So am I chasing a tangent? Should my focus be moved to mysql_query()
instead? Thoughts?

Thanks,
~Philip

PS... Yes, this is giving me a headache.

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Where"s my memory going?!

Philip Thompson wrote:
> On Sep 28, 2009, at 4:40 PM, jeff brown wrote:
>
>> Yes, that's the best way to clean up after yourself. And you really
>> should use that on anything you have sitting around daemon like.
>>
>> Jeff
>>
>> Philip Thompson wrote:
>>> On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:
>>>> well this sound clearly to me like you are not freeing resultsets
>>>> you are not going to use anymore. In long scripts you have to take
>>>> care of this. on short scripts you can be a bit weak on that,
>>>> because the resultsets are closed and freed on script ending.
>>>>
>>>> assumed u r using MySQL are u using mysql_free_result($result)
>>>>
>>>> goog luck
>>>>
>>>> ralph_deffke@yahoo.de
>>>>
>>>>
>>>> "Philip Thompson" wrote in message
>>>> news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com...
>>>>> Hi all.
>>>>>
>>>>> I have a script that opens a socket, creates a persistent mysql
>>>>> connection, and loops to receive data. When the amount of specified
>>>>> data has been received, it calls a class which processes the data and
>>>>> inserts it into the database. Each iteration, I unset/destruct that
>>>>> class I call. However, the script keeps going up in memory and
>>>>> eventually runs out, causing a fatal error. Any thoughts on where to
>>>>> start to see where I'm losing my memory?
>>>>>
>>>>> Thanks in advance,
>>>>> ~Philip
>>> I am not using mysql_free_result(). Is that highly recommended by all?
>>> Thanks,
>>> ~Philip
>
> I took your suggestions and made sure to clean up after myself. I'm
> running into something that *appears* to be a bug with
> mysql_free_result(). Here's a snippet of my db class.
>
> > class db {
> function fetch ($sql, $assoc=false)
> {
> echo "\nMemory usage before query: " . number_format
> (memory_get_usage ()) . "\n";
> $resultSet = $this->query($sql);
> echo "Memory usage after query: " . number_format
> (memory_get_usage ()) . "\n";
>
> if (!$assoc) { $result = $this->fetch_row($resultSet); }
> else {
> $result = $this->fetch_array($resultSet);
> echo "Memory usage after fetch: " . number_format
> (memory_get_usage ()) . "\n";
> }
>
> $this->freeResult($resultSet);
> echo "Memory usage after free: " . number_format
> (memory_get_usage ()) . "\n";
>
> return $result;
> }
>
> function freeResult ($result)
> {
> if (is_resource ($result)) {
> if (!mysql_free_result ($result)) { echo "Memory could not
> be freed\n"; }
> }
> unset ($result); // For good measure
> }
>
> function fetch_row ($set) {
> return mysql_fetch_row ($set);
> }
>
> function fetch_array ($set) {
> return mysql_fetch_array ($set, MYSQL_ASSOC);
> }
> }
>
> // I seem to be losing memory when I call this
> $db->fetch($sql);
> ?>
>
> The result I get with this is...
>
> Memory usage before query: 6,406,548
> Memory usage after query: 6,406,548
> Memory usage after fetch: 6,406,548
> Memory usage after free: 6,406,572
>
> As you may notice, the memory actually goes UP after the *freeing* of
> memory. Why is this happening?! What have I done wrong? Is this a bug?
> Any thoughts would be appreciated.
>

First off, my question would be, is your query actually working? Because I
would imagine that if you were getting results back from the DB, that the amount
of memory being used would increase between step 1 & 2.

Check to make sure that you are getting results.

> Thanks,
> ~Philip
>
>


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Where"s my memory going?!

On Sep 29, 2009, at 4:38 PM, Jim Lucas wrote:

> Philip Thompson wrote:
>> On Sep 28, 2009, at 4:40 PM, jeff brown wrote:
>>
>>> Yes, that's the best way to clean up after yourself. And you really
>>> should use that on anything you have sitting around daemon like.
>>>
>>> Jeff
>>>
>>> Philip Thompson wrote:
>>>> On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:
>>>>> well this sound clearly to me like you are not freeing resultsets
>>>>> you are not going to use anymore. In long scripts you have to take
>>>>> care of this. on short scripts you can be a bit weak on that,
>>>>> because the resultsets are closed and freed on script ending.
>>>>>
>>>>> assumed u r using MySQL are u using mysql_free_result($result)
>>>>>
>>>>> goog luck
>>>>>
>>>>> ralph_deffke@yahoo.de
>>>>>
>>>>>
>>>>> "Philip Thompson" wrote in message
>>>>> news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com...
>>>>>> Hi all.
>>>>>>
>>>>>> I have a script that opens a socket, creates a persistent mysql
>>>>>> connection, and loops to receive data. When the amount of
>>>>>> specified
>>>>>> data has been received, it calls a class which processes the
>>>>>> data and
>>>>>> inserts it into the database. Each iteration, I unset/destruct
>>>>>> that
>>>>>> class I call. However, the script keeps going up in memory and
>>>>>> eventually runs out, causing a fatal error. Any thoughts on
>>>>>> where to
>>>>>> start to see where I'm losing my memory?
>>>>>>
>>>>>> Thanks in advance,
>>>>>> ~Philip
>>>> I am not using mysql_free_result(). Is that highly recommended by
>>>> all?
>>>> Thanks,
>>>> ~Philip
>>
>> I took your suggestions and made sure to clean up after myself. I'm
>> running into something that *appears* to be a bug with
>> mysql_free_result(). Here's a snippet of my db class.
>>
>> >> class db {
>> function fetch ($sql, $assoc=false)
>> {
>> echo "\nMemory usage before query: " . number_format
>> (memory_get_usage ()) . "\n";
>> $resultSet = $this->query($sql);
>> echo "Memory usage after query: " . number_format
>> (memory_get_usage ()) . "\n";
>>
>> if (!$assoc) { $result = $this->fetch_row($resultSet); }
>> else {
>> $result = $this->fetch_array($resultSet);
>> echo "Memory usage after fetch: " . number_format
>> (memory_get_usage ()) . "\n";
>> }
>>
>> $this->freeResult($resultSet);
>> echo "Memory usage after free: " . number_format
>> (memory_get_usage ()) . "\n";
>>
>> return $result;
>> }
>>
>> function freeResult ($result)
>> {
>> if (is_resource ($result)) {
>> if (!mysql_free_result ($result)) { echo "Memory could not
>> be freed\n"; }
>> }
>> unset ($result); // For good measure
>> }
>>
>> function fetch_row ($set) {
>> return mysql_fetch_row ($set);
>> }
>>
>> function fetch_array ($set) {
>> return mysql_fetch_array ($set, MYSQL_ASSOC);
>> }
>> }
>>
>> // I seem to be losing memory when I call this
>> $db->fetch($sql);
>> ?>
>>
>> The result I get with this is...
>>
>> Memory usage before query: 6,406,548
>> Memory usage after query: 6,406,548
>> Memory usage after fetch: 6,406,548
>> Memory usage after free: 6,406,572
>>
>> As you may notice, the memory actually goes UP after the *freeing* of
>> memory. Why is this happening?! What have I done wrong? Is this a
>> bug?
>> Any thoughts would be appreciated.
>>
>
> First off, my question would be, is your query actually working?
> Because I
> would imagine that if you were getting results back from the DB,
> that the amount
> of memory being used would increase between step 1 & 2.
>
> Check to make sure that you are getting results.

I'm confident the queries are working (there's many of them and I know
the data they're returning is correct), but they may not always be
returning results. The memory value does change in some instances...

Memory usage before query: 5,138,372
Memory usage after query: 5,138,396
Memory usage after free: 5,138,556

This was one that use fetch_row() instead of fetch_array(), but the
same difference. I did some searching around and I think it's a bug in
PHP and/or Zend Memory Management engine. As I mentioned in a previous
post about mysql_query() not allocating memory appropriately, I
believe this could quite possibly be the case. Several people have
reported bugs similar to this... unfortunately, they are marked as
(erroneously?) bogus or said it has been fixed (when, IMO, it has not).

http://bugs.php.net/bug.php?id=40883
http://bugs.php.net/bug.php?id=28424
http://bugs.php.net/bug.php?id=41871

I'm using version 5.2.6 on Fedora 8 and the bug still appears to be
there. I think I'm going to take a different approach to fix this.
I'll create a shell script to loop and invoke the socket listener
script, and when it gathers data, call the import script. This way,
the php script(s) will end execution after each iteration and release
the memory. Is this reasonable?

This has been a long day. Thanks for your input. Any more thoughts are
welcome.

~Philip

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Where"s my memory going?!

Philip Thompson wrote:
> On Sep 29, 2009, at 4:38 PM, Jim Lucas wrote:
>
>> Philip Thompson wrote:
>>> On Sep 28, 2009, at 4:40 PM, jeff brown wrote:
>>>
>>>> Yes, that's the best way to clean up after yourself. And you really
>>>> should use that on anything you have sitting around daemon like.
>>>>
>>>> Jeff
>>>>
>>>> Philip Thompson wrote:
>>>>> On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:
>>>>>> well this sound clearly to me like you are not freeing resultsets
>>>>>> you are not going to use anymore. In long scripts you have to take
>>>>>> care of this. on short scripts you can be a bit weak on that,
>>>>>> because the resultsets are closed and freed on script ending.
>>>>>>
>>>>>> assumed u r using MySQL are u using mysql_free_result($result)
>>>>>>
>>>>>> goog luck
>>>>>>
>>>>>> ralph_deffke@yahoo.de
>>>>>>
>>>>>>
>>>>>> "Philip Thompson" wrote in message
>>>>>> news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com...
>>>>>>> Hi all.
>>>>>>>
>>>>>>> I have a script that opens a socket, creates a persistent mysql
>>>>>>> connection, and loops to receive data. When the amount of specified
>>>>>>> data has been received, it calls a class which processes the data
>>>>>>> and
>>>>>>> inserts it into the database. Each iteration, I unset/destruct that
>>>>>>> class I call. However, the script keeps going up in memory and
>>>>>>> eventually runs out, causing a fatal error. Any thoughts on where to
>>>>>>> start to see where I'm losing my memory?
>>>>>>>
>>>>>>> Thanks in advance,
>>>>>>> ~Philip
>>>>> I am not using mysql_free_result(). Is that highly recommended by all?
>>>>> Thanks,
>>>>> ~Philip
>>>
>>> I took your suggestions and made sure to clean up after myself. I'm
>>> running into something that *appears* to be a bug with
>>> mysql_free_result(). Here's a snippet of my db class.
>>>
>>> >>> class db {
>>> function fetch ($sql, $assoc=false)
>>> {
>>> echo "\nMemory usage before query: " . number_format
>>> (memory_get_usage ()) . "\n";
>>> $resultSet = $this->query($sql);
>>> echo "Memory usage after query: " . number_format
>>> (memory_get_usage ()) . "\n";
>>>
>>> if (!$assoc) { $result = $this->fetch_row($resultSet); }
>>> else {
>>> $result = $this->fetch_array($resultSet);
>>> echo "Memory usage after fetch: " . number_format
>>> (memory_get_usage ()) . "\n";
>>> }
>>>
>>> $this->freeResult($resultSet);
>>> echo "Memory usage after free: " . number_format
>>> (memory_get_usage ()) . "\n";
>>>
>>> return $result;
>>> }
>>>
>>> function freeResult ($result)
>>> {
>>> if (is_resource ($result)) {
>>> if (!mysql_free_result ($result)) { echo "Memory could not
>>> be freed\n"; }
>>> }
>>> unset ($result); // For good measure
>>> }
>>>
>>> function fetch_row ($set) {
>>> return mysql_fetch_row ($set);
>>> }
>>>
>>> function fetch_array ($set) {
>>> return mysql_fetch_array ($set, MYSQL_ASSOC);
>>> }
>>> }
>>>
>>> // I seem to be losing memory when I call this
>>> $db->fetch($sql);
>>> ?>
>>>
>>> The result I get with this is...
>>>
>>> Memory usage before query: 6,406,548
>>> Memory usage after query: 6,406,548
>>> Memory usage after fetch: 6,406,548
>>> Memory usage after free: 6,406,572
>>>
>>> As you may notice, the memory actually goes UP after the *freeing* of
>>> memory. Why is this happening?! What have I done wrong? Is this a bug?
>>> Any thoughts would be appreciated.
>>>
>>
>> First off, my question would be, is your query actually working?
>> Because I
>> would imagine that if you were getting results back from the DB, that
>> the amount
>> of memory being used would increase between step 1 & 2.
>>
>> Check to make sure that you are getting results.
>
> I'm confident the queries are working (there's many of them and I know
> the data they're returning is correct), but they may not always be
> returning results. The memory value does change in some instances...
>
> Memory usage before query: 5,138,372
> Memory usage after query: 5,138,396
> Memory usage after free: 5,138,556
>
> This was one that use fetch_row() instead of fetch_array(), but the same
> difference. I did some searching around and I think it's a bug in PHP
> and/or Zend Memory Management engine. As I mentioned in a previous post
> about mysql_query() not allocating memory appropriately, I believe this
> could quite possibly be the case. Several people have reported bugs
> similar to this... unfortunately, they are marked as (erroneously?)
> bogus or said it has been fixed (when, IMO, it has not).
>
> http://bugs.php.net/bug.php?id=40883
> http://bugs.php.net/bug.php?id=28424
> http://bugs.php.net/bug.php?id=41871
>
> I'm using version 5.2.6 on Fedora 8 and the bug still appears to be
> there. I think I'm going to take a different approach to fix this. I'll
> create a shell script to loop and invoke the socket listener script, and
> when it gathers data, call the import script. This way, the php
> script(s) will end execution after each iteration and release the
> memory. Is this reasonable?
>
> This has been a long day. Thanks for your input. Any more thoughts are
> welcome.
>
> ~Philip
>

Doesn't PHP only give back a "pointer" back from MySQL when it runs a query?
Then what PHP does is uses that "pointer" to mysql to request the next row of
results?

Couldn't you use the unbuffered-query function for this?

http://php.net/mysql-unbuffered-query

Maybe this will help?


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Where"s my memory going?!

On Tue, Sep 29, 2009 at 6:51 PM, Jim Lucas wrote:
> Philip Thompson wrote:
>> On Sep 29, 2009, at 4:38 PM, Jim Lucas wrote:
>>
>>> Philip Thompson wrote:
>>>> On Sep 28, 2009, at 4:40 PM, jeff brown wrote:
>>>>
>>>>> Yes, that's the best way to clean up after yourself.  And you re=
ally
>>>>> should use that on anything you have sitting around daemon like.
>>>>>
>>>>> Jeff
>>>>>
>>>>> Philip Thompson wrote:
>>>>>> On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:
>>>>>>> well this sound clearly to me like you are not freeing resultsets
>>>>>>> you are not going to use anymore. In long scripts you have to take
>>>>>>> care of this. on short scripts you can be a bit weak on that,
>>>>>>> because the resultsets are closed and freed on script ending.
>>>>>>>
>>>>>>> assumed u r using MySQL are u using mysql_free_result($result)
>>>>>>>
>>>>>>> goog luck
>>>>>>>
>>>>>>> ralph_deffke@yahoo.de
>>>>>>>
>>>>>>>
>>>>>>> "Philip Thompson" wrote in message
>>>>>>> news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com...
>>>>>>>> Hi all.
>>>>>>>>
>>>>>>>> I have a script that opens a socket, creates a persistent mysql
>>>>>>>> connection, and loops to receive data. When the amount of specifie=
d
>>>>>>>> data has been received, it calls a class which processes the data
>>>>>>>> and
>>>>>>>> inserts it into the database. Each iteration, I unset/destruct tha=
t
>>>>>>>> class I call. However, the script keeps going up in memory and
>>>>>>>> eventually runs out, causing a fatal error. Any thoughts on where =
to
>>>>>>>> start to see where I'm losing my memory?
>>>>>>>>
>>>>>>>> Thanks in advance,
>>>>>>>> ~Philip
>>>>>> I am not using mysql_free_result(). Is that highly recommended by al=
l?
>>>>>> Thanks,
>>>>>> ~Philip
>>>>
>>>> I took your suggestions and made sure to clean up after myself. I'm
>>>> running into something that *appears* to be a bug with
>>>> mysql_free_result(). Here's a snippet of my db class.
>>>>
>>>> >>>> class db {
>>>>    function fetch ($sql, $assoc=3Dfalse)
>>>>    {
>>>>        echo "\nMemory usage before query: " . numb=
er_format
>>>> (memory_get_usage ()) . "\n";
>>>>        $resultSet =3D $this->query($sql);
>>>>        echo "Memory usage after  query: " . n=
umber_format
>>>> (memory_get_usage ()) . "\n";
>>>>
>>>>        if (!$assoc) { $result =3D $this->fetch_row=
($resultSet); }
>>>>        else {
>>>>            $result =3D $this->fetch_arra=
y($resultSet);
>>>>            echo "Memory usage after =C2=
=A0fetch: " . number_format
>>>> (memory_get_usage ()) . "\n";
>>>>        }
>>>>
>>>>        $this->freeResult($resultSet);
>>>>        echo "Memory usage after   free: " . n=
umber_format
>>>> (memory_get_usage ()) . "\n";
>>>>
>>>>        return $result;
>>>>    }
>>>>
>>>>    function freeResult ($result)
>>>>    {
>>>>        if (is_resource ($result)) {
>>>>            if (!mysql_free_result ($resu=
lt)) { echo "Memory could not
>>>> be freed\n"; }
>>>>        }
>>>>        unset ($result); // For good measure
>>>>    }
>>>>
>>>>    function fetch_row ($set) {
>>>>        return mysql_fetch_row ($set);
>>>>    }
>>>>
>>>>    function fetch_array ($set) {
>>>>        return mysql_fetch_array ($set, MYSQL_ASSOC=
);
>>>>    }
>>>> }
>>>>
>>>> // I seem to be losing memory when I call this
>>>> $db->fetch($sql);
>>>> ?>
>>>>
>>>> The result I get with this is...
>>>>
>>>> Memory usage before query: 6,406,548
>>>> Memory usage after  query: 6,406,548
>>>> Memory usage after  fetch: 6,406,548
>>>> Memory usage after   free: 6,406,572
>>>>
>>>> As you may notice, the memory actually goes UP after the *freeing* of
>>>> memory. Why is this happening?! What have I done wrong? Is this a bug?
>>>> Any thoughts would be appreciated.
>>>>
>>>
>>> First off, my question would be, is your query actually working?
>>> Because I
>>> would imagine that if you were getting results back from the DB, that
>>> the amount
>>> of memory being used would increase between step 1 & 2.
>>>
>>> Check to make sure that you are getting results.
>>
>> I'm confident the queries are working (there's many of them and I know
>> the data they're returning is correct), but they may not always be
>> returning results. The memory value does change in some instances...
>>
>> Memory usage before query: 5,138,372
>> Memory usage after  query: 5,138,396
>> Memory usage after   free: 5,138,556
>>
>> This was one that use fetch_row() instead of fetch_array(), but the same
>> difference. I did some searching around and I think it's a bug in PHP
>> and/or Zend Memory Management engine. As I mentioned in a previous post
>> about mysql_query() not allocating memory appropriately, I believe this
>> could quite possibly be the case. Several people have reported bugs
>> similar to this... unfortunately, they are marked as (erroneously?)
>> bogus or said it has been fixed (when, IMO, it has not).
>>
>> http://bugs.php.net/bug.php?id=3D40883
>> http://bugs.php.net/bug.php?id=3D28424
>> http://bugs.php.net/bug.php?id=3D41871
>>
>> I'm using version 5.2.6 on Fedora 8 and the bug still appears to be
>> there. I think I'm going to take a different approach to fix this. I'll
>> create a shell script to loop and invoke the socket listener script, and
>> when it gathers data, call the import script. This way, the php
>> script(s) will end execution after each iteration and release the
>> memory. Is this reasonable?
>>
>> This has been a long day. Thanks for your input. Any more thoughts are
>> welcome.
>>
>> ~Philip
>>
>
> Doesn't PHP only give back a "pointer" back from MySQL when it runs a que=
ry?
> Then what PHP does is uses that "pointer" to mysql to request the next ro=
w of
> results?
>
> Couldn't you use the unbuffered-query function for this?
>
> http://php.net/mysql-unbuffered-query
>
> Maybe this will help?
>
>
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

What version of PHP (older versions like to leak memory, if MY memory
serves me right, haha)? Have you tried running with 5.3.0 and using
the garbage collection?

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Where"s my memory going?!

On Sep 29, 2009, at 5:51 PM, Jim Lucas wrote:

> Philip Thompson wrote:
>> On Sep 29, 2009, at 4:38 PM, Jim Lucas wrote:
>>
>>> Philip Thompson wrote:
>>>> On Sep 28, 2009, at 4:40 PM, jeff brown wrote:
>>>>
>>>>> Yes, that's the best way to clean up after yourself. And you
>>>>> really
>>>>> should use that on anything you have sitting around daemon like.
>>>>>
>>>>> Jeff
>>>>>
>>>>> Philip Thompson wrote:
>>>>>> On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:
>>>>>>> well this sound clearly to me like you are not freeing
>>>>>>> resultsets
>>>>>>> you are not going to use anymore. In long scripts you have to
>>>>>>> take
>>>>>>> care of this. on short scripts you can be a bit weak on that,
>>>>>>> because the resultsets are closed and freed on script ending.
>>>>>>>
>>>>>>> assumed u r using MySQL are u using mysql_free_result($result)
>>>>>>>
>>>>>>> goog luck
>>>>>>>
>>>>>>> ralph_deffke@yahoo.de
>>>>>>>
>>>>>>>
>>>>>>> "Philip Thompson" wrote in message
>>>>>>> news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com...
>>>>>>>> Hi all.
>>>>>>>>
>>>>>>>> I have a script that opens a socket, creates a persistent mysql
>>>>>>>> connection, and loops to receive data. When the amount of
>>>>>>>> specified
>>>>>>>> data has been received, it calls a class which processes the
>>>>>>>> data
>>>>>>>> and
>>>>>>>> inserts it into the database. Each iteration, I unset/
>>>>>>>> destruct that
>>>>>>>> class I call. However, the script keeps going up in memory and
>>>>>>>> eventually runs out, causing a fatal error. Any thoughts on
>>>>>>>> where to
>>>>>>>> start to see where I'm losing my memory?
>>>>>>>>
>>>>>>>> Thanks in advance,
>>>>>>>> ~Philip
>>>>>> I am not using mysql_free_result(). Is that highly recommended
>>>>>> by all?
>>>>>> Thanks,
>>>>>> ~Philip
>>>>
>>>> I took your suggestions and made sure to clean up after myself. I'm
>>>> running into something that *appears* to be a bug with
>>>> mysql_free_result(). Here's a snippet of my db class.
>>>>
>>>> >>>> class db {
>>>> function fetch ($sql, $assoc=false)
>>>> {
>>>> echo "\nMemory usage before query: " . number_format
>>>> (memory_get_usage ()) . "\n";
>>>> $resultSet = $this->query($sql);
>>>> echo "Memory usage after query: " . number_format
>>>> (memory_get_usage ()) . "\n";
>>>>
>>>> if (!$assoc) { $result = $this->fetch_row($resultSet); }
>>>> else {
>>>> $result = $this->fetch_array($resultSet);
>>>> echo "Memory usage after fetch: " . number_format
>>>> (memory_get_usage ()) . "\n";
>>>> }
>>>>
>>>> $this->freeResult($resultSet);
>>>> echo "Memory usage after free: " . number_format
>>>> (memory_get_usage ()) . "\n";
>>>>
>>>> return $result;
>>>> }
>>>>
>>>> function freeResult ($result)
>>>> {
>>>> if (is_resource ($result)) {
>>>> if (!mysql_free_result ($result)) { echo "Memory could
>>>> not
>>>> be freed\n"; }
>>>> }
>>>> unset ($result); // For good measure
>>>> }
>>>>
>>>> function fetch_row ($set) {
>>>> return mysql_fetch_row ($set);
>>>> }
>>>>
>>>> function fetch_array ($set) {
>>>> return mysql_fetch_array ($set, MYSQL_ASSOC);
>>>> }
>>>> }
>>>>
>>>> // I seem to be losing memory when I call this
>>>> $db->fetch($sql);
>>>> ?>
>>>>
>>>> The result I get with this is...
>>>>
>>>> Memory usage before query: 6,406,548
>>>> Memory usage after query: 6,406,548
>>>> Memory usage after fetch: 6,406,548
>>>> Memory usage after free: 6,406,572
>>>>
>>>> As you may notice, the memory actually goes UP after the
>>>> *freeing* of
>>>> memory. Why is this happening?! What have I done wrong? Is this a
>>>> bug?
>>>> Any thoughts would be appreciated.
>>>>
>>>
>>> First off, my question would be, is your query actually working?
>>> Because I
>>> would imagine that if you were getting results back from the DB,
>>> that
>>> the amount
>>> of memory being used would increase between step 1 & 2.
>>>
>>> Check to make sure that you are getting results.
>>
>> I'm confident the queries are working (there's many of them and I
>> know
>> the data they're returning is correct), but they may not always be
>> returning results. The memory value does change in some instances...
>>
>> Memory usage before query: 5,138,372
>> Memory usage after query: 5,138,396
>> Memory usage after free: 5,138,556
>>
>> This was one that use fetch_row() instead of fetch_array(), but the
>> same
>> difference. I did some searching around and I think it's a bug in PHP
>> and/or Zend Memory Management engine. As I mentioned in a previous
>> post
>> about mysql_query() not allocating memory appropriately, I believe
>> this
>> could quite possibly be the case. Several people have reported bugs
>> similar to this... unfortunately, they are marked as (erroneously?)
>> bogus or said it has been fixed (when, IMO, it has not).
>>
>> http://bugs.php.net/bug.php?id=40883
>> http://bugs.php.net/bug.php?id=28424
>> http://bugs.php.net/bug.php?id=41871
>>
>> I'm using version 5.2.6 on Fedora 8 and the bug still appears to be
>> there. I think I'm going to take a different approach to fix this.
>> I'll
>> create a shell script to loop and invoke the socket listener
>> script, and
>> when it gathers data, call the import script. This way, the php
>> script(s) will end execution after each iteration and release the
>> memory. Is this reasonable?
>>
>> This has been a long day. Thanks for your input. Any more thoughts
>> are
>> welcome.
>>
>> ~Philip
>>
>
> Doesn't PHP only give back a "pointer" back from MySQL when it runs
> a query?
> Then what PHP does is uses that "pointer" to mysql to request the
> next row of
> results?
>
> Couldn't you use the unbuffered-query function for this?
>
> http://php.net/mysql-unbuffered-query
>
> Maybe this will help?

I'm not exactly sure, Jim. I'll have to look into this. Thanks for the
suggestion.

~Philip


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Where"s my memory going?!

--Apple-Mail-696--1014448774
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=us-ascii;
format=flowed;
delsp=yes

On Sep 29, 2009, at 6:15 PM, Eddie Drapkin wrote:

> On Tue, Sep 29, 2009 at 6:51 PM, Jim Lucas wrote:
>> Philip Thompson wrote:
>>> On Sep 29, 2009, at 4:38 PM, Jim Lucas wrote:
>>>
>>>> Philip Thompson wrote:
>>>>> On Sep 28, 2009, at 4:40 PM, jeff brown wrote:
>>>>>
>>>>>> Yes, that's the best way to clean up after yourself. And you
>>>>>> really
>>>>>> should use that on anything you have sitting around daemon like.
>>>>>>
>>>>>> Jeff
>>>>>>
>>>>>> Philip Thompson wrote:
>>>>>>> On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:
>>>>>>>> well this sound clearly to me like you are not freeing
>>>>>>>> resultsets
>>>>>>>> you are not going to use anymore. In long scripts you have to
>>>>>>>> take
>>>>>>>> care of this. on short scripts you can be a bit weak on that,
>>>>>>>> because the resultsets are closed and freed on script ending.
>>>>>>>>
>>>>>>>> assumed u r using MySQL are u using mysql_free_result($result)
>>>>>>>>
>>>>>>>> goog luck
>>>>>>>>
>>>>>>>> ralph_deffke@yahoo.de
>>>>>>>>
>>>>>>>>
>>>>>>>> "Philip Thompson" wrote in message
>>>>>>>> news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com...
>>>>>>>>> Hi all.
>>>>>>>>>
>>>>>>>>> I have a script that opens a socket, creates a persistent
>>>>>>>>> mysql
>>>>>>>>> connection, and loops to receive data. When the amount of
>>>>>>>>> specified
>>>>>>>>> data has been received, it calls a class which processes the
>>>>>>>>> data
>>>>>>>>> and
>>>>>>>>> inserts it into the database. Each iteration, I unset/
>>>>>>>>> destruct that
>>>>>>>>> class I call. However, the script keeps going up in memory and
>>>>>>>>> eventually runs out, causing a fatal error. Any thoughts on
>>>>>>>>> where to
>>>>>>>>> start to see where I'm losing my memory?
>>>>>>>>>
>>>>>>>>> Thanks in advance,
>>>>>>>>> ~Philip
>>>>>>> I am not using mysql_free_result(). Is that highly recommended
>>>>>>> by all?
>>>>>>> Thanks,
>>>>>>> ~Philip
>>>>>
>>>>> I took your suggestions and made sure to clean up after myself.
>>>>> I'm
>>>>> running into something that *appears* to be a bug with
>>>>> mysql_free_result(). Here's a snippet of my db class.
>>>>>
>>>>> >>>>> class db {
>>>>> function fetch ($sql, $assoc=false)
>>>>> {
>>>>> echo "\nMemory usage before query: " . number_format
>>>>> (memory_get_usage ()) . "\n";
>>>>> $resultSet = $this->query($sql);
>>>>> echo "Memory usage after query: " . number_format
>>>>> (memory_get_usage ()) . "\n";
>>>>>
>>>>> if (!$assoc) { $result = $this->fetch_row($resultSet); }
>>>>> else {
>>>>> $result = $this->fetch_array($resultSet);
>>>>> echo "Memory usage after fetch: " . number_format
>>>>> (memory_get_usage ()) . "\n";
>>>>> }
>>>>>
>>>>> $this->freeResult($resultSet);
>>>>> echo "Memory usage after free: " . number_format
>>>>> (memory_get_usage ()) . "\n";
>>>>>
>>>>> return $result;
>>>>> }
>>>>>
>>>>> function freeResult ($result)
>>>>> {
>>>>> if (is_resource ($result)) {
>>>>> if (!mysql_free_result ($result)) { echo "Memory
>>>>> could not
>>>>> be freed\n"; }
>>>>> }
>>>>> unset ($result); // For good measure
>>>>> }
>>>>>
>>>>> function fetch_row ($set) {
>>>>> return mysql_fetch_row ($set);
>>>>> }
>>>>>
>>>>> function fetch_array ($set) {
>>>>> return mysql_fetch_array ($set, MYSQL_ASSOC);
>>>>> }
>>>>> }
>>>>>
>>>>> // I seem to be losing memory when I call this
>>>>> $db->fetch($sql);
>>>>> ?>
>>>>>
>>>>> The result I get with this is...
>>>>>
>>>>> Memory usage before query: 6,406,548
>>>>> Memory usage after query: 6,406,548
>>>>> Memory usage after fetch: 6,406,548
>>>>> Memory usage after free: 6,406,572
>>>>>
>>>>> As you may notice, the memory actually goes UP after the
>>>>> *freeing* of
>>>>> memory. Why is this happening?! What have I done wrong? Is this
>>>>> a bug?
>>>>> Any thoughts would be appreciated.
>>>>>
>>>>
>>>> First off, my question would be, is your query actually working?
>>>> Because I
>>>> would imagine that if you were getting results back from the DB,
>>>> that
>>>> the amount
>>>> of memory being used would increase between step 1 & 2.
>>>>
>>>> Check to make sure that you are getting results.
>>>
>>> I'm confident the queries are working (there's many of them and I
>>> know
>>> the data they're returning is correct), but they may not always be
>>> returning results. The memory value does change in some instances...
>>>
>>> Memory usage before query: 5,138,372
>>> Memory usage after query: 5,138,396
>>> Memory usage after free: 5,138,556
>>>
>>> This was one that use fetch_row() instead of fetch_array(), but
>>> the same
>>> difference. I did some searching around and I think it's a bug in
>>> PHP
>>> and/or Zend Memory Management engine. As I mentioned in a previous
>>> post
>>> about mysql_query() not allocating memory appropriately, I believe
>>> this
>>> could quite possibly be the case. Several people have reported bugs
>>> similar to this... unfortunately, they are marked as (erroneously?)
>>> bogus or said it has been fixed (when, IMO, it has not).
>>>
>>> http://bugs.php.net/bug.php?id=40883
>>> http://bugs.php.net/bug.php?id=28424
>>> http://bugs.php.net/bug.php?id=41871
>>>
>>> I'm using version 5.2.6 on Fedora 8 and the bug still appears to be
>>> there. I think I'm going to take a different approach to fix this.
>>> I'll
>>> create a shell script to loop and invoke the socket listener
>>> script, and
>>> when it gathers data, call the import script. This way, the php
>>> script(s) will end execution after each iteration and release the
>>> memory. Is this reasonable?
>>>
>>> This has been a long day. Thanks for your input. Any more thoughts
>>> are
>>> welcome.
>>>
>>> ~Philip
>>>
>>
>> Doesn't PHP only give back a "pointer" back from MySQL when it runs
>> a query?
>> Then what PHP does is uses that "pointer" to mysql to request the
>> next row of
>> results?
>>
>> Couldn't you use the unbuffered-query function for this?
>>
>> http://php.net/mysql-unbuffered-query
>>
>> Maybe this will help?
>
> What version of PHP (older versions like to leak memory, if MY memory
> serves me right, haha)? Have you tried running with 5.3.0 and using
> the garbage collection?

5.2.6. I've not looked at 5.3, but I suppose it couldn't hurt to do
some testing. =D Thank you.

~Philip



--Apple-Mail-696--1014448774--

Re: Where"s my memory going?!

Hi Philip,

before u start running arround and taking ur hand on major changes I would
like u to concider the following:

the way u use the memory_get_usage() is incomplete use the function like
memory_get_usage( true ) if this is set the REAL SIZE OF MEMORY ALLOCATED
FROM THE SYSTEM is shown.

some posters gave u little winks, like has the garbage collection been run?

freeing the results is important, it means the memory is available for other
data, but it does not necesarily mean it is reported just a step later as
free! u are running on concurrent os.

the only way u can test that is in real world. clean the whole application
with the free_result_set and see the effect.

I'm personaly not shure if the memory allocated for result sets for MySQL
are reported as used by PHP anyway. It might be that the amount of memory is
not shown as PHP used even if u set real_usage = true. it is something to
test.

I think the way u tested the effect of freeing the results is just wrong.

hope that helps

ralph_deffke@yahoo.de



u came up with the problem that ur server is running out of memory.
"Philip Thompson" wrote in message
news:098D2158-9199-4983-AEC4-6FFB06C1C14B@gmail.com...
> On Sep 29, 2009, at 6:15 PM, Eddie Drapkin wrote:
>
> > On Tue, Sep 29, 2009 at 6:51 PM, Jim Lucas wrote:
> >> Philip Thompson wrote:
> >>> On Sep 29, 2009, at 4:38 PM, Jim Lucas wrote:
> >>>
> >>>> Philip Thompson wrote:
> >>>>> On Sep 28, 2009, at 4:40 PM, jeff brown wrote:
> >>>>>
> >>>>>> Yes, that's the best way to clean up after yourself. And you
> >>>>>> really
> >>>>>> should use that on anything you have sitting around daemon like.
> >>>>>>
> >>>>>> Jeff
> >>>>>>
> >>>>>> Philip Thompson wrote:
> >>>>>>> On Sep 28, 2009, at 4:27 PM, Ralph Deffke wrote:
> >>>>>>>> well this sound clearly to me like you are not freeing
> >>>>>>>> resultsets
> >>>>>>>> you are not going to use anymore. In long scripts you have to
> >>>>>>>> take
> >>>>>>>> care of this. on short scripts you can be a bit weak on that,
> >>>>>>>> because the resultsets are closed and freed on script ending.
> >>>>>>>>
> >>>>>>>> assumed u r using MySQL are u using mysql_free_result($result)
> >>>>>>>>
> >>>>>>>> goog luck
> >>>>>>>>
> >>>>>>>> ralph_deffke@yahoo.de
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> "Philip Thompson" wrote in message
> >>>>>>>> news:9C0B9C4C-5E64-4519-862B-8A3E1DA4DE4C@gmail.com...
> >>>>>>>>> Hi all.
> >>>>>>>>>
> >>>>>>>>> I have a script that opens a socket, creates a persistent
> >>>>>>>>> mysql
> >>>>>>>>> connection, and loops to receive data. When the amount of
> >>>>>>>>> specified
> >>>>>>>>> data has been received, it calls a class which processes the
> >>>>>>>>> data
> >>>>>>>>> and
> >>>>>>>>> inserts it into the database. Each iteration, I unset/
> >>>>>>>>> destruct that
> >>>>>>>>> class I call. However, the script keeps going up in memory and
> >>>>>>>>> eventually runs out, causing a fatal error. Any thoughts on
> >>>>>>>>> where to
> >>>>>>>>> start to see where I'm losing my memory?
> >>>>>>>>>
> >>>>>>>>> Thanks in advance,
> >>>>>>>>> ~Philip
> >>>>>>> I am not using mysql_free_result(). Is that highly recommended
> >>>>>>> by all?
> >>>>>>> Thanks,
> >>>>>>> ~Philip
> >>>>>
> >>>>> I took your suggestions and made sure to clean up after myself.
> >>>>> I'm
> >>>>> running into something that *appears* to be a bug with
> >>>>> mysql_free_result(). Here's a snippet of my db class.
> >>>>>
> >>>>> > >>>>> class db {
> >>>>> function fetch ($sql, $assoc=false)
> >>>>> {
> >>>>> echo "\nMemory usage before query: " . number_format
> >>>>> (memory_get_usage ()) . "\n";
> >>>>> $resultSet = $this->query($sql);
> >>>>> echo "Memory usage after query: " . number_format
> >>>>> (memory_get_usage ()) . "\n";
> >>>>>
> >>>>> if (!$assoc) { $result = $this->fetch_row($resultSet); }
> >>>>> else {
> >>>>> $result = $this->fetch_array($resultSet);
> >>>>> echo "Memory usage after fetch: " . number_format
> >>>>> (memory_get_usage ()) . "\n";
> >>>>> }
> >>>>>
> >>>>> $this->freeResult($resultSet);
> >>>>> echo "Memory usage after free: " . number_format
> >>>>> (memory_get_usage ()) . "\n";
> >>>>>
> >>>>> return $result;
> >>>>> }
> >>>>>
> >>>>> function freeResult ($result)
> >>>>> {
> >>>>> if (is_resource ($result)) {
> >>>>> if (!mysql_free_result ($result)) { echo "Memory
> >>>>> could not
> >>>>> be freed\n"; }
> >>>>> }
> >>>>> unset ($result); // For good measure
> >>>>> }
> >>>>>
> >>>>> function fetch_row ($set) {
> >>>>> return mysql_fetch_row ($set);
> >>>>> }
> >>>>>
> >>>>> function fetch_array ($set) {
> >>>>> return mysql_fetch_array ($set, MYSQL_ASSOC);
> >>>>> }
> >>>>> }
> >>>>>
> >>>>> // I seem to be losing memory when I call this
> >>>>> $db->fetch($sql);
> >>>>> ?>
> >>>>>
> >>>>> The result I get with this is...
> >>>>>
> >>>>> Memory usage before query: 6,406,548
> >>>>> Memory usage after query: 6,406,548
> >>>>> Memory usage after fetch: 6,406,548
> >>>>> Memory usage after free: 6,406,572
> >>>>>
> >>>>> As you may notice, the memory actually goes UP after the
> >>>>> *freeing* of
> >>>>> memory. Why is this happening?! What have I done wrong? Is this
> >>>>> a bug?
> >>>>> Any thoughts would be appreciated.
> >>>>>
> >>>>
> >>>> First off, my question would be, is your query actually working?
> >>>> Because I
> >>>> would imagine that if you were getting results back from the DB,
> >>>> that
> >>>> the amount
> >>>> of memory being used would increase between step 1 & 2.
> >>>>
> >>>> Check to make sure that you are getting results.
> >>>
> >>> I'm confident the queries are working (there's many of them and I
> >>> know
> >>> the data they're returning is correct), but they may not always be
> >>> returning results. The memory value does change in some instances...
> >>>
> >>> Memory usage before query: 5,138,372
> >>> Memory usage after query: 5,138,396
> >>> Memory usage after free: 5,138,556
> >>>
> >>> This was one that use fetch_row() instead of fetch_array(), but
> >>> the same
> >>> difference. I did some searching around and I think it's a bug in
> >>> PHP
> >>> and/or Zend Memory Management engine. As I mentioned in a previous
> >>> post
> >>> about mysql_query() not allocating memory appropriately, I believe
> >>> this
> >>> could quite possibly be the case. Several people have reported bugs
> >>> similar to this... unfortunately, they are marked as (erroneously?)
> >>> bogus or said it has been fixed (when, IMO, it has not).
> >>>
> >>> http://bugs.php.net/bug.php?id=40883
> >>> http://bugs.php.net/bug.php?id=28424
> >>> http://bugs.php.net/bug.php?id=41871
> >>>
> >>> I'm using version 5.2.6 on Fedora 8 and the bug still appears to be
> >>> there. I think I'm going to take a different approach to fix this.
> >>> I'll
> >>> create a shell script to loop and invoke the socket listener
> >>> script, and
> >>> when it gathers data, call the import script. This way, the php
> >>> script(s) will end execution after each iteration and release the
> >>> memory. Is this reasonable?
> >>>
> >>> This has been a long day. Thanks for your input. Any more thoughts
> >>> are
> >>> welcome.
> >>>
> >>> ~Philip
> >>>
> >>
> >> Doesn't PHP only give back a "pointer" back from MySQL when it runs
> >> a query?
> >> Then what PHP does is uses that "pointer" to mysql to request the
> >> next row of
> >> results?
> >>
> >> Couldn't you use the unbuffered-query function for this?
> >>
> >> http://php.net/mysql-unbuffered-query
> >>
> >> Maybe this will help?
> >
> > What version of PHP (older versions like to leak memory, if MY memory
> > serves me right, haha)? Have you tried running with 5.3.0 and using
> > the garbage collection?
>
> 5.2.6. I've not looked at 5.3, but I suppose it couldn't hurt to do
> some testing. =D Thank you.
>
> ~Philip
>
>
>



--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php