whats wrong with perl threads

whats wrong with perl threads

am 14.07.2005 02:23:01 von Anon

I've written a perl script to grab (wget) and tranverse (HTML-Parser) html
files .. in order to download some files ... I'm using the 'threads' module
to spawn off new threads, which then 'system( "wget " ) etc .. the files.
However, even after a thread ended it's routine the memory doesn't seem to
be released. Memory (real and virtual) builds up to huge size's ...
Clutching at straws i first throught it could be down to use cygwin, and
it's memory allocation ... but i've tried it on a linux platform, and
recieved the same results. Here's some snippets ...

------
while ( $threadsNUse == $totalThreads ) {
verb( 0, "($threadsNUse) ($totalThreads) games downloading ...
pausing" );
sleep( 5 );
}

{
lock( $threadsNUse );
$threadsNUse++;
}

my $thr = threads->create( \&grabGame );
# $thr->detach();
verb( 3, "created thread ($threadsNUse)" );
}

sub grabGame {
my $link = $url;
verb( 0, "[ DL ] game ( $gamesDL ) : link ( $link )" );
if ( $options{w} ) {
system( "wget -T $timeout -c -q -P $romDir/ $link" );
{
lock( $threadsNUse );
$threadsNUse--;
}
{
lock( $gamesDL );
$gamesDL++;
}
verb( 3, "thread stopped" );
} else {
# were not using threads, downloading one by one ...
system( "wget -T $timeout -c -P $romDir/ $link" );
$gamesDL++;
}
return;
}

Quick explanation .. $threadsNUse is obviously the number of threads been
created, $totalThreads the maximum allowed. $threadsNUse is incremented on
every threaded created, and as you can see, decrement as a thread in nearer
the end. The while loop used to simply pause if the max threads have been
allocated, and wait until space is available. Anyhow ... Why does the
threads not release its resources when the subroutine ends? ...

cheers for any help you can give ... :-) ...