Perl threads + HTTPS = Crash :(

Hi,

I have a program in perl which do several HTTP requests, so I
implemented some threads to get it running more fast, in general I use
5 threads at the same time.

The program work perfectly when i'm requesting only HTTP pages with
multiple threads, however when I change HTTP to HTTPS with multiple
threads the program crash (segmentation fault / access violation). If
I use HTTPS and only one thread the program work perfectly. :(

That's looks like really strange for me, which come to my mind is that
HTTPS is not thread safe in perl?

Some extra informations...

$ perl -v

This is perl, v5.8.8 built for i486-linux-gnu-thread-multi

I have SSL modules instaled...

libcrypt-ssleay-perl 0.51-5
libio-socket-ssl-perl 1.01-1
libnet-ssleay-perl 1.30-1

Module versions....

'AutoLoader.pm' => '5.63 from /usr/local/share/perl/5.8.8/
AutoLoader.pm'
'Carp.pm' => '1.04 from /usr/share/perl/5.8/Carp.pm'
'Carp/Heavy.pm' => '/usr/share/perl/5.8/Carp/Heavy.pm'
'Config.pm' => '/usr/lib/perl/5.8/Config.pm'
'Config_heavy.pl' => '/usr/lib/perl/5.8/Config_heavy.pl'
'DynaLoader.pm' => '1.04 from /usr/lib/perl/5.8/DynaLoader.pm'
'Exporter.pm' => '5.58 from /usr/share/perl/5.8/Exporter.pm'
'Exporter/Heavy.pm' => '5.58 from /usr/share/perl/5.8/Exporter/
Heavy.pm'
'Getopt/Long.pm' => '2.35 from /usr/share/perl/5.8/Getopt/Long.pm'
'HTTP/Cookies.pm' => '1.39 from /usr/share/perl5/HTTP/Cookies.pm'
'HTTP/Cookies/Netscape.pm' => '1.26 from /usr/share/perl5/HTTP/Cookies/
Netscape.pm'
'HTTP/Date.pm' => '1.47 from /usr/share/perl5/HTTP/Date.pm'
'HTTP/Headers.pm' => '1.64 from /usr/share/perl5/HTTP/Headers.pm'
'HTTP/Headers/Util.pm' => '1.13 from /usr/share/perl5/HTTP/Headers/
Util.pm'
'HTTP/Message.pm' => '1.57 from /usr/share/perl5/HTTP/Message.pm'
'HTTP/Request.pm' => '1.40 from /usr/share/perl5/HTTP/Request.pm'
'HTTP/Response.pm' => '1.53 from /usr/share/perl5/HTTP/Response.pm'
'HTTP/Status.pm' => '1.28 from /usr/share/perl5/HTTP/Status.pm'
'IO.pm' => '1.22 from /usr/lib/perl/5.8/IO.pm'
'IO/Handle.pm' => '1.25 from /usr/lib/perl/5.8/IO/Handle.pm'
'LWP.pm' => '5.805 from /usr/share/perl5/LWP.pm'
'LWP/Debug.pm' => '/usr/share/perl5/LWP/Debug.pm'
'LWP/MemberMixin.pm' => '/usr/share/perl5/LWP/MemberMixin.pm'
'LWP/Protocol.pm' => '1.43 from /usr/share/perl5/LWP/Protocol.pm'
'LWP/UserAgent.pm' => '2.033 from /usr/share/perl5/LWP/UserAgent.pm'
'SelectSaver.pm' => '1.01 from /usr/share/perl/5.8/SelectSaver.pm'
'SelfLoader.pm' => '1.0904 from /usr/share/perl/5.8/SelfLoader.pm'
'Symbol.pm' => '1.06 from /usr/share/perl/5.8/Symbol.pm'
'Term/Cap.pm' => '1.09 from /usr/share/perl/5.8/Term/Cap.pm'
'Term/ReadKey.pm' => '2.30 from /usr/lib/perl5/Term/ReadKey.pm'
'Term/ReadLine.pm' => '1.02 from /usr/share/perl/5.8/Term/ReadLine.pm'
'Term/ReadLine/Perl.pm' => '1.0302 from /usr/local/share/perl/5.8.4/
Term/ReadLine/Perl.pm'
'Term/ReadLine/readline.pm' => '1.0302 from /usr/local/share/perl/
5.8.4/Term/ReadLine/readline.pm'
'Time/Local.pm' => '1.11 from /usr/share/perl/5.8/Time/Local.pm'
'URI.pm' => '1.35 from /usr/share/perl5/URI.pm'
'URI/Escape.pm' => '3.28 from /usr/share/perl5/URI/Escape.pm'
'XSLoader.pm' => '0.06 from /usr/lib/perl/5.8/XSLoader.pm'
'attributes.pm' => '0.06 from /usr/share/perl/5.8/attributes.pm'
'constant.pm' => '1.05 from /usr/share/perl/5.8/constant.pm'
'integer.pm' => '1.00 from /usr/share/perl/5.8/integer.pm'
'overload.pm' => '1.04 from /usr/share/perl/5.8/overload.pm'
'perl5db.pl' => '1.28 from /usr/share/perl/5.8/perl5db.pl'
'strict.pm' => '1.03 from /usr/share/perl/5.8/strict.pm'
'threads.pm' => '1.07 from /usr/lib/perl/5.8/threads.pm'
'threads/shared.pm' => '0.94 from /usr/lib/perl/5.8/threads/shared.pm'
'vars.pm' => '1.01 from /usr/share/perl/5.8/vars.pm'
'warnings.pm' => '1.05 from /usr/share/perl/5.8/warnings.pm'
'warnings/register.pm' => '1.01 from /usr/share/perl/5.8/warnings/
register.pm'

If I run the program with multiple threads and HTTPS inside perl -d in
Linux I get this error:

$perl -d program.pl
Attempt to free unreferenced scalar: SV 0xa579768, Perl interpreter:
0xa4154e0 during global destruction.
*** glibc detected *** corrupted double-linked list: 0x089dc180 ***
Aborted.

In Windows I get this one:

Unhandled exception in SSLEAVY32.DLL: 0xC0000005: Access Violation.

A disasm part of this exception is:
01EC7FCB push 2A7h
01EC7FD0 push 1ED5990h
01EC7FD5 push 0Ch
01EC7FD7 push 9
01EC7FD9 mov dword ptr [esp+18h],eax
01EC7FDD call 01ECB17A
01EC7FE2 mov eax,dword ptr [esp+1Ch]
01EC7FE6 mov esi,dword ptr [eax+20h] <- BREAK here
01EC7FE9 lea ecx,[esp+14h]
01EC7FED push ecx
01EC7FEE mov dword ptr [eax+20h],0
01EC7FF5 mov edx,dword ptr [esp+20h]
01EC7FF9 push 1EC8030h
01EC7FFE push edx
01EC7FFF call 01ECB564
01EC8004 mov eax,dword ptr [esp+28h]
01EC8008 push 2ACh
01EC800D push 1ED5990h
01EC8012 push 0Ch
01EC8014 push 0Ah
01EC8016 mov dword ptr [eax+20h],esi
01EC8019 call 01ECB17A
01EC801E add esp,2Ch
01EC8021 pop esi
01EC8022 add esp,0Ch
01EC8025 ret
01EC8026 nop
01EC8027 nop
01EC8028 nop
01EC8029 nop

My SSL libs in windows are:

libeay32.dll 908 KB (929.792 bytes)
SSLeay32.dll 192 KB (196.608 bytes)

They do not have versions, so I pasted the size.

I do my requests like this:

$req = new HTTP::Request GET => "http://$host/$url";

Or for HTTPS:

$req = new HTTP::Request GET => "https://$host/$url";

Is this a know problem in perl? How to fix it?

Thank you a lot.

Regards,

Re: Perl threads + HTTPS = Crash :(

no one? :(

On 15 out, 00:27, Dan wrote:
> Hi,
>
> I have a program in perl which do several HTTP requests, so I
> implemented some threads to get it running more fast, in general I use
> 5 threads at the same time.
>
> The program work perfectly when i'm requesting only HTTP pages with
> multiple threads, however when I change HTTP to HTTPS with multiple
> threads the program crash (segmentation fault / access violation). If
> I use HTTPS and only one thread the program work perfectly. :(
>
> That's looks like really strange for me, which come to my mind is that
> HTTPS is not thread safe in perl?
>
> Some extra informations...
>
> $ perl -v
>
> This is perl, v5.8.8 built for i486-linux-gnu-thread-multi
>
> I have SSL modules instaled...
>
> libcrypt-ssleay-perl 0.51-5
> libio-socket-ssl-perl 1.01-1
> libnet-ssleay-perl 1.30-1
>
> Module versions....
>
> 'AutoLoader.pm' => '5.63 from /usr/local/share/perl/5.8.8/
> AutoLoader.pm'
> 'Carp.pm' => '1.04 from /usr/share/perl/5.8/Carp.pm'
> 'Carp/Heavy.pm' => '/usr/share/perl/5.8/Carp/Heavy.pm'
> 'Config.pm' => '/usr/lib/perl/5.8/Config.pm'
> 'Config_heavy.pl' => '/usr/lib/perl/5.8/Config_heavy.pl'
> 'DynaLoader.pm' => '1.04 from /usr/lib/perl/5.8/DynaLoader.pm'
> 'Exporter.pm' => '5.58 from /usr/share/perl/5.8/Exporter.pm'
> 'Exporter/Heavy.pm' => '5.58 from /usr/share/perl/5.8/Exporter/
> Heavy.pm'
> 'Getopt/Long.pm' => '2.35 from /usr/share/perl/5.8/Getopt/Long.pm'
> 'HTTP/Cookies.pm' => '1.39 from /usr/share/perl5/HTTP/Cookies.pm'
> 'HTTP/Cookies/Netscape.pm' => '1.26 from /usr/share/perl5/HTTP/Cookies/
> Netscape.pm'
> 'HTTP/Date.pm' => '1.47 from /usr/share/perl5/HTTP/Date.pm'
> 'HTTP/Headers.pm' => '1.64 from /usr/share/perl5/HTTP/Headers.pm'
> 'HTTP/Headers/Util.pm' => '1.13 from /usr/share/perl5/HTTP/Headers/
> Util.pm'
> 'HTTP/Message.pm' => '1.57 from /usr/share/perl5/HTTP/Message.pm'
> 'HTTP/Request.pm' => '1.40 from /usr/share/perl5/HTTP/Request.pm'
> 'HTTP/Response.pm' => '1.53 from /usr/share/perl5/HTTP/Response.pm'
> 'HTTP/Status.pm' => '1.28 from /usr/share/perl5/HTTP/Status.pm'
> 'IO.pm' => '1.22 from /usr/lib/perl/5.8/IO.pm'
> 'IO/Handle.pm' => '1.25 from /usr/lib/perl/5.8/IO/Handle.pm'
> 'LWP.pm' => '5.805 from /usr/share/perl5/LWP.pm'
> 'LWP/Debug.pm' => '/usr/share/perl5/LWP/Debug.pm'
> 'LWP/MemberMixin.pm' => '/usr/share/perl5/LWP/MemberMixin.pm'
> 'LWP/Protocol.pm' => '1.43 from /usr/share/perl5/LWP/Protocol.pm'
> 'LWP/UserAgent.pm' => '2.033 from /usr/share/perl5/LWP/UserAgent.pm'
> 'SelectSaver.pm' => '1.01 from /usr/share/perl/5.8/SelectSaver.pm'
> 'SelfLoader.pm' => '1.0904 from /usr/share/perl/5.8/SelfLoader.pm'
> 'Symbol.pm' => '1.06 from /usr/share/perl/5.8/Symbol.pm'
> 'Term/Cap.pm' => '1.09 from /usr/share/perl/5.8/Term/Cap.pm'
> 'Term/ReadKey.pm' => '2.30 from /usr/lib/perl5/Term/ReadKey.pm'
> 'Term/ReadLine.pm' => '1.02 from /usr/share/perl/5.8/Term/ReadLine.pm'
> 'Term/ReadLine/Perl.pm' => '1.0302 from /usr/local/share/perl/5.8.4/
> Term/ReadLine/Perl.pm'
> 'Term/ReadLine/readline.pm' => '1.0302 from /usr/local/share/perl/
> 5.8.4/Term/ReadLine/readline.pm'
> 'Time/Local.pm' => '1.11 from /usr/share/perl/5.8/Time/Local.pm'
> 'URI.pm' => '1.35 from /usr/share/perl5/URI.pm'
> 'URI/Escape.pm' => '3.28 from /usr/share/perl5/URI/Escape.pm'
> 'XSLoader.pm' => '0.06 from /usr/lib/perl/5.8/XSLoader.pm'
> 'attributes.pm' => '0.06 from /usr/share/perl/5.8/attributes.pm'
> 'constant.pm' => '1.05 from /usr/share/perl/5.8/constant.pm'
> 'integer.pm' => '1.00 from /usr/share/perl/5.8/integer.pm'
> 'overload.pm' => '1.04 from /usr/share/perl/5.8/overload.pm'
> 'perl5db.pl' => '1.28 from /usr/share/perl/5.8/perl5db.pl'
> 'strict.pm' => '1.03 from /usr/share/perl/5.8/strict.pm'
> 'threads.pm' => '1.07 from /usr/lib/perl/5.8/threads.pm'
> 'threads/shared.pm' => '0.94 from /usr/lib/perl/5.8/threads/shared.pm'
> 'vars.pm' => '1.01 from /usr/share/perl/5.8/vars.pm'
> 'warnings.pm' => '1.05 from /usr/share/perl/5.8/warnings.pm'
> 'warnings/register.pm' => '1.01 from /usr/share/perl/5.8/warnings/
> register.pm'
>
> If I run the program with multiple threads and HTTPS inside perl -d in
> Linux I get this error:
>
> $perl -d program.pl
> Attempt to free unreferenced scalar: SV 0xa579768, Perl interpreter:
> 0xa4154e0 during global destruction.
> *** glibc detected *** corrupted double-linked list: 0x089dc180 ***
> Aborted.
>
> In Windows I get this one:
>
> Unhandled exception in SSLEAVY32.DLL: 0xC0000005: Access Violation.
>
> A disasm part of this exception is:
> 01EC7FCB push 2A7h
> 01EC7FD0 push 1ED5990h
> 01EC7FD5 push 0Ch
> 01EC7FD7 push 9
> 01EC7FD9 mov dword ptr [esp+18h],eax
> 01EC7FDD call 01ECB17A
> 01EC7FE2 mov eax,dword ptr [esp+1Ch]
> 01EC7FE6 mov esi,dword ptr [eax+20h] <- BREAK here
> 01EC7FE9 lea ecx,[esp+14h]
> 01EC7FED push ecx
> 01EC7FEE mov dword ptr [eax+20h],0
> 01EC7FF5 mov edx,dword ptr [esp+20h]
> 01EC7FF9 push 1EC8030h
> 01EC7FFE push edx
> 01EC7FFF call 01ECB564
> 01EC8004 mov eax,dword ptr [esp+28h]
> 01EC8008 push 2ACh
> 01EC800D push 1ED5990h
> 01EC8012 push 0Ch
> 01EC8014 push 0Ah
> 01EC8016 mov dword ptr [eax+20h],esi
> 01EC8019 call 01ECB17A
> 01EC801E add esp,2Ch
> 01EC8021 pop esi
> 01EC8022 add esp,0Ch
> 01EC8025 ret
> 01EC8026 nop
> 01EC8027 nop
> 01EC8028 nop
> 01EC8029 nop
>
> My SSL libs in windows are:
>
> libeay32.dll 908 KB (929.792 bytes)
> SSLeay32.dll 192 KB (196.608 bytes)
>
> They do not have versions, so I pasted the size.
>
> I do my requests like this:
>
> $req = new HTTP::Request GET => "http://$host/$url";
>
> Or for HTTPS:
>
> $req = new HTTP::Request GET => "https://$host/$url";
>
> Is this a know problem in perl? How to fix it?
>
> Thank you a lot.
>
> Regards,

Re: Perl threads + HTTPS = Crash :(

Dan wrote:

> Hi,
>
> I have a program in perl which do several HTTP requests, so I
> implemented some threads to get it running more fast, in general I use
> 5 threads at the same time.
>
> The program work perfectly when i'm requesting only HTTP pages with
> multiple threads, however when I change HTTP to HTTPS with multiple
> threads the program crash (segmentation fault / access violation). If
> I use HTTPS and only one thread the program work perfectly. :(
>
> That's looks like really strange for me, which come to my mind is that
> HTTPS is not thread safe in perl?


I don't know what could cause that, but have you tried using
LWP::Parallel::UserAgent?

http://search.cpan.org/~marclang/ParallelUserAgent-2.57/lib/ LWP/Parallel/UserAgent.pm


--
Brian Wakem

Re: Perl threads + HTTPS = Crash :(

Quoth Dan :
>
> I have a program in perl which do several HTTP requests, so I
> implemented some threads to get it running more fast, in general I use
> 5 threads at the same time.
>
> The program work perfectly when i'm requesting only HTTP pages with
> multiple threads, however when I change HTTP to HTTPS with multiple
> threads the program crash (segmentation fault / access violation). If
> I use HTTPS and only one thread the program work perfectly. :(
>
> That's looks like really strange for me, which come to my mind is that
> HTTPS is not thread safe in perl?

It looks to me as though Crypt::SSLeay is not thread-safe. OpenSSL
itself can be thread-safe, but Crypt::SSLeay doesn't appear to call the
functions required to set up multithreaded access. Note that this is
just from a quick glance through the code: I don't have any real
knowledge of either OpenSSL or Crypt::SSLeay's internals. You could try
filing a bug against Crypt::SSLeay.

Ben