can"t get authorization

I have a strange behaviour.
If I type in my browser (Firefox) this address:

http://?login=3D&password=3D

I can enter and get authorized. If I use the same line from my
useragent or even from wget, the site diverts me to an error page and
doesn't let me in.


here are some snippets taken from an etherreal session:

FROM THE BROWSER:

Hypertext Transfer Protocol
GET /alice/portal/service/entry.do?service=3Dsms&from=3Dalice HTTP/1.1\=
r\n
Request Method: GET
Request URI: /alice/portal/service/entry.do?service=3Dsms&from=3Dal=
ice
Request Version: HTTP/1.1
Host: www.rossoalice.virgilio.it\r\n
User-Agent: Mozilla/5.0 (X11; U; Linux i686; it-IT; rv:1.7.12)
Gecko/20051010 Firefox/1.0.4 (Ubuntu package 1.0.7)\r\n
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=3 D0.=
9,text/plain;q=3D0.8,image/png,*/*;q=3D0.5\r\n
Accept-Language: it,it-it;q=3D0.8,en-us;q=3D0.5,en;q=3D0.3\r\n
Accept-Encoding: gzip,deflate\r\n
Accept-Charset: ISO-8859-1,utf-8;q=3D0.7,*;q=3D0.7\r\n
Keep-Alive: 300\r\n
Connection: keep-alive\r\n
Cookie: kp=3D80.23.61.194.1128499183660059;
JSESSIONID=3DDMFScOLfRWokil5pC6FEGboZvuNaHfoNAT4Z29uDNuVw9Yg BAisW!-10470466=
49;
ra11=3DY3rnqDn1VDOpGHbeyTBiBq3vYp/M4EEbMNTwGKnvWL+K7LIeSN921 T70GTUSD7797AIH=
Yx2UP69uRZVPPbYea8MbAa3MF92HxekcW+nIrbyz8HD2fGb
\r\n

........
.......
Hypertext Transfer Protocol
HTTP/1.1 200 OK\r\n
Request Version: HTTP/1.1
Response Code: 200
Date: Tue, 11 Oct 2005 18:18:43 GMT\r\n
Server: WebLogic Server 8.1 SP2 Fri Dec 5 15:01:51 PST 2003 316284\r\n
Keep-Alive: timeout=3D20, max=3D19999\r\n
Connection: Keep-Alive\r\n
Transfer-Encoding: chunked\r\n
Content-Type: text/html; charset=3DISO-8859-1\r\n
Content-Language: it\r\n
\r\n
HTTP chunked response
Data chunk (757 octets)
Chunk size: 757 octets
Data (757 bytes)

0000 3c 21 44 4f 43 54 59 50 45 20 48 54 4d 4c 20 50 ************************************************************ ***************=
***************

FROM LWP

Hypertext Transfer Protocol
GET /alice/portal/service/entry.do?service=3Dsms HTTP/1.1\r\n
Request Method: GET
Request URI: /alice/portal/service/entry.do?service=3Dsms
Request Version: HTTP/1.1
Keep-Alive: 300\r\n
Connection: Keep-Alive\r\n
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=3 D0.=
9,text/plain;q=3D0.8,image/png,*/*;q=3D0.5\r\n
Accept-Charset: ISO-8859-1,utf-8;q=3D0.7,*;q=3D0.7\r\n
Accept-Encoding: gzip,deflate\r\n
Accept-Language: it,it-it;q=3D0.8,en-us;q=3D0.5,en;q=3D0.3\r\n
Host: www.rossoalice.virgilio.it\r\n
User-Agent: Mozilla/5.0 (X11; U; Linux i686; it-IT; rv:1.7.12)
Gecko/20051010 Firefox/1.0.4 (Ubuntu package 1.0.7)\r\n
Cookie: JSESSIONID=3DDNCBa1HlP8oxk6ciTG1K3N73PfHlCTdh4qcPSbTAaVNRqCz CKR=
ee!-1720153440;
kp=3D82.57.176.190.1129136834217268;
ra-1=3Da=3D1813739:o=3DA:t=3D0:s=3DC:p=3D1:m=3D1:;
ra11=3D"Y3rnqDn1VDOpGHbeyTBiBquqoglSPuqACQeoLjgq/4piyYhMCUsn BZtkQmyFplnC+ZG=
sjPtqLk3n
\r\n

.........................
.....................

Hypertext Transfer Protocol
HTTP/1.1 200 OK\r\n
Request Version: HTTP/1.1
Response Code: 200
Date: Wed, 12 Oct 2005 17:07:14 GMT\r\n
Server: WebLogic Server 8.1 SP2 Fri Dec 5 15:01:51 PST 2003 316284\r\n
Keep-Alive: timeout=3D20, max=3D20000\r\n
Connection: Keep-Alive\r\n
Transfer-Encoding: chunked\r\n
Content-Type: text/html; charset=3DISO-8859-1\r\n
Content-Language: it\r\n
\r\n
HTTP chunked response
[Unreassembled Packet: HTTP]

No. Time Source Destination Protocol In=
fo
30 0.710024 81.74.238.6 192.168.1.3 HTTP =20
Continuation or non-HTTP traffic


********************************************

Where is the problem? I am getting lost..
Thanks to all

Re: can"t get authorization

I have put here: http://socrates.homedns.org/lwp/
the two complete ethereal sessions (one from browser and the other
from LWP). As you can see the browser reaches the correct page, while
LWP is diverted to an error page...

Re: can"t get authorization

Fabio Marzocca writes:

> I have put here: http://socrates.homedns.org/lwp/
> the two complete ethereal sessions (one from browser and the other
> from LWP).

Those are pretty unreadable. At least try to use the "Follow TCP
stream" function of ethereal so it is possible to read what the data
passed between the peers are.

It is also more likely that you get a useful response if you point out
how the requests are different and why you think it matters.

Regards,
Gisle

Re: can"t get authorization

On 14 Oct 2005 10:06:21 -0700, Gisle Aas wrote:
> Those are pretty unreadable. At least try to use the "Follow TCP
> stream" function of ethereal so it is possible to read what the data
> passed between the peers are.

You are right, sorry. So I have replaced the 2 files with a snippet
from ethereal sessions, filtered by "Follow TCP stream". I have
reported only the pasrt of interest..
Same url:
http://socrates.homedns.org/lwp/

Re: can"t get authorization

What I have realized is that the session from lwp is divided into 3
sub-sessions, each one preceded by its own DNS standard query. So
each time the lwp session make a DNS query, it gets 3 different
addresses.
The web session is made by just one query thru all, with the same IP addres=
s..

I have realized this by looking at the "follow TCP stream" analysis.
From LWP I have 3 streams, from the browser only one.

Re: can"t get authorization

More on this...
I have seen that standard query from the browser is Type: AAAA (IPv6
address). Maybe LWP does not support IPv6?

Re: can"t get authorization

Fabio Marzocca writes:

> I have seen that standard query from the browser is Type: AAAA (IPv6
> address). Maybe LWP does not support IPv6?

It doesn't.

Regards,
Gisle

Re: can"t get authorization

On 21 Oct 2005 05:26:48 -0700, Gisle Aas wrote:
> It doesn't.

But looking better at the flow, I am convinced it is not a matter on IPv6.

Summarizing, this is the stream:

1) get the page, sending userid and pwd
2) user agent is diverted to an authentication page
3) user agent is diverted back to the site.

With Firefox, points 1 and 3 are on the same TCP stream. With LWP they
are 3 separated TCP streams..

Re: can"t get authorization

Fabio Marzocca writes:

> On 21 Oct 2005 05:26:48 -0700, Gisle Aas wrote:
> > It doesn't.
>
> But looking better at the flow, I am convinced it is not a matter on IPv6.
>
> Summarizing, this is the stream:
>
> 1) get the page, sending userid and pwd
> 2) user agent is diverted to an authentication page
> 3) user agent is diverted back to the site.
>
> With Firefox, points 1 and 3 are on the same TCP stream. With LWP they
> are 3 separated TCP streams..

You can get LWP to use a single TCP connection by turing on the
keep_alive feature. Done something like this:

my $ua = LWP::UserAgent->new(keep_alive => 2);

Regards,
Gisle

Re: can"t get authorization

On 21 Oct 2005 06:38:14 -0700, Gisle Aas wrote:
> You can get LWP to use a single TCP connection by turing on the
> keep_alive feature. Done something like this:

Ok, thanks. I did that and now I have a full TCP stream on both tests,
but no way to reach the same page. Only firefox reaches the page,
while LWP is diverted to an error page.

I have put both TCP stream analysis here: http://socrates.homedns.org/lwp/
Can anybody have a look??

Thanks in advance.

Fabio

Re: can"t get authorization

On 10/24/05, Fabio Marzocca wrote:
>
> Firefox sends this cookie at the very first GET (see the last line):
>
> GET /alice/portal/service/login/entry.do?service=3Dsms&from=3Dal ice&login=
=3D...........&password=3D....
> HTTP/1.1
> Host: www.rossoalice.virgilio.it
> User-Agent: Mozilla/5.0 (X11; U; Linux i686; it-IT; rv:1.7.12)
> Gecko/20051010 Firefox/1.0.4 (Ubuntu package 1.0.7)
> Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=3 D0.9,=
text/plain;q=3D0.8,image/png,*/*;q=3D0.5
> Accept-Language: it,it-it;q=3D0.8,en-us;q=3D0.5,en;q=3D0.3
> Accept-Encoding: gzip,deflate
> Accept-Charset: ISO-8859-1,utf-8;q=3D0.7,*;q=3D0.7
> Keep-Alive: 300
> Connection: keep-alive
> Cookie: kp=3D82.57.176.190.1129216047341631
>


I have investigated some more on this, and I have discovered that the
servers sends a JSESSIONID cookie to the browser (how to handle it
with LWP?).
More, I need to add a special cookie to my $ua->get(), but I don't
know how to manually add a cookie to a request..