LWP gets 400, Firefox gets a 200.

I'm using LWP to get a pdf file via a form submission in a GET.

Seems very standard, I get the URLs from LiveHTTPHeaders and reconstruct
from there.

There's five or six steps leading up to this last step, logging in,
collecting cookies and variables and whatnot.
Then this last step fails with a "400 Bad Request".
(URLs have been modified to protect possible sensitive stuff).

$url =3D
URI->new("https://ooga.bogus.com/foo/bar;jsessionid=3D$jsess ionId");
$url->query_form(
requestor =3D> 'AGE',
doctype =3D> 'Statement',
docid =3D> $docid,
token =3D> $token,
);
$response =3D $agent->get($url);

The funny thing is that the server replies with a page that gives me
back the URL that I constructed above, which should have gotten the PDF
file, which can then be pasted into the location line of Firefox and it
works.

Here's the constructed URL's __form__:

https://ooga.bogus.com/foo/bar;jsessionid=3D000YTYxBDdFsZG5u D:-1?requestor
=3DAGE&doctype=3DStatement&docid=3DN1NzM0LTY3MzEtWEpBMS0zRkF BQS0zMzkwOTctMT=
MyN
Tg2LTYAc5LTMtMS0wLQ%3D%3D&token=3Dd77c1fac91asdfkljad67d5

What could be different about the two processes?


Andy Ford
314-955-6647
andyford@agedwards.com



------------------------------------------------------------ ---------------=
----------
A.G. Edwards & Sons' outgoing and incoming e-mails are electronically
archived and subject to review and/or disclosure to someone other=20
than the recipient.

------------------------------------------------------------ ---------------=
----------

RE: LWP gets 400, Firefox gets a 200.

The site could use javascript to surreptitiously change your request. =
=0AI've also seen sites that require your referrer to be set to something=
specific before they give you what you want. =0A-----=
Original Message----- From: Ford, Andy [mailto:andyford@AGEDWARDSco=
m] Sent: Thursday, July 06, 2006 3:41 PM To: libwww@perlorg=
Subject: LWP gets 400, Firefox gets a 200. =0AWhat could be=
different about the two processes? ----------------------------=
------------- ***************************************** ***************=
*********** * This e-mail is intended only for the addresse=
e named above. =0AAs this e-mail may contain confidential or privileged=
information, if you are not the named addressee, you are not authoriz=
ed to retain, read, copy or disseminate this message or any part of=0D=
=0Ait. =0A**************************************** ***************=
************ *

RE: LWP gets 400, Firefox gets a 200.

On Thu, 6 Jul 2006, Worenklein, David, GCM wrote:

Its a long shot, but i found the %D%D at the end of one of the variables
suspicious. Ive rarely run into such problems however. I concur that its
usually javascript when there are discrepancies.


The site could use javascript to surreptitiously change your request.
I've also seen sites that require your referrer to be set to something
specific before they give you what you want.


-----Original Message-----
From: Ford, Andy [mailto:andyford@AGEDWARDS.com]
Sent: Thursday, July 06, 2006 3:41 PM
To: libwww@perl.org
Subject: LWP gets 400, Firefox gets a 200.

What could be different about the two processes?

-----------------------------------------
************************************************************ *******
*

This e-mail is intended only for the addressee named above.
As this e-mail may contain confidential or privileged information,
if you are not the named addressee, you are not authorized
to retain, read, copy or disseminate this message or any part of
it.

************************************************************ *******
*



--
gedanken

RE: LWP gets 400, Firefox gets a 200.

But wouldn't any javascript changes show up in the LiveHTTPheaders
output?=20

I mean the javascript would rewrite the URL, and then I get the final
thing that's sent by the browser from my LiveHTTPheaders output, no?

or am I assuming too much?

The referer looks just the previous page, but I set that "by hand"
anyway, but that doesn't help.

If LiveHTTPheaders tells me:

Referer: https://bogus.com/foo/goo.jsp

Then I say=20

my @headers =3D ( 'Referer' =3D> 'https://bogus.com/foo/goo.jsp' );
$response =3D $agent->get($url,@headers);

Right?


Andy Ford
314-955-6647
andyford@agedwards.com
=20

> -----Original Message-----
> From: Worenklein, David, GCM [mailto:David.Worenklein@rbsgc.com]=20
> Sent: Thursday, July 06, 2006 2:50 PM
> To: Ford, Andy; libwww@perl.org
> Subject: RE: LWP gets 400, Firefox gets a 200.
>=20
> The site could use javascript to surreptitiously change your request.
> I've also seen sites that require your referrer to be set to something
> specific before they give you what you want.
>=20
>=20
> -----Original Message-----
> From: Ford, Andy [mailto:andyford@AGEDWARDS.com]=20
> Sent: Thursday, July 06, 2006 3:41 PM
> To: libwww@perl.org
> Subject: LWP gets 400, Firefox gets a 200.
>=20
> What could be different about the two processes?
>=20
> -----------------------------------------
> ************************************************************ *******
> *
>=20
> This e-mail is intended only for the addressee named above.
> As this e-mail may contain confidential or privileged information,
> if you are not the named addressee, you are not authorized
> to retain, read, copy or disseminate this message or any part of
> it.
>=20
> ************************************************************ *******
> *
>=20


------------------------------------------------------------ ---------------=
----------
A.G. Edwards & Sons' outgoing and incoming e-mails are electronically
archived and subject to review and/or disclosure to someone other=20
than the recipient.

------------------------------------------------------------ ---------------=
----------

RE: LWP gets 400, Firefox gets a 200.

> -----Original Message-----
> From: Gedanken [mailto:gedanken@io.com]=20
> Sent: Thursday, July 06, 2006 2:53 PM
> To: Worenklein, David, GCM
> Cc: Ford, Andy; libwww@perl.org
> Subject: RE: LWP gets 400, Firefox gets a 200.
>=20
> On Thu, 6 Jul 2006, Worenklein, David, GCM wrote:
>=20
> Its a long shot, but i found the %D%D at the end of one of=20
> the variables=20
> suspicious.=20

Those are %3D , URI escapes for "=3D" signs.
I thought that should be what URI->new does, no?

AndyF.


------------------------------------------------------------ ---------------=
----------
A.G. Edwards & Sons' outgoing and incoming e-mails are electronically
archived and subject to review and/or disclosure to someone other=20
than the recipient.

------------------------------------------------------------ ---------------=
----------

RE: LWP gets 400, Firefox gets a 200.

On Thu, 6 Jul 2006, Ford, Andy wrote:

Yeah Im definitely reaching. interpolation problems are rare. Still, it
wouldnt take but a second to change the var => $value to a var => "$value"
and see if theres any change on the wire.

> -----Original Message-----
> From: Gedanken [mailto:gedanken@io.com]
> Sent: Thursday, July 06, 2006 2:53 PM
> To: Worenklein, David, GCM
> Cc: Ford, Andy; libwww@perl.org
> Subject: RE: LWP gets 400, Firefox gets a 200.
>
> On Thu, 6 Jul 2006, Worenklein, David, GCM wrote:
>
> Its a long shot, but i found the %D%D at the end of one of
> the variables
> suspicious.

Those are %3D , URI escapes for "=" signs.
I thought that should be what URI->new does, no?

AndyF.


------------------------------------------------------------ -------------------------
A.G. Edwards & Sons' outgoing and incoming e-mails are electronically
archived and subject to review and/or disclosure to someone other
than the recipient.

------------------------------------------------------------ -------------------------



--
gedanken

RE: LWP gets 400, Firefox gets a 200.

> -----Original Message-----
> From: Gedanken [mailto:gedanken@io.com]=20
> Sent: Thursday, July 06, 2006 3:47 PM
> To: Ford, Andy
> Cc: libwww@perl.org
> Subject: RE: LWP gets 400, Firefox gets a 200.
>=20
> On Thu, 6 Jul 2006, Ford, Andy wrote:
>=20
> Yeah Im definitely reaching. interpolation problems are=20
> rare. Still, it=20
> wouldnt take but a second to change the var =3D> $value to a=20
> var =3D> "$value"=20
> and see if theres any change on the wire.

You mean that "%3D" could be interpreted as a hash?


------------------------------------------------------------ ---------------=
----------
A.G. Edwards & Sons' outgoing and incoming e-mails are electronically
archived and subject to review and/or disclosure to someone other=20
than the recipient.

------------------------------------------------------------ ---------------=
----------