Help Needed - LWP Redirect problems

Help Needed - LWP Redirect problems

am 18.10.2005 23:55:13 von mark1.thompson45

Hi,
I am quite new to LWP but familiar with Perl so be kind. I am trying
to pull some data from a webpage using LWP, the actual page I need the
data from is accessed by clicking on a link off the main page, this
is:-

URL1 =
http://www.youthemanager.co.uk/Clever/servlet/DreamLogin?hom ename=YouTheManager2006&language=MIRROR&aff_id=7

If I take the link I need it starts a servlet and when I view the URL
for the servlet it gives me the following:-

URL2 =
http://www.youthemanager.co.uk/Clever/servlet/PlayerList?gam eid=184&catidx=4

So I set about writing some code to get the information from URL2:-

------------------------------------------------------------ -------
#!C:\\Perl\\bin\\perl -w
use strict;
use warnings;
use LWP::UserAgent;
use HTTP::Request::Common qw(GET);
use HTTP::Cookies;

my $ua= LWP::UserAgent->new(
requests_redirectable => ['GET', 'HEAD', 'POST'],
cookie_jar => HTTP::Cookies->new(
file => 'mycookies.txt',
autosave => 1
),
);

# Define user agent type
$ua->agent('Mozilla/8.0');


# Request object

my $req = GET
'http://www.youthemanager.co.uk/Clever/servlet/DreamLogin?ho mename=YouTheManager2006&language=MIRROR&aff_id=7';

#my $req = GET
'http://www.youthemanager.co.uk/Clever/servlet/PlayerList?ca tidx=3';

# Make the request
my $res = $ua->request($req);

# Check the response
if ($res->is_success) {
print ">>".$res->content;
} else {
print ">> FAILED <<\n";
print $res->status_line ."\n";
}

exit 0;
------------------------------------------------------------ -------

When I enable URL1 in my code and run it I get the contents back from
the webpage, when I enable URL2 in the code (this is the page I want)
then I get the following when the code is run:-

LANGUAGE="JAVASCRIPT">location.replace("/");

I think this maybe something to do with redirects but I thought the
code above would handle that, I know that the GET is used as apposed to
POST for this page as I have snooped the activity on my LAN.

Can anyone offer any advice on where to go from here ....


cheers, Mark.