Help Needed - LWP Redirect problems
am 18.10.2005 23:54:47 von mark1.thompson45Hi,
I am quite new to LWP but familiar with Perl so be kind. I am trying
to pull some data from a webpage using LWP, the actual page I need the
data from is accessed by clicking on a link off the main page, this
is:-
URL1 =
http://www.youthemanager.co.uk/Clever/servlet/DreamLogin?hom ename=YouTheManager2006&language=MIRROR&aff_id=7
If I take the link I need it starts a servlet and when I view the URL
for the servlet it gives me the following:-
URL2 =
http://www.youthemanager.co.uk/Clever/servlet/PlayerList?gam eid=184&catidx=4
So I set about writing some code to get the information from URL2:-
------------------------------------------------------------ -------
#!C:\\Perl\\bin\\perl -w
use strict;
use warnings;
use LWP::UserAgent;
use HTTP::Request::Common qw(GET);
use HTTP::Cookies;
my $ua= LWP::UserAgent->new(
requests_redirectable => ['GET', 'HEAD', 'POST'],
cookie_jar => HTTP::Cookies->new(
file => 'mycookies.txt',
autosave => 1
),
);
# Define user agent type
$ua->agent('Mozilla/8.0');
# Request object
my $req = GET
'http://www.youthemanager.co.uk/Clever/servlet/DreamLogin?ho mename=YouTheManager2006&language=MIRROR&aff_id=7';
#my $req = GET
'http://www.youthemanager.co.uk/Clever/servlet/PlayerList?ca tidx=3';
# Make the request
my $res = $ua->request($req);
# Check the response
if ($res->is_success) {
print ">>".$res->content;
} else {
print ">> FAILED <<\n";
print $res->status_line ."\n";
}
exit 0;
------------------------------------------------------------ -------
When I enable URL1 in my code and run it I get the contents back from
the webpage, when I enable URL2 in the code (this is the page I want)
then I get the following when the code is run:-
I think this maybe something to do with redirects but I thought the
code above would handle that, I know that the GET is used as apposed to
POST for this page as I have snooped the activity on my LAN.
Can anyone offer any advice on where to go from here ....
cheers, Mark.