How do you send a custom request with LWP::UserAgent?

I need to set a request that looks like the following:

"GET http://asdfsadfsdfsadfa.com/page.html\n\n"

Nevermind standard compliance or whatever, this is what I need to send.
I have tried constructing my own HTTP::Request or what have you, even
modifying module sources etc, but I always end up with this, even with
empty HTTP::Headers objects:

------
GET page.html HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: asdfasdfasdfasd.com:80


------


At the very least, I need to have the Connection and TE headers thrown
out. I figured out how to remove the TE header, but not the Connection.
Any help is appreciated

Thanks,

Sebastian

Re: How do you send a custom request with LWP::UserAgent?

sebastian writes:

> I need to set a request that looks like the following:
>
> "GET http://asdfsadfsdfsadfa.com/page.html\n\n"
>
> Nevermind standard compliance or whatever, this is what I need to
> send.

That's what is usually called a "HTTP/0.9 request". There's no RFC
for this version, but hey, this is how everything has begun :-)

Well, almost.... I'd recommend to send a true CRLF instead of "\n\n".
And a single CRLF should be enough for HTTP/0.9.

> I have tried constructing my own HTTP::Request or what have you, even
> modifying module sources etc, but I always end up with this, even with
> empty HTTP::Headers objects:
>
> ------
> GET page.html HTTP/1.1
> TE: deflate,gzip;q=0.3
> Connection: TE, close
> Host: asdfasdfasdfasd.com:80
>
>
> ------

libwww-perl per default creates HTTP/1.1 requests. There is a hook in
the low level modules (Net::HTTP::Methods) where the HTTP version can
be configured - but only HTTP/1.1 and HTTP/1.0 are considered valid,
and I haven't figured out how to pass the HTTPVersion down from
LWP....

Since libwww-perl can handle HTTP/0.9 responses, for a quick and dirty
workaround you could try to write your own
Net::HTTP::Methods::format_request subroutine which simply returns the
line you want given the method and URI as parameters. I've tested the
following (note that it doesn't do all the decent checks which the
original format_request does):

sub format_request {
my $self = shift;
my $method = shift;
my $uri = shift;

my $content = (@_ % 2) ? pop : "";

for ($method, $uri) {
require Carp;
Carp::croak("Bad method or uri") if /\s/ || !length;
}

return join($CRLF, "$method $uri", "", $content);
}

Just to repeat: This is quick, and in particular, it is dirty.

I don't know whether we can (and whether I would want to) convince
anyone to add HTTP/0.9 support back into libwww-perl, though.
--
Cheers,
haj

Re: How do you send a custom request with LWP::UserAgent?

thanks


Harald Joerg wrote:
> sebastian writes:
>
> > I need to set a request that looks like the following:
> >
> > "GET http://asdfsadfsdfsadfa.com/page.html\n\n"
> >
> > Nevermind standard compliance or whatever, this is what I need to
> > send.
>
> That's what is usually called a "HTTP/0.9 request". There's no RFC
> for this version, but hey, this is how everything has begun :-)
>
> Well, almost.... I'd recommend to send a true CRLF instead of "\n\n".
> And a single CRLF should be enough for HTTP/0.9.
>
> > I have tried constructing my own HTTP::Request or what have you, even
> > modifying module sources etc, but I always end up with this, even with
> > empty HTTP::Headers objects:
> >
> > ------
> > GET page.html HTTP/1.1
> > TE: deflate,gzip;q=0.3
> > Connection: TE, close
> > Host: asdfasdfasdfasd.com:80
> >
> >
> > ------
>
> libwww-perl per default creates HTTP/1.1 requests. There is a hook in
> the low level modules (Net::HTTP::Methods) where the HTTP version can
> be configured - but only HTTP/1.1 and HTTP/1.0 are considered valid,
> and I haven't figured out how to pass the HTTPVersion down from
> LWP....
>
> Since libwww-perl can handle HTTP/0.9 responses, for a quick and dirty
> workaround you could try to write your own
> Net::HTTP::Methods::format_request subroutine which simply returns the
> line you want given the method and URI as parameters. I've tested the
> following (note that it doesn't do all the decent checks which the
> original format_request does):
>
> sub format_request {
> my $self = shift;
> my $method = shift;
> my $uri = shift;
>
> my $content = (@_ % 2) ? pop : "";
>
> for ($method, $uri) {
> require Carp;
> Carp::croak("Bad method or uri") if /\s/ || !length;
> }
>
> return join($CRLF, "$method $uri", "", $content);
> }
>
> Just to repeat: This is quick, and in particular, it is dirty.
>
> I don't know whether we can (and whether I would want to) convince
> anyone to add HTTP/0.9 support back into libwww-perl, though.
> --
> Cheers,
> haj