Selective Redirect with LWP::Parallelel::Useragent

Hi I have a problem.

I use LWP::Parallel::Useragent to fetch prices to my Meta Search
Engine.

It usually works great but now i have a problem.
There is one site on which it does not work if redirects are enabled.
At the moment I use simple_request and works my way through every step
and it works great.

But I want to be able to use the Register method so its truly parallel.
I know how to turn Redirect of but several of the other sites need
Redirect to work.

Do you see my problem.
I want to be able to turn of Redirect on one site but not on the
others.

I really need help on this one.

Re: Selective Redirect with LWP::Parallelel::Useragent

jonas.soderstrom@gmail.com wrote:
> Hi I have a problem.
>
> I use LWP::Parallel::Useragent to fetch prices to my Meta Search
> Engine.

It is UserAgent, not Useragent. Case often matters.

>
> It usually works great but now i have a problem.
> There is one site on which it does not work if redirects are enabled.
> At the moment I use simple_request and works my way through every step
> and it works great.
>
> But I want to be able to use the Register method so its truly parallel.
> I know how to turn Redirect of[f] but several of the other sites need
> Redirect to work.
>
> Do you see my problem.
> I want to be able to turn of Redirect on one site but not on the
> others.

I'm not sure if I see the problem or not. The docs for $ua->redirect tells
you to "See $ua->register for how to change the behaviour for particular
requests only." So it seems like all you have to do is decide, before
registering each request, whether the request is to that specific site or
not, and register it accordingly.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB

Re: Selective Redirect with LWP::Parallelel::Useragent

Hi xho

Thanks for you answer.

This is exactly what I wanted to hear. I have been looking in the
documentation for this solution but I can't seem to find the proper
argument/parameters to send. Maybe I am looking in the wrong place.

I will looking deeper this time now that I have your answer. If anybody
knows what arguments/parameters so send to $ua->register to turn of the
redirect for the specific request please let me know. Either here or by
email (or both)

Jonas

Re: Selective Redirect with LWP::Parallelel::Useragent

jonas.soderstrom@gmail.com wrote:
> Hi xho
>
> Thanks for you answer.
>
> This is exactly what I wanted to hear. I have been looking in the
> documentation for this solution but I can't seem to find the proper
> argument/parameters to send. Maybe I am looking in the wrong place.
>
> I will looking deeper this time now that I have your answer. If anybody
> knows what arguments/parameters so send to $ua->register to turn of the
> redirect for the specific request please let me know. Either here or by
> email (or both)
>
> Jonas

Hi Jonas,

Perhaps you are using an older version which does not have this feature?
This is from the docs for my version:

$ua->register ( $request [, $arg [, $size [, $redirect_ok]]] )
Registers the given request with the User Agent. In case of an
error, a "HTTP::Request" object containing the HTML-Error
message is returned. Otherwise (that is, in case of a success)
it will return undef.
...
Specifying $redirect_ok will alter the redirection behaviour for
this particular request only. '1' or any other true value will
force Parallel::UserAgent to follow redirects, even if the
default is set to 'no_redirect'. (see "$ua-"redirect>) '0' or
any other false value should do the reverse. See LWP::UserAgent
for using an object's "requests_redirectable" list for
fine-tuning this behav- ior.

I don't know how to describe it better than that (other than to say that it
should read "...'0' or any other false but defined value should do the
reverse").


Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB