Thread alternative for multi-processing?

Sometimes, I want to take the advantage of multi-core processor, and
make the full use of my CPU time,

e.g. I have a function, which takes a file name as input, do some text
processing (e.g. CPU intensive), and save it back.

I don't want to use thread, as it need to recompile perl on all my
machines and I am not sure if it is stable enough to affect other
apps.

I don't mind to have a little overhead in using the fork mode (i.e.
multi-process). I want to know are there any recommended module for
my purpose?

For example, I am trying Parallel::ForkManager, POE::Wheel::Run, so
anyone want to share the experiences?


Thanks.

Re: Thread alternative for multi-processing?

howa writes:

> For example, I am trying Parallel::ForkManager, POE::Wheel::Run, so
> anyone want to share the experiences?

I have used Parallel::ForkManager a lot with no problems, it just
works. I have never had the need to communicate enything more complex
than the exit code back to the parrent though.

//Makholm

Re: Thread alternative for multi-processing?

On Apr 16, 4:50=A0am, Peter Makholm wrote:
> howa writes:
> > For example, I am trying Parallel::ForkManager, POE::Wheel::Run, so
> > anyone want to share the experiences?
>
> I have used Parallel::ForkManager a lot with no problems, it just
> works. I have never had the need to communicate enything more complex
> than the exit code back to the parrent though.

I've found it useful to pass an $ident which i can use in the
run_on_finish callback to associate with the exit code.

Re: Thread alternative for multi-processing?

nolo contendere writes:

>> I have used Parallel::ForkManager a lot with no problems, it just
>> works. I have never had the need to communicate enything more complex
>> than the exit code back to the parrent though.
>
> I've found it useful to pass an $ident which i can use in the
> run_on_finish callback to associate with the exit code.

Yes, but I consider this to be communication from the parrent on fork
time to the parrent on wait time. Even if the $ident is an reference
to something, the child can't change it for the parrent.

//Makholm

Re: Thread alternative for multi-processing?

On Apr 16, 11:10=A0am, Peter Makholm wrote:
> nolo contendere writes:
> >> I have used Parallel::ForkManager a lot with no problems, it just
> >> works. I have never had the need to communicate enything more complex
> >> than the exit code back to the parrent though.
>
> > I've found it useful to pass an $ident which i can use in the
> > run_on_finish callback to associate with the exit code.
>
> Yes, but I consider this to be communication from the parrent on fork
> time to the parrent on wait time. Even if the $ident is an reference
> to something, the child can't change it for the parrent.
>

True, perhaps I should have replied to the OP. I was simply sharing my
experience with Parallel::ForkManager, didn't mean to imply that the
$ident was a child->parent communication.

Re: Thread alternative for multi-processing?

howa wrote:
> Sometimes, I want to take the advantage of multi-core processor, and
> make the full use of my CPU time,
>
> e.g. I have a function, which takes a file name as input, do some text
> processing (e.g. CPU intensive), and save it back.

The saving it back might be a problem. Do the lines have to be saved
back in the same order they are originally present? With parallel,
it requires special efforts to make that happen.

>
> I don't want to use thread, as it need to recompile perl on all my
> machines and I am not sure if it is stable enough to affect other
> apps.
>
> I don't mind to have a little overhead in using the fork mode (i.e.
> multi-process). I want to know are there any recommended module for
> my purpose?
>
> For example, I am trying Parallel::ForkManager, POE::Wheel::Run, so
> anyone want to share the experiences?

In this case, I'd tend to role my own forking. I generally use
Parallel::ForkManager for when the number of processes I want run is
greater than the number I want to run simultaneously, which doesn't seem to
be the case for you. You certainly *can* use it in other cases, I just
don't see much need to. I've looked into POE::Wheel::Run a few times but
never got over the activation barrier to actually use it.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.