Extremely fast template processor

--000e0cd20d302e64ca04a44178ea
Content-Type: text/plain; charset=ISO-8859-1

I have a requirement of generating large number (> 100 thousand ) files
from a single template by replacing the placeholders with respective values
Currently I use Template-Toolkit, but is there a faster way ?

Since my template is just dumb template ( no IF , no FOR etc ) .. I just
need a simple tool to replace the placeholders.

Any suggestions for a super fast placeholder replacement tool,

Or if I write this program in C will this be any faster





--
Thanks
Ram





n

--000e0cd20d302e64ca04a44178ea--

Re: Extremely fast template processor

--bcaec547c8ffaa9f6704a4447181
Content-Type: text/plain; charset=ISO-8859-1

On Fri, May 27, 2011 at 5:56 AM, Ramprasad Prasad wrote:

> I have a requirement of generating large number (> 100 thousand ) files
> ....
>

rather suspect that the limiting factor is going to be I/O.

Even with massive assistance from the Operating System, writing to a disk
will take a long time, measured in milliseconds. If you are not doing much
processing other than string-replacement, your process-time will be
relatively short, probably a few microseconds per file.

You may get better throughput by running multiple processes. Batch up your
input files and run several instances of your code, one per batch. If you
can't use divide-and-conquer, you might look at fork() and try running
multiple child-processes under the covers.

>
>
>
>
>
> --
> Thanks
> Ram
>
>
>
>
>
> n
>



--

Bob Goolsby
bob.goolsby@gmail.com

--bcaec547c8ffaa9f6704a4447181--

Re: Extremely fast template processor

>>>>> "RP" == Ramprasad Prasad writes:

RP> I have a requirement of generating large number (> 100 thousand )
RP> files from a single template by replacing the placeholders with
RP> respective values Currently I use Template-Toolkit, but is there a
RP> faster way ?

RP> Since my template is just dumb template ( no IF , no FOR etc )
RP> .. I just need a simple tool to replace the placeholders.

RP> Any suggestions for a super fast placeholder replacement tool,

RP> Or if I write this program in C will this be any faster

try Template::Simple on cpan. it is my module is it is extremely
fast. it will likely even be faster than some c templaters as it can
compile your templates or run then as is. i have benchmarks showing it
is faster (by a good deal) over the large template systems in perl and
it still beats the smaller leaner ones. also it is very simple to use
and has simple markups.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: Extremely fast template processor

>>>>> "Bg" == Bob goolsby writes:

Bg> On Fri, May 27, 2011 at 5:56 AM, Ramprasad Prasad wrote:
>> I have a requirement of generating large number (> 100 thousand ) files
>> ....
>>

Bg> rather suspect that the limiting factor is going to be I/O.

Bg> Even with massive assistance from the Operating System, writing to
Bg> a disk will take a long time, measured in milliseconds. If you
Bg> are not doing much processing other than string-replacement, your
Bg> process-time will be relatively short, probably a few microseconds
Bg> per file.

Bg> You may get better throughput by running multiple processes.
Bg> Batch up your input files and run several instances of your code,
Bg> one per batch. If you can't use divide-and-conquer, you might
Bg> look at fork() and try running multiple child-processes under the
Bg> covers.

if the problem is i/o bound, then how would multiple processes help?
they only help if you are cpu bound and have actual multiple processors
(on one or multiple boxes). the i/o bandwidth on a single box is the
same for one or a bunch of processors.

i recommended Template::Simple which is the fastest templater out
there. if the OP also uses File::Slurp to read/write the templates and
results, that will optimize the I/O as well. in fact Template::Simple
does use File::Slurp to read in templates but it doesn't write the
results to a file but returns them to the caller for speed an
flexibility.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: Extremely fast template processor

On Fri, May 27, 2011 at 18:26:42 +0530 , Ramprasad Prasad wrote:
> I have a requirement of generating large number (> 100 thousand ) files
> from a single template by replacing the placeholders with respective values
> Currently I use Template-Toolkit, but is there a faster way ?
>
> Since my template is just dumb template ( no IF , no FOR etc ) .. I just
> need a simple tool to replace the placeholders.
>
> Any suggestions for a super fast placeholder replacement tool,
>
> Or if I write this program in C will this be any faster

Do you have profiler (Devel::NYTProf; Devel::DProf is deprecated and
abandoned) runs and output showing that Template::Toolkit is your
bottleneck? If so, then yes, consider replacing your templating engine
after you've investigated ways of making it faster. If not, then you're
only guessing where the hotspots are, and programmers are notoriously
bad at guessing.

--
Chris Nehren | Coder, Sysadmin, Masochist
Shadowcat Systems Ltd. | http://shadowcat.co.uk/

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/