cron or command line: run a very long php script

Hi there,

I have a list of links which point to e.g.
thescript.php?album=somePictures1
thescript.php?album=somePictures2

This list is about 3000 links. Each album may have 500 or more pictures in it.
the script looks in the specified dir, and creates thumbnails if they are not
present. So, displaying a particular album often takes quite a while. I've set
the proper php.ini stuff to accomodate the long script_execution's. Thats all
working fine.

Now, I dont want the end user to have to wait for the thumbs if they land on an
album that has not yet had thumbnails created.

So the question is, what is the best way to run all those links automatically,
thereby creating all the thumbs?

Thanks for your time,

Re: cron or command line: run a very long php script

> I have a list of links which point to e.g.
> thescript.php?album=somePictures1
> thescript.php?album=somePictures2
>
> This list is about 3000 links. Each album may have 500 or more pictures in it.
> the script looks in the specified dir, and creates thumbnails if they are not
> present. So, displaying a particular album often takes quite a while. I've set
> the proper php.ini stuff to accomodate the long script_execution's. Thats all
> working fine.
>
> Now, I dont want the end user to have to wait for the thumbs if they land on an
> album that has not yet had thumbnails created.
>
> So the question is, what is the best way to run all those links automatically,
> thereby creating all the thumbs?
>
> Thanks for your time,

As you suggested in your subject line, probably the best route to go
would be to set up a command line script to process the file, and use
cron to do it regularly if new files are being added all the time (or
just process files at the time of introduction in the system).

Just remember, in command-line scripts for PHP, you have to first give
a path to the interpreter like so:
#!/usr/bin/php

Just replace "/usr/bin/php" with the path to your interpreter.

You will also of course, have to go through all directories not just
one, so this is a brief recursive function to iterate a directory
structure. It takes the root directory containing images and their
directories as its parameter. Just replace the comment with the
appropriate means of processing your images and making thumbnails:


//(not tested)
function iterateDir($dir){



if (is_dir($dir)) {
if ($dh = opendir($dir)) {
while (($file = readdir($dh)) !== false) {
if($file != "." && $file != ".."){
if(is_dir($file)){
iterateDir($file);
}else{
//...process file...
}
}
}
closedir($dh);
}
}


}

Re: cron or command line: run a very long php script

On 17 Aug, 19:13, "burgermeiste...@gmail.com"
wrote:
>
> > This list is about 3000 links. Each album may have 500 or more pictures in it.
> > the script looks in the specified dir, and creates thumbnails if they are not
> > present.
>
> As you suggested in your subject line, probably the best route to go
> would be to set up a command line script to process the file,

No - that's the worst route to go.

> process files at the time of introduction in the system
is the best solution.

Admittedly, that is not going to solve the problem for existing
images. But tying a check/generation into the viewing page provides a
simple interface to updating the images which should only need done
once.

> Just remember, in command-line scripts for PHP, you have to first give
> a path to the interpreter like so:
> #!/usr/bin/php
>

....if you are running on a POSIX type system, but you'll also need to
make the script executable. RTFM - http://www.php.net/manual/en/features.commandline.php

Yes you do it via a cron job if you prefer - there's lot of
discussions about how to do this - and even toolkits which emulate it
when you don't have access to cron. But for your purposes this is just
adding unnecessary complexity.

C.

Re: cron or command line: run a very long php script

On Sun, 19 Aug 2007 12:38:15 -0000, "C." wrote:

>...if you are running on a POSIX type system, but you'll also need to
>make the script executable. RTFM - http://www.php.net/manual/en/features.commandline.php
>
>Yes you do it via a cron job if you prefer -

Ok, thanks, I guess my question was more about whether my thumbnail generation
script will choke if the cron, command line script, etc tries to run too many
instances of it at once. I dont know how that part would work. Like if I run
script.php?album=dir1 and dont have a way for the cron to know whether the
script is done with dir1 before moving on to dir2.

Jeez, just this one parent dir with its 3300 sub dirs is going to take around
100 hours.

Re: cron or command line: run a very long php script

On Aug 19, 5:47 pm, J. Frank Parnell wrote:
> On Sun, 19 Aug 2007 12:38:15 -0000, "C." wrote:
> >...if you are running on a POSIX type system, but you'll also need to
> >make the script executable. RTFM -http://www.php.net/manual/en/features.commandline.php
>
> >Yes you do it via a cron job if you prefer -
>
> Ok, thanks, I guess my question was more about whether my thumbnail generation
> script will choke if the cron, command line script, etc tries to run too many
> instances of it at once. I dont know how that part would work. Like if I run
> script.php?album=dir1 and dont have a way for the cron to know whether the
> script is done with dir1 before moving on to dir2.
>
> Jeez, just this one parent dir with its 3300 sub dirs is going to take around
> 100 hours.

Just to clarify on what C added to this thread: he/she is right. You
will need some script to run once and to process all the existing
images from command-line, and will probably want to just run the
routine on new, incoming images after that. If that's not a
possibility for some reason, *then* cron is the way to go.

Anyways, about cron or CLI choking: that's the reason I posted that
recursive function. You won't need multiple instances to run on each
directory, you can instead run one script that will process all
directories. Of course, like you said, this is going to take a long
time to do because you have so many images. That means you should do
these two things:
1) Be sure that you don't have any time limits set on the script's
execution time, i.e. use the function: set_time_limit(0)
2) Fully test the script before you run it on all your images. You
don't want to run the script, wait 100 hours or whatever, and then
find out that the script causes a weird distortion on some of your
files.

I hope this helps you.