Can I iterate through a file on a CGI page?

Apologies to Perl purists - comp.infosystems.www.authoring.cgi doesn't
work on my newsreader, and this is a CGI question:

What I want to do is, I've got a large collection of image files:
$ wc gallery-pix
37448 62619 3218967 gallery-pix

and what I'd like to do is look at each of the 37488 image files on
some kind of page, with buttons like "Keep", "Skip", and "Quit",
so I can page through all of these images, which are strewn all
over the Samba server, and decide which ones might look good on
the website.

So, is it possible to do something like (in pseudocode):

for each $line in {
show webpage with tag, and the three buttons;
get button response, decide what to do with file;
if button == "Quit", save place in source file;
next;

or so?

Thanks,
Rich

Re: Can I iterate through a file on a CGI page?

Rich Grise wrote:
> for each $line in {

This is gonna be your problem. CGI is stateless. Each time you hit one
of the submit buttons you will re-invoke the program, and the new
invocation knows nothing about the state of the previous invocation. If
you attempted something like you wrote then you would keep showing the
first item over and over again, because it would start at the beginning
of the file each time you invoked the program.

There are several things you can do to get around this issue. My
preference is to use a database instead of a file, and run my CGI under
a mod_perl webserver which is smart enough to cache the database handle.
But maybe that's overkill for what you want to do.

If you are willing to pay the price of opening up your flatfile each
time you invoke the program, you could pass the current (or next) line
number as a hidden() parameter. Thus the program knows which line of
the file to process next (I would recommend tying the file to an array
so the line number simply becomes the array subscript).

Re: Can I iterate through a file on a CGI page?

Rich Grise wrote:
> Apologies to Perl purists - comp.infosystems.www.authoring.cgi doesn't
> work on my newsreader, and this is a CGI question:
>
> What I want to do is, I've got a large collection of image files:
> $ wc gallery-pix
> 37448 62619 3218967 gallery-pix
>
> and what I'd like to do is look at each of the 37488 image files on
> some kind of page, with buttons like "Keep", "Skip", and "Quit",
> so I can page through all of these images, which are strewn all
> over the Samba server, and decide which ones might look good on
> the website.
>
> So, is it possible to do something like (in pseudocode):
>
> for each $line in {

In CGI, your program won't survive for the loop to iterate. Unless
you are making one page with all 37448 files on it.

> show webpage with tag, and the three buttons;
> get button response, decide what to do with file;

What would you do with the file in each case?

> if button == "Quit", save place in source file;
> next;

Make one directory with all the files (or with a symbolic links for each
file). Each time the program is invoked, take the first entry in the
directory and display it. Based on the response, either move it to the
accept directory or the reject directory (or move it to accept vs delete
it, whatever.) Since the file is no longer there, place is inherently
saved. Quit doesn't have to do anything, nor even have to exist--closing
the browser without responding is a form of quiting.

Or use a database.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

Re: Can I iterate through a file on a CGI page?

Rich Grise wrote:
> Apologies to Perl purists - comp.infosystems.www.authoring.cgi doesn't
> work on my newsreader, and this is a CGI question:
>
> What I want to do is, I've got a large collection of image files:
> $ wc gallery-pix
> 37448 62619 3218967 gallery-pix
>
> and what I'd like to do is look at each of the 37488 image files on
> some kind of page, with buttons like "Keep", "Skip", and "Quit",
> so I can page through all of these images, which are strewn all
> over the Samba server, and decide which ones might look good on
> the website.
>
> So, is it possible to do something like (in pseudocode):
>
> for each $line in {
> show webpage with tag, and the three buttons;
> get button response, decide what to do with file;
> if button == "Quit", save place in source file;
> next;
>
> or so?
>

Surely, looking at 37448 separate images one by one and making
subjective judgements about them is going to take a human a long long
time. If you can make a judgement in a second then it would take over
ten hours without a break to view all of them. This kind of monotonous
task would drive me crazy after a few hundred images.

I'd expect there are lots of similar images and that they may not be
naturally grouped together - which makes selecting the best of a kind
impossible, you're bound to later encounter a "better" image than one
you'd selected to "keep" yesterday or a week ago. Which means more
passes through the "keep" collection.

If I had to do it, I'd first categorise the images, then display
thumbnails for each category and pick ones to keep from each category.

Better to pay a graphics artist to produce a new consistent set for a
website?

--
RGB

Re: Can I iterate through a file on a CGI page?

On Wed, 02 Apr 2008 04:07:42 +0000, xhoster wrote:
> Rich Grise wrote:
>> Apologies to Perl purists - comp.infosystems.www.authoring.cgi doesn't
>> work on my newsreader, and this is a CGI question:
>>
>> What I want to do is, I've got a large collection of image files:
>> $ wc gallery-pix
>> 37448 62619 3218967 gallery-pix
>>
>> and what I'd like to do is look at each of the 37488 image files on
>> some kind of page, with buttons like "Keep", "Skip", and "Quit",
>> so I can page through all of these images, which are strewn all
>> over the Samba server, and decide which ones might look good on
>> the website.
>>
>> So, is it possible to do something like (in pseudocode):
>>
>> for each $line in {
>
> In CGI, your program won't survive for the loop to iterate. Unless
> you are making one page with all 37448 files on it.
>
>> show webpage with tag, and the three buttons;
>> get button response, decide what to do with file;
>
> What would you do with the file in each case?
>
>> if button == "Quit", save place in source file;
>> next;
>
> Make one directory with all the files (or with a symbolic links for each
> file). Each time the program is invoked, take the first entry in the
> directory and display it. Based on the response, either move it to the
> accept directory or the reject directory (or move it to accept vs delete
> it, whatever.) Since the file is no longer there, place is inherently
> saved. Quit doesn't have to do anything, nor even have to exist--closing
> the browser without responding is a form of quiting.

Thanks - I really like this answer - I hadn't even consider loading up
a directory with symlinks.

I'm going to try this next, as a way to avoid slurping the whole file.

So, anybody got a quick and dirty script that will make 38,000 symlinks?
;-)

I'm not too worried about how long it will take - I'm doing this in
my "spare" time, and the boss doesn't bother me much as long as I look
busy. :-)

Thanks!
Rich

Re: Can I iterate through a file on a CGI page?

In article , Rich Grise
wrote:

>
> So, anybody got a quick and dirty script that will make 38,000 symlinks?

for my $i ( 1..38000 ) {
symlink( '/path/to/somefile', sprintf("symlink%5.5d",$i));
}

--
Jim Gibson

Posted Via Usenet.com Premium Usenet Newsgroup Services
----------------------------------------------------------
** SPEED ** RETENTION ** COMPLETION ** ANONYMITY **
----------------------------------------------------------
http://www.usenet.com

Re: Can I iterate through a file on a CGI page?

Jim Gibson wrote:
> In article , Rich Grise
> wrote:
>
>> So, anybody got a quick and dirty script that will make 38,000 symlinks?
>
> for my $i ( 1..38000 ) {
> symlink( '/path/to/somefile', sprintf("symlink%5.5d",$i));
> }

for my $name ( 'aaaaaa' .. 'aacefn' ) {
symlink '/path/to/somefile', $name;
}


John
--
Perl isn't a toolbox, but a small machine shop where you
can special-order certain sorts of tools at low cost and
in short order. -- Larry Wall

Re: Can I iterate through a file on a CGI page?

On Thu, 03 Apr 2008 04:10:07 +0000, John W. Krahn wrote:
> Jim Gibson wrote:
>> In article , Rich Grise
>>
>>> So, anybody got a quick and dirty script that will make 38,000 symlinks?
>>
>> for my $i ( 1..38000 ) {
>> symlink( '/path/to/somefile', sprintf("symlink%5.5d",$i));
>> }
>
> for my $name ( 'aaaaaa' .. 'aacefn' ) {
> symlink '/path/to/somefile', $name;
> }
>

Thanks for these. :-)

OK, so now I've got $mypath/source-links/symlink00000 through
$mypath/source-links/symlink19634 , and I figured out how to iterate them:
my $thislink=`ls source-links | head -1` , but then when I use the link in
an the actual file that the link is pointing to? I suppose I could do an ls
-l and parse the link, but isn't there an easier/quicker/more elegant way
to get that info? (I think they call it "dereferencing", but I'm not sure.)

In other words, I want to show the pic, and show its "real" name, not the
link name; can that be done easily?

Thanks,
Rich




Now, I've got source-links/

Re: Can I iterate through a file on a CGI page?

On Tue, 08 Apr 2008 17:34:35 +0000, Rich Grise wrote:

> the actual file that the link is pointing to? I suppose I could do an ls
> -l and parse the link, but isn't there an easier/quicker/more elegant way
> to get that info? (I think they call it "dereferencing", but I'm not sure.)

Nah:
### start script ###
#!/usr/bin/perl -w

my $line=`ls -l source-links | head -2 | tail -1`;

chomp $line;

my $linkname = substr("$line", 50, 12);
my $targetname = substr("$line", 66);

print("$line\n");

print("name = $linkname, target = $targetname\n");
### end script ###

$ parse-ls
lrwxrwxrwx 1 richgrise users 75 2008-04-07 13:47 symlink00000 -> /C/Documents and Settings/Administrator/My Documents/My Pictures/Sample.jpg
name = symlink00000, target = /C/Documents and Settings/Administrator/My Documents/My Pictures/Sample.jpg
$

Cheers!
Rich

Re: Can I iterate through a file on a CGI page?

Rich Grise wrote:
> On Tue, 08 Apr 2008 17:34:35 +0000, Rich Grise wrote:
>
>> the actual file that the link is pointing to? I suppose I could do an ls
>> -l and parse the link, but isn't there an easier/quicker/more elegant way
>> to get that info? (I think they call it "dereferencing", but I'm not sure.)
>
> Nah:
> ### start script ###
> #!/usr/bin/perl -w
>
> my $line=`ls -l source-links | head -2 | tail -1`;
>
> chomp $line;
>
> my $linkname = substr("$line", 50, 12);
> my $targetname = substr("$line", 66);
>
> print("$line\n");
>
> print("name = $linkname, target = $targetname\n");

Do you have your own print() subroutine? No need for '()'.

> ### end script ###
>
> $ parse-ls
> lrwxrwxrwx 1 richgrise users 75 2008-04-07 13:47 symlink00000 -> /C/Documents and Settings/Administrator/My Documents/My Pictures/Sample.jpg
> name = symlink00000, target = /C/Documents and Settings/Administrator/My Documents/My Pictures/Sample.jpg

Not very reliable. If the owner or group change to something
of different length, it could easily return the wrong data. Using
split would be more reliable, but still not very good.

Use the correct function: perldoc -f readlink

Re: Can I iterate through a file on a CGI page?

In article <47fbe8c0$0$33227$815e3792@news.qwest.net>, J. Gleixner
wrote:

> Rich Grise wrote:
> > On Tue, 08 Apr 2008 17:34:35 +0000, Rich Grise wrote:
> >
> >> the actual file that the link is pointing to? I suppose I could do an ls
> >> -l and parse the link, but isn't there an easier/quicker/more elegant way
> >> to get that info? (I think they call it "dereferencing", but I'm not sure.)
> >
> > Nah:
> > ### start script ###
> > #!/usr/bin/perl -w
> >
> > my $line=`ls -l source-links | head -2 | tail -1`;
> >
> > chomp $line;
> >
> > my $linkname = substr("$line", 50, 12);
> > my $targetname = substr("$line", 66);
> >
> > print("$line\n");
> >
> > print("name = $linkname, target = $targetname\n");
>
> Do you have your own print() subroutine? No need for '()'.
>
> > ### end script ###
> >
> > $ parse-ls
> > lrwxrwxrwx 1 richgrise users 75 2008-04-07 13:47 symlink00000 ->
> > /C/Documents and Settings/Administrator/My Documents/My Pictures/Sample.jpg
> > name = symlink00000, target = /C/Documents and Settings/Administrator/My
> > Documents/My Pictures/Sample.jpg
>
> Not very reliable. If the owner or group change to something
> of different length, it could easily return the wrong data. Using
> split would be more reliable, but still not very good.
>
> Use the correct function: perldoc -f readlink

This should work and doesn't require shelling out to ls:

use strict;
use warnings;
use File::Find;
find(
sub {
my $linkname = $_;
return unless -l $_;
my $targetname = readlink $linkname;
print "name = $linkname, target = $targetname\n";
},
'source-links'
);

--
Jim Gibson

Posted Via Usenet.com Premium Usenet Newsgroup Services
----------------------------------------------------------
** SPEED ** RETENTION ** COMPLETION ** ANONYMITY **
----------------------------------------------------------
http://www.usenet.com

Re: Can I iterate through a file on a CGI page?

On Tue, 08 Apr 2008 16:12:14 -0700, Jim Gibson wrote:

> In article <47fbe8c0$0$33227$815e3792@news.qwest.net>, J. Gleixner
> wrote:
>> Rich Grise wrote:

>> >
>> > my $line=`ls -l source-links | head -2 | tail -1`;
>> > chomp $line;
>> > my $linkname = substr("$line", 50, 12);
>> > my $targetname = substr("$line", 66);
>> > print("$line\n");
>> > print("name = $linkname, target = $targetname\n");
>>
>> Not very reliable. If the owner or group change to something
>> of different length, it could easily return the wrong data. Using
>> split would be more reliable, but still not very good.
>>
>> Use the correct function: perldoc -f readlink
>
> This should work and doesn't require shelling out to ls:
>
> use strict;
> use warnings;
> use File::Find;
> find(
> sub {
> my $linkname = $_;
> return unless -l $_;
> my $targetname = readlink $linkname;
> print "name = $linkname, target = $targetname\n";
> },
> 'source-links'
> );

OK, I do this, and it shows me _all_ of the links. What have I
missed?

And am I supposed to pass the name of the link I'm looking for? Why
do you initialize $linkname to $_? I want a script that doesn't
have to already know $linkname - it needs to find the next one in
the subdir, whichever one that is.

What am I missing?

Thanks,
Rich



Thanks,
Rich

Re: Can I iterate through a file on a CGI page?

Rich Grise wrote:
> On Tue, 08 Apr 2008 16:12:14 -0700, Jim Gibson wrote:
>
>> In article <47fbe8c0$0$33227$815e3792@news.qwest.net>, J. Gleixner
>> wrote:
>>> Rich Grise wrote:
>
>>>> my $line=`ls -l source-links | head -2 | tail -1`;
>>>> chomp $line;
>>>> my $linkname = substr("$line", 50, 12);
>>>> my $targetname = substr("$line", 66);
>>>> print("$line\n");
>>>> print("name = $linkname, target = $targetname\n");
>>> Not very reliable. If the owner or group change to something
>>> of different length, it could easily return the wrong data. Using
>>> split would be more reliable, but still not very good.
>>>
>>> Use the correct function: perldoc -f readlink
>> This should work and doesn't require shelling out to ls:
>>
>> use strict;
>> use warnings;
>> use File::Find;
>> find(
>> sub {
>> my $linkname = $_;
>> return unless -l $_;
>> my $targetname = readlink $linkname;
>> print "name = $linkname, target = $targetname\n";
>> },
>> 'source-links'
>> );
>
> OK, I do this, and it shows me _all_ of the links. What have I
> missed?
>
> And am I supposed to pass the name of the link I'm looking for? Why
> do you initialize $linkname to $_? I want a script that doesn't
> have to already know $linkname - it needs to find the next one in
> the subdir, whichever one that is.
>
> What am I missing?

First, you missed reading the documentation before posting yet
another question. Second, you missed defining to us what
'the next one' means. If there is some order, then you need to
sort the files/directories and do whatever you want with that
information:

perldoc File::Find
perldoc -f stat
perldoc -f sort

Reading those should help you answer your questions.

Re: Can I iterate through a file on a CGI page?

On Thu, 10 Apr 2008 14:41:32 -0500, J. Gleixner wrote:

> Rich Grise wrote:
>> On Tue, 08 Apr 2008 16:12:14 -0700, Jim Gibson wrote:
>>
>>> In article <47fbe8c0$0$33227$815e3792@news.qwest.net>, J. Gleixner
>>> wrote:
>>>> Rich Grise wrote:
>>
>>>>> my $line=`ls -l source-links | head -2 | tail -1`;
>>>>> chomp $line;
>>>>> my $linkname = substr("$line", 50, 12);
>>>>> my $targetname = substr("$line", 66);
>>>>> print("$line\n");
>>>>> print("name = $linkname, target = $targetname\n");
>>>> Not very reliable. If the owner or group change to something
>>>> of different length, it could easily return the wrong data. Using
>>>> split would be more reliable, but still not very good.
>>>>
>>>> Use the correct function: perldoc -f readlink
>>> This should work and doesn't require shelling out to ls:
>>>
>>> use strict;
>>> use warnings;
>>> use File::Find;
>>> find(
>>> sub {
>>> my $linkname = $_;
>>> return unless -l $_;
>>> my $targetname = readlink $linkname;
>>> print "name = $linkname, target = $targetname\n";
>>> },
>>> 'source-links'
>>> );
>>
>> OK, I do this, and it shows me _all_ of the links. What have I
>> missed?
>>
>> And am I supposed to pass the name of the link I'm looking for? Why
>> do you initialize $linkname to $_? I want a script that doesn't
>> have to already know $linkname - it needs to find the next one in
>> the subdir, whichever one that is.
>>
>> What am I missing?
>
> First, you missed reading the documentation before posting yet
> another question. Second, you missed defining to us what
> 'the next one' means. If there is some order, then you need to
> sort the files/directories and do whatever you want with that
> information:
>
> perldoc File::Find
> perldoc -f stat
> perldoc -f sort
>
> Reading those should help you answer your questions.

Yeah, you're right, I'm sorry, but I've been reading for going on about
a month now and it's terribly frustrating - it's pretty much the same
problem with all the Linux docs - you have to already know what feature
you're looking for before you know which doc to read, like for example:
`readlink` - how is a person supposed to know that such a thing even
exists, except by asking, or plowing through every possible doc there
is until you stumble upon it? ?:-/

Thanks,
Rich

Re: Can I iterate through a file on a CGI page?

Rich Grise writes:

> Yeah, you're right, I'm sorry, but I've been reading for going on about
> a month now and it's terribly frustrating - it's pretty much the same
> problem with all the Linux docs - you have to already know what feature
> you're looking for before you know which doc to read, like for example:
> `readlink` - how is a person supposed to know that such a thing even
> exists, except by asking, or plowing through every possible doc there
> is until you stumble upon it? ?:-/

For *nix programming in general, I really recommend the "advanced
programming in the unix environment, 2nd edition" book from Addisson
Wesley. For perl, get the "programming perl" book from O'Reilly, and for
quick searches, see perldoc perltoc.

--
Joost Diepenmaat | blog: http://joost.zeekat.nl/ | work: http://zeekat.nl/

Re: Can I iterate through a file on a CGI page?

Joost Diepenmaat wrote:
> Rich Grise writes:
>
>> Yeah, you're right, I'm sorry, but I've been reading for going on about
>> a month now and it's terribly frustrating - it's pretty much the same
>> problem with all the Linux docs - you have to already know what feature
>> you're looking for before you know which doc to read, like for example:
>> `readlink` - how is a person supposed to know that such a thing even
>> exists, except by asking, or plowing through every possible doc there
>> is until you stumble upon it? ?:-/
>
> For *nix programming in general, I really recommend the "advanced
> programming in the unix environment, 2nd edition" book from Addisson
> Wesley. For perl, get the "programming perl" book from O'Reilly, and for
> quick searches, see perldoc perltoc.
>

Also you may use the power of the that there Internet. For fun I
searched on "perl symbolic link" and readlink was covered in the
second link of the results.

Re: Can I iterate through a file on a CGI page?

"J. Gleixner" wrote in
news:47fe9899$0$89395$815e3792@news.qwest.net:

> Joost Diepenmaat wrote:
>> Rich Grise writes:
>>
....
>>> which doc to read, like for example: `readlink` - how is a
>>> person supposed to know that such a thing even exists, except by
>>> asking, or plowing through every possible doc there is until you
>>> stumble upon it? ?:-/
>>
....
>> for quick searches, see perldoc perltoc.
>>
>
> Also you may use the power of the that there Internet. For fun I
> searched on "perl symbolic link" and readlink was covered in the
> second link of the results.

I also think the most natural place to look first is:

perldoc perltoc

from which one finds out

perlfunc - Perl builtin functions

Then, reading perldoc perlfunc, one notices

Functions for filehandles, files, or directories
"-*X*", "chdir", "chmod", "chown", "chroot", "fcntl", "glob",
"ioctl", "link", "lstat", "mkdir", "open", "opendir", "readlink",
"rename", "rmdir", "stat", "symlink", "sysopen", "umask",
"unlink", "utime"

One can then learn more about the four functions that have 'link' as
part of their names.

perldoc -f link
perldoc -f readlink
perldoc -f symlink
perldoc -f unlink

I would assume that somewhere along this short and painless process,
one would find out what one needs.

Look 'ma: No Google. No UseNet. No books. Just the basic Perl
documentation.

After a while, one can develop a sense of where to find information
with even less searching.

On the other hand, if one can always rely on being handed a fish for
the asking, one will never have an incentive to put one's own time
and effort into researching answers.

Sinan

--
A. Sinan Unur <1usa@llenroc.ude.invalid>
(remove .invalid and reverse each component for email address)

comp.lang.perl.misc guidelines on the WWW:
http://www.rehabitation.com/clpmisc/

Re: Can I iterate through a file on a CGI page?

In article , Rich Grise
wrote:

> On Tue, 08 Apr 2008 16:12:14 -0700, Jim Gibson wrote:
>
> > In article <47fbe8c0$0$33227$815e3792@news.qwest.net>, J. Gleixner
> > wrote:
> >> Rich Grise wrote:
>
> >> >
> >> > my $line=`ls -l source-links | head -2 | tail -1`;
> >> > chomp $line;
> >> > my $linkname = substr("$line", 50, 12);
> >> > my $targetname = substr("$line", 66);
> >> > print("$line\n");
> >> > print("name = $linkname, target = $targetname\n");
> >>
> >> Not very reliable. If the owner or group change to something
> >> of different length, it could easily return the wrong data. Using
> >> split would be more reliable, but still not very good.
> >>
> >> Use the correct function: perldoc -f readlink
> >
> > This should work and doesn't require shelling out to ls:
> >
> > use strict;
> > use warnings;
> > use File::Find;
> > find(
> > sub {
> > my $linkname = $_;
> > return unless -l $_;
> > my $targetname = readlink $linkname;
> > print "name = $linkname, target = $targetname\n";
> > },
> > 'source-links'
> > );
>
> OK, I do this, and it shows me _all_ of the links. What have I
> missed?
>
> And am I supposed to pass the name of the link I'm looking for? Why
> do you initialize $linkname to $_? I want a script that doesn't
> have to already know $linkname - it needs to find the next one in
> the subdir, whichever one that is.
>
> What am I missing?

Nothing, really. I didn't explain why my program prints out all of the
links while yours only does one. I thought you might be able to figure
out how to modify this program, which is just an example of fetching
_all_ symbolic links in a directory.

If you read the documentation for File::Find, you will see that the
module will first cd to the target directory containing the file and
put the name of the file in the variable $_ before calling what the
docs call the "wanted" subroutine, which is actually an anonymous
subroutine in my program. I copied $_ to $linkname to save the name in
case you wanted to do anything that might change the $_ variable, which
is global.

If you only want to process one file at each execution, you can
terminate your program after the first one, or save the name of the
first one in a variable and only process that one after find() has
finished.

However, you may want to process _all_ files in your subdirectory, in
which case using File::Find makes it very easy to iterate over all of
the files in a subdirectory tree.

--
Jim Gibson

Posted Via Usenet.com Premium Usenet Newsgroup Services
----------------------------------------------------------
http://www.usenet.com

Re: Can I iterate through a file on a CGI page?

On Thu, 10 Apr 2008 18:28:20 +0000, Rich Grise wrote:

For what it's worth, I've used a lot of the suggestions here, and I
have a successful script - now comes the tedious part - sitting and
looking at 17,000 pictures, one at a time, with a "keep" and "skip"
button. :-)

Many Thanks to All!
Rich