File::Find problem?

I'm using the File::Find module to search through a directory tree and
perform operations on files with a .hdr suffix for the file name. I
have a test environment set up and known defects in the files, but I
get an error I hadn't expected with the find() function. The set up
is like this:

use File::Find;

find (\&process_file,".");

sub process_file {
if (/\.hdr/) {
print "$File::Find::name\n";
open HDR_FILE, "<$File::Find::name" or die "Can't open file
for input: $!\n";
...etc.

It prints the correct file name, complete with relative path, but it
dies on the open statement every time. What am I doing wrong?

Thanks in advance.

Re: File::Find problem?

Monty wrote:
> I'm using the File::Find module to search through a directory tree and
> perform operations on files with a .hdr suffix for the file name. I
> have a test environment set up and known defects in the files, but I
> get an error I hadn't expected with the find() function. The set up
> is like this:
>
> use File::Find;
>
> find (\&process_file,".");
>
> sub process_file {
> if (/\.hdr/) {

Note: this only means that '.hdr' occurs in the name of the file. It
would match some.file.hdr.gif too.

> print "$File::Find::name\n";

print " is readable\n" if -r $File::Find::name;
print " does exist\n" if -e $File::Find::name;

> open HDR_FILE, "<$File::Find::name" or die "Can't open file
> for input: $!\n";
> ...etc.
>
> It prints the correct file name, complete with relative path, but it
> dies on the open statement every time. What am I doing wrong?

Well, what's the error?
What's a value of $File::Find::name?
Do you have read access to the file?
Is it a symlink to a file that doesn't exist?

Re: File::Find problem?

Ah, I left a few things out.

The error is as in the die statement: Can't open file for input. All
files in the test environment have permissions of 777 (that includes
the directories and sub-directories), so anyone should be able to
read, write, or execute them (although they're plain text files). On
top of that, I'm running this as root.

Also, I mistyped the 'if' statement some, it actually reads 'if (/\.hdr
$/)'. That part seems to work just fine.

I'm stumped, but thanks for taking a look at this.

Monty

Re: File::Find problem?

Oh yes, none of these files are symbolic links.

Monty

Re: File::Find problem?

On Aug 10, 12:42 pm, Monty wrote:
> Ah, I left a few things out.
>
> The error is as in the die statement: Can't open file for input. All

What's the second half of the error message i.e., what's the value of
$! (the OS portion of the error)?
> files in the test environment have permissions of 777 (that includes
> the directories and sub-directories), so anyone should be able to
> read, write, or execute them (although they're plain text files). On
> top of that, I'm running this as root.
>
> Also, I mistyped the 'if' statement some, it actually reads 'if (/\.hdr
> $/)'. That part seems to work just fine.
>
> I'm stumped, but thanks for taking a look at this.
>
> Monty

Re: File::Find problem?

The O/S portion of the error is "No such file or directory".

The file exists where $File::Find::name says it is. I wonder if the
problem is in the relative pathing?

Re: File::Find problem?

Monty wrote:
> Oh yes, none of these files are symbolic links.

Please post your real code (don't re-type it), together with the output
you get when you run that very code.

This, for example, is based on your code and works on my Win XP machine.
(I've replaced your "hdr" extension with "doc" for convenience).

#! /usr/bin/perl
use strict;
use warnings;
use File::Find;

find (\&process_file,".");

sub process_file {
if (/\.doc$/) {
print "$File::Find::name ...";
print "exists, " if -e $File::Find::name;
print "is readable, " if -r $File::Find::name;
open HDR_FILE, "<$File::Find::name" or die "Can't open file for
input: $!\n";
print "opened OK\n";
close HDR_FILE;
}
}

Output:
F:\WIP>clpm.pl
../IAM Capability Strategy V1.11.doc ...exists, is readable, opened OK
../Annual General Meeting 07.doc ...exists, is readable, opened OK
and so on.


--

Henry Law Manchester, England

Re: File::Find problem?

Henry Law wrote:

> Output:
> F:\WIP>clpm.pl
> ../IAM Capability Strategy V1.11.doc ...exists, is readable, opened OK
> ../Annual General Meeting 07.doc ...exists, is readable, opened OK
> and so on.

For some reason Thunderbird has put two dots where my original output
has one. The console shows one dot (the current directory).
../IAM Capability Strategy V1.11.doc ...exists, is readable, opened OK
../Annual General Meeting 07.doc ...exists, is readable, opened OK

(Unless TBird munges it again!)

--

Henry Law Manchester, England

Re: File::Find problem?

On Aug 10, 1:04 pm, Monty wrote:
>>> I wonder if the problem is in the relative pathing?

I'll bet it is.

Are you running the program from the same root path that the find is
looking at? Do you chdir() either to or away from that root path in
your program?


--
The best way to get a good answer is to ask a good question.
David Filmer (http://DavidFilmer.com)

Re: File::Find problem?

On Aug 10, 12:21 pm, "J. Gleixner" no.invalid> wrote:
> Monty wrote:
> > open HDR_FILE, "<$File::Find::name" or die "Can't open file
> > for input: $!\n";

Are there spaces in the filename and/or path?

Since the default is to cd into each dir, you should be able ti use $_
instead of $File::Find::name
Try changing the open call to this:

open(HDR_FILE, '<', "$_") || die "Can't open file or input: $!";

Re: File::Find problem?

Ron Bergin wrote:

> Try changing the open call to this:
>
> open(HDR_FILE, '<', "$_") || die "Can't open file or input: $!";


perldoc -q vars

What’s wrong with always quoting "$vars"?

then try instead:

open(HDR_FILE, '<', $_) || die "Can't open file or input: $!";


--
Tad McClellan
email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"

Re: File::Find problem?

On Aug 10, 5:45 pm, Tad McClellan wrote:
> Ron Bergin wrote:
> > Try changing the open call to this:
>
> > open(HDR_FILE, '<', "$_") || die "Can't open file or input: $!";
>
> perldoc -q vars
>
> What's wrong with always quoting "$vars"?
>
> then try instead:
>
> open(HDR_FILE, '<', $_) || die "Can't open file or input: $!";
>
> --
> Tad McClellan
> email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"

Tad,

I'm aware of the reasons for not always quoting "$vars" and I rarely
do quote them, however, I did in this case to safeguard against the
possibility of having spaces in the filename, in which case the open
call would fail if it was not quoted. I know that spaces in filenames
aren't real common on *nix systems, but on occasion it does happen and
on Windows systems it's very common.

I probably should have been more clear as to my reasoning when I
posted that suggestion. Additionally, we both left out one important
piece...we should have included $_ or $File::Find::name within the die
statement.

Re: File::Find problem?

On Aug 10, 7:24 pm, Ron Bergin wrote:
> On Aug 10, 5:45 pm, Tad McClellan wrote:
>
>
>
> > Ron Bergin wrote:
> > > Try changing the open call to this:
>
> > > open(HDR_FILE, '<', "$_") || die "Can't open file or input: $!";
>
> > perldoc -q vars
>
> > What's wrong with always quoting "$vars"?
>
> > then try instead:
>
> > open(HDR_FILE, '<', $_) || die "Can't open file or input: $!";
>
> > --
> > Tad McClellan
> > email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
>
> Tad,
>
> I'm aware of the reasons for not always quoting "$vars" and I rarely
> do quote them, however, I did in this case to safeguard against the
> possibility of having spaces in the filename, in which case the open
> call would fail if it was not quoted.

Hmmm It looks like I need to correct myself. I just ran a test on my
*nix box without quoting the var and it did open a file with spaces in
the name. I know that I've had problems in the past with spaces in
the names, but that may have been on older version of Perl on Windows.

Re: File::Find problem?

Ron Bergin writes:

> I'm aware of the reasons for not always quoting "$vars" and I rarely
> do quote them, however, I did in this case to safeguard against the
> possibility of having spaces in the filename, in which case the open
> call would fail if it was not quoted.

Where on Earth did you get that idea???

Filenames with spaces need to be quoted when used in a shell command,
because the command interpreter would otherwise interpret the spaces as
delimiters in between arguments. Open() is not a shell command.

sherm--

--
Web Hosting by West Virginians, for West Virginians: http://wv-www.net
Cocoa programming in Perl: http://camelbones.sourceforge.net

Re: File::Find problem?

Ron Bergin writes:

> Hmmm It looks like I need to correct myself. I just ran a test on my
> *nix box without quoting the var and it did open a file with spaces in
> the name. I know that I've had problems in the past with spaces in
> the names, but that may have been on older version of Perl on Windows.

The Perl version is irrelevant - spaces are treated no differently in Perl
now than they were ten years ago.

You may have been passing a string to a command interpeter with system; in
that case you would have needed to insert the proper quoting in the command.

system 'copy c:\foo "c:\foo bar baz"';

But that's got nothing to do with Perl; you would need the quotes if you
were running the command from Ruby, C, or typing it by hand.

sherm--

--
Web Hosting by West Virginians, for West Virginians: http://wv-www.net
Cocoa programming in Perl: http://camelbones.sourceforge.net

Re: File::Find problem?

>>>>> "RB" == Ron Bergin writes:

RB> Hmmm It looks like I need to correct myself. I just ran a test on my
RB> *nix box without quoting the var and it did open a file with spaces in
RB> the name. I know that I've had problems in the past with spaces in
RB> the names, but that may have been on older version of Perl on Windows.

spaces in file names are allowed under almost all filesystems. most any
programming language api for open can handle spaces just fine. you are
probably thinking about quoting those names when used in the shell. but
when you shell out you need to use quotes at the shell level but inside
a string in the perl code and then it can be confusing.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org

Re: File::Find problem?

Monty wrote:
> The O/S portion of the error is "No such file or directory".
>
> The file exists where $File::Find::name says it is. I wonder if the
> problem is in the relative pathing?

That is exactly the problem. $File::Find::name contains the complete path of
the file. The problem is that File::Find::find defaults to changing
directories so that the file in question is always in the current directory.
So if $File::Find::name contains './one/two/three/file.txt' you are trying to
open a file in './one/two/three/./one/two/three/file.txt'. You need to either
use the "no_chdir" option or use just the file name in $_.


John
--
Perl isn't a toolbox, but a small machine shop where you
can special-order certain sorts of tools at low cost and
in short order. -- Larry Wall

Re: File::Find problem?

Ron Bergin wrote:
> On Aug 10, 5:45 pm, Tad McClellan wrote:
>> Ron Bergin wrote:
>>> Try changing the open call to this:
>>> open(HDR_FILE, '<', "$_") || die "Can't open file or input: $!";
>
> I'm aware of the reasons for not always quoting "$vars" and I rarely
> do quote them, however, I did in this case to safeguard against the
> possibility of having spaces in the filename, in which case the open
> call would fail if it was not quoted.

Not true for the three-argument open(). Check the docs, and look
for the line "Use 3-argument form to open a file with arbitrary
weird characters in it...". Leading and trailing blanks
are preserved when the file name is in the third arg.

It's the two-argument open() that eats leading and trailing blanks, leading to

open FH,"< ./$_\0" or die;

-Joe

Re: File::Find problem?

>>>>> "Monty" == Monty writes:

Monty> I'm using the File::Find module to search through a directory tree and
Monty> perform operations on files with a .hdr suffix for the file name. I
Monty> have a test environment set up and known defects in the files, but I
Monty> get an error I hadn't expected with the find() function. The set up
Monty> is like this:

Monty> use File::Find;

Monty> find (\&process_file,".");

Monty> sub process_file {
Monty> if (/\.hdr/) {
Monty> print "$File::Find::name\n";
Monty> open HDR_FILE, "<$File::Find::name" or die "Can't open file
Monty> for input: $!\n";
Monty> ...etc.

Monty> It prints the correct file name, complete with relative path, but it
Monty> dies on the open statement every time. What am I doing wrong?

You are opening $File::Find::name while still in the context of the
callback. You should either (a) put $File::Find::name into an array
to process when everything is done or (b) use $_ instead of $File::Find::name,
which is properly adjusted for the current directory.

File::Find does a whole lot of chdir'ing. So you have to comply.

print "Just another Perl hacker,"; # the original

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095

Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

--
Posted via a free Usenet account from http://www.teranews.com

Re: File::Find problem?

>>>>> "Henry" == Henry Law writes:

Henry> This, for example, is based on your code and works on my Win XP machine. (I've
Henry> replaced your "hdr" extension with "doc" for convenience).

Try an example with a subdir, and it won't.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095

Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

--
Posted via a free Usenet account from http://www.teranews.com

Re: File::Find problem?

Ok, my apologies for not getting back to anyone over the weekend.

I've had a chance to look over the solutions and I've implemented some
additional troubleshooting steps, but the result is the same.

Henry Law: you pretty much have verbatim what I've coded.
John Krahn: I replaced $File::Find::name with $_ and got the same
results.

There are no spaces in any of the file names. I also verified there
were no trailing spaces by doing an ls -aF on all the files.

How about setting the depth of the find()? The documentation leads me
to believe that the depth is infinite (not really, but such as that
is) by default, but I wonder is that really the case? This find()
I've coded has to go two directories deep before it finds a matching
file.

Thanks

Re: File::Find problem?

Monty wrote:

> How about setting the depth of the find()?

That is one of the reasons why I added 'preprocess' to File::find().

our $depth;
find( {
wanted => &your_sub,
preprocess => {return() if ++$depth > $max_depth},
postprocess => {$depth--}
}, @ARGV);




-Joe

Re: File::Find problem?

Monty wrote:

> How about setting the depth of the find()?

The version of File::Find distributed with perl v5.8.8 for cygwin has a problem.

It does not recognize that Windows Vista has links to directories.
The lstat() function treats these links as actual directories and not
as a symlink. The result is that File::Find processes certain directories
more than once, sometimes infinitely. (The option 'follow_skip' only looks
at symlinks, so is no help here.)

(/cygdrive/c) jms@wiggin# cat /tmp/find
#! /usr/bin/perl
use strict; use warnings;
use File::Find ();
use vars qw/*name *dir *prune/;
*name = *File::Find::name;
*dir = *File::Find::dir;
*prune = *File::Find::prune;

sub wanted;

# Traverse desired filesystems
File::Find::find({wanted => \&wanted}, 'C:/Users');
exit unless @prune;
print "File system loop (hardlink to directory) detected:\n", sort @prune;

sub wanted {
if ((my($dev,$ino,$mode,$nlink,$uid,$gid) = lstat $_) && -d _) {
# print("$name\n");
my $dev_ino = ($dev||1) . ":" . ($ino||2);
if (defined $prune{$dev_ino}) {
push @prune,"$name => $prune{$dev_ino}\n";
print STDERR $prune[-1];
$prune = 1;
} else {
$prune{$dev_ino} = $name; # Watch out for Windows Vista
}
}
}

(/cygdrive/c) jms@wiggin# perl /tmp/find
File system loop (hardlink to directory) detected:
C:/Users/Default/AppData/Local/Application Data => C:/Users/Default/AppData/Local
C:/Users/Default/AppData/Local/Microsoft/Windows/History => C:/Users/Default/AppData/Local/History
C:/Users/Default/AppData/Local/Temporary Internet Files => C:/Users/Default/AppData/Local/Microsoft/Windows/Temporary Internet Files
C:/Users/Default/Application Data => C:/Users/Default/AppData/Roaming
C:/Users/Default/Cookies => C:/Users/Default/AppData/Roaming/Microsoft/Windows/Cookies
C:/Users/Default/Local Settings => C:/Users/Default/AppData/Local
C:/Users/Default/NetHood => C:/Users/Default/AppData/Roaming/Microsoft/Windows/Network Shortcuts
C:/Users/Default/PrintHood => C:/Users/Default/AppData/Roaming/Microsoft/Windows/Printer Shortcuts
C:/Users/Default/Recent => C:/Users/Default/AppData/Roaming/Microsoft/Windows/Recent
C:/Users/Default/SendTo => C:/Users/Default/AppData/Roaming/Microsoft/Windows/SendTo
C:/Users/Default/Start Menu => C:/Users/Default/AppData/Roaming/Microsoft/Windows/Start Menu
C:/Users/Default/Templates => C:/Users/Default/AppData/Roaming/Microsoft/Windows/Templates
C:/Users/Guest/AppData/Local/Application Data => C:/Users/Guest/AppData/Local
C:/Users/Guest/AppData/Local/Microsoft/Windows/History => C:/Users/Guest/AppData/Local/History
C:/Users/Guest/AppData/Local/Temporary Internet Files => C:/Users/Guest/AppData/Local/Microsoft/Windows/Temporary Internet Files
C:/Users/cyg_server/AppData/Local/Application Data => C:/Users/cyg_server/AppData/Local
C:/Users/cyg_server/AppData/Local/Microsoft/Windows/History => C:/Users/cyg_server/AppData/Local/History
C:/Users/cyg_server/AppData/Local/Temporary Internet Files => C:/Users/cyg_server/AppData/Local/Microsoft/Windows/Temporar y Internet Files
C:/Users/jms/AppData/Local/Application Data => C:/Users/jms/AppData/Local
C:/Users/jms/AppData/Local/Microsoft/Windows/History => C:/Users/jms/AppData/Local/History
C:/Users/jms/AppData/Local/Temporary Internet Files => C:/Users/jms/AppData/Local/Microsoft/Windows/Temporary Internet Files
C:/Users/jms/Application Data => C:/Users/jms/AppData/Roaming
C:/Users/jms/Cookies => C:/Users/jms/AppData/Roaming/Microsoft/Windows/Cookies
C:/Users/jms/Local Settings => C:/Users/jms/AppData/Local
C:/Users/jms/NetHood => C:/Users/jms/AppData/Roaming/Microsoft/Windows/Network Shortcuts
C:/Users/jms/PrintHood => C:/Users/jms/AppData/Roaming/Microsoft/Windows/Printer Shortcuts
C:/Users/jms/Recent => C:/Users/jms/AppData/Roaming/Microsoft/Windows/Recent
C:/Users/jms/SendTo => C:/Users/jms/AppData/Roaming/Microsoft/Windows/SendTo
C:/Users/jms/Start Menu => C:/Users/jms/AppData/Roaming/Microsoft/Windows/Start Menu
C:/Users/jms/Templates => C:/Users/jms/AppData/Roaming/Microsoft/Windows/Templates
C:/Users/sshd_server/AppData/Local/Application Data => C:/Users/sshd_server/AppData/Local
C:/Users/sshd_server/AppData/Local/Microsoft/Windows/History => C:/Users/sshd_server/AppData/Local/History
C:/Users/sshd_server/AppData/Local/Temporary Internet Files => C:/Users/sshd_server/AppData/Local/Microsoft/Windows/Tempora ry Internet Files

Re: File::Find problem?

To all who have responded with potential solutions, I thank you for
your interest.

I did find a solution to my problem once I understood more of the
documentation. I coded a hash with the process name and the no_chdir
option set to 1, and everything works as I expected. The call to find
then becomes find(\%refhash,".") and the script appears to find, open,
and process all the files in all the subdirectories (there are 256 of
them, and a count within the script agrees with that).

Thanks again for all your input.
Monty