concatenation and moving files

concatenation and moving files

am 20.11.2008 17:38:45 von zilore mumba

--===============0728158843==
Content-Type: multipart/alternative; boundary="0-696951707-1227199125=:68863"

--0-696951707-1227199125=:68863
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

Thanks so much for all who assisted me with my script, and more especially =
Bill Luebkert. The script now works, but I have a minor problem. Sorry for =
sending so much code, but see my comments within the code for the minor pro=
blem.
Zilore

#!/usr/bin/perl -w
use strict;
use warnings;
use POSIX;
use File::Path;
use File::Copy;
#
# define stuff you're using that's not defined
#
my $debug =3D 0;
#
my @now =3D localtime;
my $hr =3D strftime "%H", @now;
my $yr =3D strftime "%y", @now;
my $mn =3D strftime "%B", @now;
my $mnn =3D strftime "%m", @now;
#
my $dat =3D `date +%Y%m%d`; chomp $dat;
my $dat1 =3D $dat + 1;
my $dsh ="-";
#
my $model =3D 'ukmo';
my $uksufx =3D '2';
my $cycle =3D '00'; $cycle=3D'12' if $hr >15;
my $grib_ext =3D '.grib';
my $ctl_ext =3D '.ctl';
#
my $base_dir =3D "/home/Mumba/archive/$mnn$dsh$mn$yr/$model";
my $ofile =3D "$base_dir/$model$cycle$grib_ext";
#
# cd to base/date dir
#
chdir "$base_dir/$dat" or die "chdir '$base_dir/$dat': $! ($^E)";
Here am in the directory: "/home/Mumba/archive/11-November08/ukmo/20081120"
#
# open output file - put it up a dir in base_dir so it doesn't get deleted
#
print "Creating output file '$ofile'\n" if $debug;
open OUT, ">$ofile1" or die "Create '$ofile: $! ($^E)";
binmode OUT;
$ofile is created in: "/home/Mumba/archive/11-November08/ukmo", which is go=
od because
20081120 contains many files to be concatenated and $ofile should'nt be in =
the bundle.
#
# do foreach file in dir
#
opendir DIR, "$base_dir/$dat" or die "opendir '$base_dir/$dat' Failed: $! (=
$^E)";
while (my $file =3D readdir DIR) {
  =A0 next if $file =3D~ /^\.\.?/;
  =A0 print "Adding '$file'\n" if $debug;
  =A0 open IN, $file or die "open '$file': $! ($^E)";
  =A0 binmode IN;
  =A0 while () { print OUT $_; }
  =A0 close IN;
}
closedir DIR;
close OUT;
The files in "/home/Mumba/archive/11-November08/ukmo/20081120" have been co=
ncatenated and written to $ofile (ukmo00.grb) which is still in "/home/Mumb=
a/archive/11-November08/ukmo".
$ofile has to be moved into the date directory because the next day a new $=
ofile will be created with the same name and will replace the one created t=
he previous day.
I tried the code below for moving $ofile from:
"/home/Mumba/archive/11-November08/ukmo"
to
"/home/Mumba/archive/11-November08/ukmo/20081120"
The file does not move (and no error is indicated).
Assistance please.
opendir DIR, "$base_dir" or die "opendir '$base_dir' Failed: $! ($^E)";
while (my $file=3Dreaddir DIR) {
next if $file=3D"$model$cycle$grib_ext";
while (<$file>) { rename "$file", "$base_dir/$dat" or warn "rename '$file' =
failed: $! ($^E)";}
}
closedir DIR;
#
my $cmd =3D "/usr/bin/perl /usr/local/bin/grib2ctl.pl -verf $base_dir/$mode=
l$cycle$grib_ext > $base_dir/$model$cycle$ctl_ext";
print "doing system ($cmd)\n" if $debug;
system ($cmd);
#
$cmd =3D "/usr/local/grads/bin/gribmap -i $base_dir/$model$cycle$ctl_ext -v=
-e";
print "doing system ($cmd)\n" if $debug;
system ($cmd);
#
opendir DIR, "$base_dir/$dat" or die "opendir '$base_dir/$dat': $! ($^E)";
while (my $file =3D readdir DIR) {
  =A0 next if $file =3D~ /^\.\.?/;
  =A0 print "unlinking '$file'\n" if $debug;
  =A0 unlink $file or warn "unlink '$file' failed: $! ($^E)";
}
closedir DIR;
#

=0A
--0-696951707-1227199125=:68863
Content-Type: text/html; charset=us-ascii

Thanks so much for all who assisted me with my script, and more especially Bill Luebkert. The script now works, but I have a minor problem. Sorry for sending so much code, but see my comments within the code for the minor problem.

Zilore


#!/usr/bin/perl -w
use strict;
use warnings;
use POSIX;
use File::Path;
use File::Copy;
#
# define stuff you're using that's not defined
#
my $debug = 0;
#
my @now = localtime;
my $hr = strftime "%H", @now;
my $yr = strftime "%y", @now;
my $mn = strftime "%B", @now;
my $mnn = strftime "%m", @now;
#
my $dat = `date +%Y%m%d`; chomp $dat;
my $dat1 = $dat + 1;
my $dsh ="-";
#
my $model = 'ukmo';
my $uksufx = '2';
my $cycle = '00'; $cycle='12' if $hr >15;
my $grib_ext = '.grib';
my $ctl_ext = '.ctl';
#
my $base_dir = "/home/Mumba/archive/$mnn$dsh$mn$yr/$model";
my $ofile = "$base_dir/$model$cycle$grib_ext";
#
# cd to base/date dir
#
chdir "$base_dir/$dat" or die "chdir '$base_dir/$da
t': $! ($^E)";

Here am in the directory: "/home/Mumba/archive/11-November08/ukmo/20081120"
#
# open output file - put it up a dir in base_dir so it doesn't get deleted
#
print "Creating output file '$ofile'\n" if $debug;
open OUT, ">$ofile1" or die "Create '$ofile: $! ($^E)";
binmode OUT;

$ofile is created in: "/home/Mumba/archive/11-November08/ukmo", which is good because

20081120 contains many files to be concatenated and $ofile should'nt be in the bundle.
#
# do foreach file in dir
#
opendir DIR, "$base_dir/$dat" or die "opendir '$base_dir/$dat' Failed: $! ($^E)";
while (my $file = readdir DIR) {
    next if $file =~ /^\.\.?/;
    print "Adding '$file'\n" if $debug;
    open IN, $file or die "open '$file': $! ($^E)";
    binmode IN;
    while (<IN>) { print OUT $_; }
    close IN;
}
closedir DIR;
close OUT;

The files in "/home/Mumba/archive/11-November08/ukmo/20081120" have been concatenated and written to $ofile (ukmo00.grb) which is still in "/home/Mumba/archive/11-November08/ukmo".

$ofile has to be moved into the date directory because the next day a new $ofile will be created with the same name and will replace the one created the previous day.

I tried the code below for moving $ofile from:

"/home/Mumba/archive/11-November08/ukmo"

to

"/home/Mumba/archive/11-November08/ukmo/20081120"

The file does not move (and no error is indicated).

Assistance please.
opendir DIR, "$base_dir" or die "opendir '$base_dir' Failed: $! ($^E)";
while (my $file=readdir DIR) {
next if $file="$model$cycle$grib_ext";
while (<$file>) { rename "$file", "$base_dir/$dat" or warn "rename '$file' failed: $! ($^E)";}
}
closedir DIR;
#
my $cmd = "/usr/bin/perl /usr/local/bin/grib2ctl.pl -verf $base_dir/$model$cycle$grib_ext > $base_dir/$model$cycle$ctl_ext";
print "doing system ($cmd)\n" if $debug;
system ($cmd);
#
$cmd = "/usr/local/grads/bin/gribmap -i $base_dir/$model$cycle$ctl_ext -v -e";
print "doing system ($cmd)\n" if $debug;
system ($cmd);
#
opendir DIR, "$base_dir/$dat" or die "opendir '$base_dir/$dat': $! ($^E)";
while (my $f
ile = readdir DIR) {
    next if $file =~ /^\.\.?/;
    print "unlinking '$file'\n" if $debug;
    unlink $file or
warn "unlink '$file' failed: $! ($^E)";
}
closedir DIR;
#





--0-696951707-1227199125=:68863--

--===============0728158843==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
ActivePerl mailing list
ActivePerl@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
--===============0728158843==--

RE: concatenation and moving files

am 20.11.2008 18:18:47 von Henry Hartley

zilore mumba wrote:

>> Thanks so much for all who assisted me with my script, and
>> more especially Bill Luebkert. The script now works, but I
>> have a minor problem. Sorry for sending so much code, but
>> see my comments within the code for the minor problem.
>>
>> The file does not move (and no error is indicated).
>> Assistance please.
>> while (<$file>) { rename "$file", "$base_dir/$dat" or warn
>> "rename '$file' failed: $! ($^E)";}

Try using File::Copy.

--
Henry
_______________________________________________
ActivePerl mailing list
ActivePerl@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs

Re: concatenation and moving files

am 21.11.2008 03:29:19 von Bill Luebkert

zilore mumba wrote:
>
> The files in "/home/Mumba/archive/11-November08/ukmo/20081120" have been
> concatenated and written to $ofile (ukmo00.grb) which is still in
> "/home/Mumba/archive/11-November08/ukmo".
> $ofile has to be moved into the date directory because the next day a
> new $ofile will be created with the same name and will replace the one
> created the previous day.
> I tried the code below for moving $ofile from:
> "/home/Mumba/archive/11-November08/ukmo"
> to
> "/home/Mumba/archive/11-November08/ukmo/20081120"

Just move it:

rename "$base_dir/$ofile", "$base_dir/ukmo/$dat" or
... die stuff here

> The file does not move (and no error is indicated).
> Assistance please.
> opendir DIR, "$base_dir" or die "opendir '$base_dir' Failed: $! ($^E)";
> while (my $file=readdir DIR) {
> next if $file="$model$cycle$grib_ext";
> while (<$file>) { rename "$file", "$base_dir/$dat" or warn "rename
> '$file' failed: $! ($^E)";}
> }
> closedir DIR;
> #
_______________________________________________
ActivePerl mailing list
ActivePerl@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs