wget in scripts

Hi,
Can anyone help me with this.

I'm using wget, a perl script and a shell script to download files
from serveral directories on a web server.

The URLs I'm passing to wget are of the form:

http://www.foo.br/001/ - http://www.foo.br/050/

My perl script loops through the numbers 1-50, inserting each
iteration in a string representing the URL. I call it from my shell
script and redirect output to a text file. I then call wget from the
same shell script passing the name of this text file to wget's
--input-file option.

I want to do this all in Perl and avoid messing around with shell
scripts or batch files.

Also, is there a simple way of constructing the numbers 001-050 for my
URL strings?

I don't like what I'm doing at the moment which is....

$max = 51;
$zero = "0";
$zerozero = "00"

for ($i=1; $i<10; $i++) {
print "$zerozero$i\n";
}
for ($i=10; $i<$max; $i++) {
print "$zero$i\n";
}

Cheers,

Malcolm

Re: wget in scripts

Malcolm Mill wrote:
> Hi,
> Can anyone help me with this.
>
> I'm using wget, a perl script and a shell script to download files
> from serveral directories on a web server.
>
> The URLs I'm passing to wget are of the form:
>
> http://www.foo.br/001/ - http://www.foo.br/050/
>
> My perl script loops through the numbers 1-50, inserting each
> iteration in a string representing the URL. I call it from my shell
> script and redirect output to a text file. I then call wget from the
> same shell script passing the name of this text file to wget's
> --input-file option.

This depends a little on what options you're passing to wget. Do you
wish to just get the file? In which case LWP::Simple might be suitable.

#!/usr/bin/perl -wT
use strict;
use LWP::Simple;

my $url =
"http://search.cpan.org/~gaas/libwww-perl-5.800/lib/LWP/Simp le.pm";
my $file = "LWP::Simple_Documentation.html";

my $response = getstore($url, $file);

> I want to do this all in Perl and avoid messing around with shell
> scripts or batch files.
>
> Also, is there a simple way of constructing the numbers 001-050 for my
> URL strings?

Use sprintf.

foreach (1..50) {
print sprintf("%03d", $_);
}

All the best,

Jacinta

--
("`-''-/").___..--''"`-._ | Jacinta Richardson |
`6_ 6 ) `-. ( ).`-.__.`) | Perl Training Australia |
(_Y_.)' ._ ) `._ `. ``-..-' | +61 3 9354 6001 |
_..`--'_..-_/ /--'_.' ,' | contact@perltraining.com.au |
(il),-'' (li),' ((!.-' | www.perltraining.com.au |

Re: wget in scripts

Malcolm Mill wrote:
> Hi,
> Can anyone help me with this.
>
> I'm using wget, a perl script and a shell script to download files
> from serveral directories on a web server.
>
> The URLs I'm passing to wget are of the form:
>
> http://www.foo.br/001/ - http://www.foo.br/050/

Untested:

#!/usr/bin/perl

use strict;
use warnings;

use LWP::Simple;

foreach my $id ( '001'..'050' ) {
my $content = get("http://www.foo.br/$id/");
die "Couldn't get it!" unless defined $content;
# ...
# do something with $content
}

_______________________________________________
ActivePerl mailing list
ActivePerl@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs

Re: wget in scripts

Malcolm Mill wrote:
> Hi,
> Can anyone help me with this.
>
> I'm using wget, a perl script and a shell script to download files
> from serveral directories on a web server.
>
> The URLs I'm passing to wget are of the form:
>
> http://www.foo.br/001/ - http://www.foo.br/050/

Untested:

#!/usr/bin/perl

use strict;
use warnings;

use LWP::Simple;

foreach my $id ( '001'..'050' ) {
my $content = get("http://www.foo.br/$id/");
die "Couldn't get it!" unless defined $content;
# ...
# do something with $content
}

_______________________________________________
ActivePerl mailing list
ActivePerl@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs

Re: wget in scripts

Malcolm Mill wrote:

> Hi,
> Can anyone help me with this.
>
> I'm using wget, a perl script and a shell script to download files
> from serveral directories on a web server.
>
> The URLs I'm passing to wget are of the form:
>
> http://www.foo.br/001/ - http://www.foo.br/050/
>
> My perl script loops through the numbers 1-50, inserting each
> iteration in a string representing the URL. I call it from my shell
> script and redirect output to a text file. I then call wget from the
> same shell script passing the name of this text file to wget's
> --input-file option.
>
> I want to do this all in Perl and avoid messing around with shell
> scripts or batch files.
>
> Also, is there a simple way of constructing the numbers 001-050 for my
> URL strings?
>
> I don't like what I'm doing at the moment which is....
>
> $max = 51;
> $zero = "0";
> $zerozero = "00"
>
> for ($i=1; $i<10; $i++) {
> print "$zerozero$i\n";
> }
> for ($i=10; $i<$max; $i++) {
> print "$zero$i\n";
> }

DO NOT CROSS-POST. One group is sufficient beginners or active-perl
would be fine. I'm responding to active-perl only.

use strict;

my $url = 'http://www.foo.br';
foreach (1 .. 50) {
printf "%s/%03u/\n", $url, $_;
}

__END__


--
,-/- __ _ _ $Bill Luebkert Mailto:dbecoll@adelphia.net
(_/ / ) // // DBE Collectibles Mailto:dbe@todbe.com
/ ) /--< o // // Castle of Medieval Myth & Magic http://www.todbe.com/
-/-' /___/_<_ _______________________________________________
ActivePerl mailing list
ActivePerl@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs

Re: wget in scripts

On 8 Nov 2004, at 05:44, Malcolm Mill wrote:
> I'm using wget, a perl script and a shell script to download files
> from serveral directories on a web server.
>
> The URLs I'm passing to wget are of the form:
>
> http://www.foo.br/001/ - http://www.foo.br/050/
>
> My perl script loops through the numbers 1-50, inserting each
> iteration in a string representing the URL. I call it from my shell
> script and redirect output to a text file. I then call wget from the
> same shell script passing the name of this text file to wget's
> --input-file option.
>
> I want to do this all in Perl and avoid messing around with shell
> scripts or batch files.
>
> Also, is there a simple way of constructing the numbers 001-050 for my
> URL strings?

$ perl -e 'system sprintf("wget http://www.for.br/%03d/", $_) for
(1..50);'


--
Andy Armstrong, hexten.net

Re: wget in scripts

On Sunday 07 November 2004 23:44, Malcolm Mill wrote:
> Also, is there a simple way of constructing the numbers 001-050 for
> my URL strings?

printf will do what you want.

for ($i=1; $i<51; $i++) {
printf("%03d", $i);
}

HTH

--
Bud Rogers KD5SZ