Get file list over http

Hi, I am trying to write a script, which downloads files over http. I
am using curl command to download the file, but using curl I have to
pass filename. Filename on server would be changing frequently, so is
there any way to get list of files over http? Once I get filename,
then I can pass that filename to curl and download the file.

I would also be thankful, if you can suggest any other way to achieve
this task.

If I have posted this message in wrong group, please rout me to
correct group.

I would appreciate your reply. Thanks in advance.

Thanks,
Rohit

Re: Get file list over http

Rohit wrote:
> Hi, I am trying to write a script, which downloads files over http. I
> am using curl command to download the file, but using curl I have to
> pass filename. Filename on server would be changing frequently, so is
> there any way to get list of files over http? Once I get filename,
> then I can pass that filename to curl and download the file.
>
> I would also be thankful, if you can suggest any other way to achieve
> this task.

To 'get a list of files over http' usually means to retrieve
the index of a directory. Depending on the configuration of
the server, this may be allowed/provided or not. If the browser
displays an explorer-like view when accessing a URL ending with
a directory name, it serves back a html page with a list of
links to each file.

To retrieve this index page, you can use the LWP::Simple module.
To extract links from the page, HTML::LinkExtor or HTML::TreeBuilder
can aid you. If you have successfully extracted the links (all
modules come with a number of examples) you can once again
use LWP::Simple and retrieve the files directly to disk with its
getstore() method.

HTH
-Chris

--
print +(chr(-32+ord)^("Only Waiting for Perl 6" =~
/./g)[$c++])for('%;?- V/& !+5 V* l 8$1\'ed'=~/./g);

Re: Get file list over http

On Sep 2, 3:59 pm, Christian Winter wrote:
> Rohitwrote:
> > Hi, I am trying to write a script, which downloads files over http. I
> > am usingcurlcommand to download the file, but usingcurlI have to
> > pass filename. Filename on server would be changing frequently, so is
> > there any way to get list of files over http? Once I get filename,
> > then I can pass that filename tocurland download the file.
>
> > I would also be thankful, if you can suggest any other way to achieve
> > this task.
>
> To 'get a list of files over http' usually means to retrieve
> the index of a directory. Depending on the configuration of
> the server, this may be allowed/provided or not. If the browser
> displays an explorer-like view when accessing a URL ending with
> a directory name, it serves back a html page with a list of
> links to each file.
>
> To retrieve this index page, you can use the LWP::Simple module.
> To extract links from the page, HTML::LinkExtor or HTML::TreeBuilder
> can aid you. If you have successfully extracted the links (all
> modules come with a number of examples) you can once again
> use LWP::Simple and retrieve the files directly to disk with its
> getstore() method.
>
> HTH
> -Chris
>
> --
> print +(chr(-32+ord)^("Only Waiting for Perl 6" =~
> /./g)[$c++])for('%;?- V/& !+5 V* l 8$1\'ed'=~/./g);

Thanks Chris, It works great.