Yahoo Gmail Google Facebook Delicious Twitter Reddit Stumpleupon Myspace Digg

Search queries

docmd.close 2585, WWWXXXAPC, WWWXXXDOCO, nu vot, dhcpd lease file "binding state", WWWXXXDOCO, how to setup procmail to process html2text, how to setup procmail html2text, WWWXXXAPC., XXXCNZZZ



#1: file crawler

Posted on 2008-04-24 02:03:04 by Ali

I have just started learning SQL. I would like to write a program that
crawls the hard drive and collects file names (with full path), size
of files and directories, date of last modification, etc. (I know
Google Desktop Search offers more.)

I am already stuck with the path, how should i handle that? It is a
good candidate key, however there is repetition in it when storing
subdirectories, and it is not easy to update the database if the
directory is renamed.

After all, if we think of a single machine, i could make this program
and would work just fine. But i would like to later extend this
program to crawl FTP servers (such as pro-search ftp crawler), and i
am pretty sure the database would become the bottleneck very soon.

I would appreciate tips on how this databse should be designed.

Report this message