Bookmarks

Yahoo Gmail Google Facebook Delicious Twitter Reddit Stumpleupon Myspace Digg

Search queries

sqldatasource dal, wwwxxxenden, convert raid5 to raid 10 mdadm, apache force chunked, nrao wwwxxx, xxxxxdup, procmail change subject header, wwwXxx not20, Wwwxxx.doks sas, linux raid resync after reboot

Links

XODOX
Impressum

#1: file crawler

Posted on 2008-04-24 02:03:04 by Ali

I have just started learning SQL. I would like to write a program that
crawls the hard drive and collects file names (with full path), size
of files and directories, date of last modification, etc. (I know
Google Desktop Search offers more.)

I am already stuck with the path, how should i handle that? It is a
good candidate key, however there is repetition in it when storing
subdirectories, and it is not easy to update the database if the
directory is renamed.

After all, if we think of a single machine, i could make this program
and would work just fine. But i would like to later extend this
program to crawl FTP servers (such as pro-search ftp crawler), and i
am pretty sure the database would become the bottleneck very soon.

I would appreciate tips on how this databse should be designed.

Report this message