Read directory; store filenames found in mySQL table?

Ave,

This is what I'm trying to do; I want to read a directory (eg: W:\Test
\) and take all the filenames found in the directory (eg: 1.vox,
2.wav, 3.txt) and store them in a simple mySQL table.

Can I do this?

---
Rahul Sitaram Johari
Founder, Internet Architects Group, Inc.

[Email] sleepwalker@rahulsjohari.com
[Web] http://www.rahulsjohari.com





--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Read directory; store filenames found in mySQL table?

--=-bU7/HRxslEjPK7g4M+Zw
Content-Type: text/plain
Content-Transfer-Encoding: 7bit

On Wed, 2010-01-13 at 09:25 -0500, Rahul S. Johari wrote:

> Ave,
>
> This is what I'm trying to do; I want to read a directory (eg: W:\Test
> \) and take all the filenames found in the directory (eg: 1.vox,
> 2.wav, 3.txt) and store them in a simple mySQL table.
>
> Can I do this?
>
> ---
> Rahul Sitaram Johari
> Founder, Internet Architects Group, Inc.
>
> [Email] sleepwalker@rahulsjohari.com
> [Web] http://www.rahulsjohari.com
>
>
>
>
>


You'll probably want to look at the readdir() function. The manual page
also has dozens of different example scripts that would be easy to tweak
for your purpose.

http://php.net/manual/en/function.readdir.php


Thanks,
Ash
http://www.ashleysheridan.co.uk



--=-bU7/HRxslEjPK7g4M+Zw--

Re: Read directory; store filenames found in mySQL table?

Ashley Sheridan wrote:
> On Wed, 2010-01-13 at 09:25 -0500, Rahul S. Johari wrote:
>
>
>> Ave,
>>
>> This is what I'm trying to do; I want to read a directory (eg: W:\Test
>> \) and take all the filenames found in the directory (eg: 1.vox,
>> 2.wav, 3.txt) and store them in a simple mySQL table.
>>
>> Can I do this?
>>
>> ---
>> Rahul Sitaram Johari
>> Founder, Internet Architects Group, Inc.
>>
>> [Email] sleepwalker@rahulsjohari.com
>> [Web] http://www.rahulsjohari.com
>>
>>
>>
>>
>>
>>
>
>
> You'll probably want to look at the readdir() function. The manual page
> also has dozens of different example scripts that would be easy to tweak
> for your purpose.
>
> http://php.net/manual/en/function.readdir.php
>
>
> Thanks,
> Ash
> http://www.ashleysheridan.co.uk
>
>
>
>
I use the glob function in my little homemade "web cam" page, which can
really swell up in memory when used against a large amount of files (in
my case around 30k files).
=====
$imgdir = 'img/south*';
$files = glob( $imgdir );

// Sort files by modified time, latest to earliest
// Use SORT_ASC in place of SORT_DESC for earliest to latest
array_multisort( array_map( 'filemtime', $files ), SORT_NUMERIC,
SORT_DESC, $files );
=====
http://www.php.net/manual/en/function.glob.php
DISCLAIMER: I found this code on a how-to somewhere out there and
modified it to fit my need. Quite possibly there are much better means
to this end.

Ken Sande/KC8QNI


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Read directory; store filenames found in mySQL table?

On Jan 13, 2010, at 9:56 AM, Warren Windvogel wrote:

> On 2010/01/13 04:25 PM, Rahul S. Johari wrote:
>> Ave,
>>
>> This is what I'm trying to do; I want to read a directory (eg: W:
>> \Test\) and take all the filenames found in the directory (eg:
>> 1.vox, 2.wav, 3.txt) and store them in a simple mySQL table.
>>
>
> Sorry. Forgot to include this.
>
> function dirList ($directory)
> {
>
> // create an array to hold directory list
> $results = array();
>
> // create a handler for the directory
> $handler = opendir($directory);
>
> // keep going until all files in directory have been read
> while ($file = readdir($handler)) {
>
> // if $file isn't this directory or its parent,
> // add it to the results array
> if ($file != '.' && $file != '..')
> $results[] = $file;
> }
>
> // tidy up: close the handler
> closedir($handler);
>
> // done!
> return $results;
>
> }
>
> If you're dealing with 1 directory you can use it to read all files
> in it to an array.
>
> Kind regards
> Warren
>


This is an interesting approach. Following is what I came up with to
scan a directory and store the filenames into an array ... very
similar to your example:

$listDir = array();
$dir = "../mounts/wd/IDT/IDT/";
if($handler = opendir($dir)) {
while (($sub = readdir($handler)) !== FALSE) {
if ($sub != "." && $sub != ".." && $sub !=
"Thumb.db") {
if(is_file($dir."/".$sub)) {
$listDir[] = $sub;
}elseif(is_dir($dir."/".$sub)){
$listDir[$sub] = $this-
>ReadFolderDirectory($dir."/".$sub);
}
}
}
closedir($handler);
}

and this is what I'm trying to implement in order to store the array
into a mysql table ..

$db = mysql_connect("localhost","usr","pwd");
mysql_select_db("db",$db);
$colors=serialize($listDir); //takes the data from a post
operation...
$sql="INSERT INTO recordings (ID, RECORDING, ADDED)
VALUES('','$colors','')";
$result = mysql_query($sql) or die (mysql_error());

I'm not sure if this the best or fastest approach ... but it's running
in the background as I write this (I should have tested on a smaller
folder). The folder I'm scanning has literally over 80,000 files ...
so it's taking LOOONG!!

---
Rahul Sitaram Johari
Founder, Internet Architects Group, Inc.

[Email] sleepwalker@rahulsjohari.com
[Web] http://www.rahulsjohari.com





--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Read directory; store filenames found in mySQL table?

On Jan 13, 2010, at 10:07 AM, Kenneth Sande wrote:

>
> Ashley Sheridan wrote:
>> On Wed, 2010-01-13 at 09:25 -0500, Rahul S. Johari wrote:
>>
>>
>>> Ave,
>>>
>>> This is what I'm trying to do; I want to read a directory (eg: W:
>>> \Test \) and take all the filenames found in the directory (eg:
>>> 1.vox, 2.wav, 3.txt) and store them in a simple mySQL table.
>>>
>>> Can I do this?
>>>
>>> ---
>>> Rahul Sitaram Johari
>>> Founder, Internet Architects Group, Inc.
>>>
>>> [Email] sleepwalker@rahulsjohari.com
>>> [Web] http://www.rahulsjohari.com
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> You'll probably want to look at the readdir() function. The manual
>> page
>> also has dozens of different example scripts that would be easy to
>> tweak
>> for your purpose.
>>
>> http://php.net/manual/en/function.readdir.php
>>
>>
>> Thanks,
>> Ash
>> http://www.ashleysheridan.co.uk
>>
>>
>>
>>
> I use the glob function in my little homemade "web cam" page, which
> can really swell up in memory when used against a large amount of
> files (in my case around 30k files).
> =====
> $imgdir = 'img/south*';
> $files = glob( $imgdir );
>
> // Sort files by modified time, latest to earliest
> // Use SORT_ASC in place of SORT_DESC for earliest to latest
> array_multisort( array_map( 'filemtime', $files ), SORT_NUMERIC,
> SORT_DESC, $files );
> =====
> http://www.php.net/manual/en/function.glob.php
> DISCLAIMER: I found this code on a how-to somewhere out there and
> modified it to fit my need. Quite possibly there are much better
> means to this end.
>
> Ken Sande/KC8QNI
>
>


Considering that I have over 80K files in the folder, would this be a
faster/efficient then the readdir() method? Or should I stick to what
I'm doing (other email)?

---
Rahul Sitaram Johari
Founder, Internet Architects Group, Inc.

[Email] sleepwalker@rahulsjohari.com
[Web] http://www.rahulsjohari.com





--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Read directory; store filenames found in mySQL table?

On Jan 13, 2010, at 9:50 AM, Warren Windvogel wrote:

> On 2010/01/13 04:25 PM, Rahul S. Johari wrote:
>> Ave,
>>
>> This is what I'm trying to do; I want to read a directory (eg: W:
>> \Test\) and take all the filenames found in the directory (eg:
>> 1.vox, 2.wav, 3.txt) and store them in a simple mySQL table.
>>
>> Can I do this?
> I tried to very quickly convert something I've done. It may need
> some work. Will work in linux env.
>
> $origin = "Path"
>
> #load file listing into an array
> $shell = shell_exec("du $origin");
> $array = array_reverse(explode("\n",$shell));
> $contIdArr = array();
>
> $newArr = array();
> foreach($array as $elem){
> $newDir = "";
> $pathArray = explode("/", $elem);
> $nodeDepth = count($pathArray);
> for($count=1; $count<$nodeDepth; $count++){
> $newDir = $newDir.$pathArray[$count].'/';
> }
> $newArr[] = '/'.$newDir;
> }
> sort($newArr);
>
>
> foreach($newArr as $dir){
> $pathArray = explode("/", $dir);
>
> $fileListArr = dirList($dir);
>
> foreach($fileListArr as $file){
> //Insert file($file) and current dir/path($dir) into db
> }
> }
>
> Kind regards
> Warren
>


Warren,

I tried using your code and it definitely is very efficient & fast;
however I'm running into a small problem and I'm not sure how to
correct it. I'm getting the array with filenames from the folder I'm
searching in PLUS all the root folders of the machine as well.

This is the code I'm using (note that I'm just echoing the array right
now; I'll move to inserting data into mySQL after):

function dirList ($directory) {
$results = array();
$handler = opendir($directory);
while ($file = readdir($handler)) {
if ($file != '.' && $file != '..')
$results[] = $file;
}
closedir($handler);
return $results;
}

$origin = "/Library/WebServer/Documents/folder1/folder2/images/";

#load file listing into an array
$shell = shell_exec("du $origin");
$array = array_reverse(explode("\n",$shell));
$contIdArr = array();

$newArr = array();
foreach($array as $elem){
$newDir = "";
$pathArray = explode("/", $elem);
$nodeDepth = count($pathArray);
for($count=1; $count<$nodeDepth; $count++){
$newDir = $newDir.$pathArray[$count].'/';
}
$newArr[] = '/'.$newDir;
}
sort($newArr);

foreach($newArr as $dir){
$pathArray = explode("/", $dir);
$fileListArr = dirList($dir);

foreach($fileListArr as $file){
echo $file."
";
//Insert file($file) and current dir/path($dir) into db
}
}


As an output ... i get a list of all the files in the "images" folder
preceeded by the all the list of root folders on my machine!! How do I
eliminate the list of root folders?

---
Rahul Sitaram Johari
Founder, Internet Architects Group, Inc.

[Email] sleepwalker@rahulsjohari.com
[Web] http://www.rahulsjohari.com





--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Read directory; store filenames found in mySQL table?

On Jan 13, 2010, at 10:40 AM, Rahul S. Johari wrote:

>
> On Jan 13, 2010, at 9:50 AM, Warren Windvogel wrote:
>
>> On 2010/01/13 04:25 PM, Rahul S. Johari wrote:
>>> Ave,
>>>
>>> This is what I'm trying to do; I want to read a directory (eg: W:
>>> \Test\) and take all the filenames found in the directory (eg:
>>> 1.vox, 2.wav, 3.txt) and store them in a simple mySQL table.
>>>
>>> Can I do this?
>> I tried to very quickly convert something I've done. It may need
>> some work. Will work in linux env.
>>
>> $origin = "Path"
>>
>> #load file listing into an array
>> $shell = shell_exec("du $origin");
>> $array = array_reverse(explode("\n",$shell));
>> $contIdArr = array();
>>
>> $newArr = array();
>> foreach($array as $elem){
>> $newDir = "";
>> $pathArray = explode("/", $elem);
>> $nodeDepth = count($pathArray);
>> for($count=1; $count<$nodeDepth; $count++){
>> $newDir = $newDir.$pathArray[$count].'/';
>> }
>> $newArr[] = '/'.$newDir;
>> }
>> sort($newArr);
>>
>>
>> foreach($newArr as $dir){
>> $pathArray = explode("/", $dir);
>>
>> $fileListArr = dirList($dir);
>>
>> foreach($fileListArr as $file){
>> //Insert file($file) and current dir/path($dir) into db
>> }
>> }
>>
>> Kind regards
>> Warren
>>
>
>
> Warren,
>
> I tried using your code and it definitely is very efficient & fast;
> however I'm running into a small problem and I'm not sure how to
> correct it. I'm getting the array with filenames from the folder I'm
> searching in PLUS all the root folders of the machine as well.
>
> This is the code I'm using (note that I'm just echoing the array
> right now; I'll move to inserting data into mySQL after):
>
> function dirList ($directory) {
> $results = array();
> $handler = opendir($directory);
> while ($file = readdir($handler)) {
> if ($file != '.' && $file != '..')
> $results[] = $file;
> }
> closedir($handler);
> return $results;
> }
>
> $origin = "/Library/WebServer/Documents/folder1/folder2/images/";
>
> #load file listing into an array
> $shell = shell_exec("du $origin");
> $array = array_reverse(explode("\n",$shell));
> $contIdArr = array();
>
> $newArr = array();
> foreach($array as $elem){
> $newDir = "";
> $pathArray = explode("/", $elem);
> $nodeDepth = count($pathArray);
> for($count=1; $count<$nodeDepth; $count++){
> $newDir = $newDir.$pathArray[$count].'/';
> }
> $newArr[] = '/'.$newDir;
> }
> sort($newArr);
>
> foreach($newArr as $dir){
> $pathArray = explode("/", $dir);
> $fileListArr = dirList($dir);
>
> foreach($fileListArr as $file){
> echo $file."
";
> //Insert file($file) and current dir/path($dir) into db
> }
> }
>
>
> As an output ... i get a list of all the files in the "images"
> folder preceeded by the all the list of root folders on my machine!!
> How do I eliminate the list of root folders?


Nevermind, I was looking at the wrong output. I got it!! I've got all
my filenames in my $fileListArr[] array!!
Now I just to get the values in a mySQL table.


---
Rahul Sitaram Johari
Founder, Internet Architects Group, Inc.

[Email] sleepwalker@rahulsjohari.com
[Web] http://www.rahulsjohari.com





--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Read directory; store filenames found in mySQL table? :: SOLVED!!

Thanks All (Especially Warren); Final Code:

function dirList ($directory) {
$results = array();
$handler = opendir($directory);
while ($file = readdir($handler)) {
if ($file != '.' && $file != '..')
$results[] = $file;
}
closedir($handler);
return $results;
}

$origin = "pathto/images/";

#load file listing into an array
$shell = shell_exec("du $origin");
$array = array_reverse(explode("\n",$shell));
$contIdArr = array();

$newArr = array();
foreach($array as $elem){
$newDir = "";
$pathArray = explode("/", $elem);
$nodeDepth = count($pathArray);
for($count=1; $count<$nodeDepth; $count++){
$newDir = $newDir.$pathArray[$count].'/';
}
$newArr[] = '/'.$newDir;
}
sort($newArr);

foreach($newArr as $dir){
$pathArray = explode("/", $dir);
$fileListArr = dirList($dir);
}

$db = mysql_connect("localhost","usr","pwd");
mysql_select_db("db",$db);

foreach($fileListArr AS $value) {
$sql="INSERT INTO r2 (ID, RECORDING) VALUES('','$value')";
$result = mysql_query($sql) or die (mysql_error());
}


---
Rahul Sitaram Johari
Founder, Internet Architects Group, Inc.

[Email] sleepwalker@rahulsjohari.com
[Web] http://www.rahulsjohari.com





--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Read directory; store filenames found in mySQL table?

On Wed, Jan 13, 2010 at 10:50 AM, Rahul S. Johari
wrote:
>
> On Jan 13, 2010, at 10:40 AM, Rahul S. Johari wrote:
>
>>
>> On Jan 13, 2010, at 9:50 AM, Warren Windvogel wrote:
>>
>>> On 2010/01/13 04:25 PM, Rahul S. Johari wrote:
>>>>
>>>> Ave,
>>>>
>>>> This is what I'm trying to do; I want to read a directory (eg: W:\Test=
\)
>>>> and take all the filenames found in the directory (eg: 1.vox, 2.wav, 3=
..txt)
>>>> and store them in =A0a simple mySQL table.
>>>>
>>>> Can I do this?
>>>
>>> I tried to very quickly convert something I've done. It may need some
>>> work. Will work in =A0linux env.
>>>
>>> $origin =3D "Path"
>>>
>>> #load file listing into an array
>>> $shell =3D shell_exec("du $origin");
>>> $array  = array_reverse(explode("\n",$shell));
>>> $contIdArr =3D array();
>>>
>>> $newArr =3D array();
>>> foreach($array as $elem){
>>> =A0$newDir =3D "";
>>> =A0$pathArray =3D explode("/", $elem);
>>> =A0$nodeDepth =3D count($pathArray);
>>> =A0for($count=3D1; $count<$nodeDepth; $count++){
>>> =A0 =A0 =A0$newDir =3D $newDir.$pathArray[$count].'/';
>>> =A0}
>>> =A0$newArr[] =3D '/'.$newDir;
>>> }
>>> sort($newArr);
>>>
>>>
>>> foreach($newArr as $dir){
>>> =A0$pathArray =3D explode("/", $dir);
>>>
>>> =A0$fileListArr =3D dirList($dir);
>>>
>>> =A0foreach($fileListArr as $file){
>>> =A0 =A0 =A0//Insert file($file) and current dir/path($dir) into db
>>> =A0}
>>> }
>>>
>>> Kind regards
>>> Warren
>>>
>>
>>
>> Warren,
>>
>> I tried using your code and it definitely is very efficient & fast;
>> however I'm running into a small problem and I'm not sure how to correct=
it.
>> I'm getting the array with filenames from the folder I'm searching in PL=
US
>> all the root folders of the machine as well.
>>
>> This is the code I'm using (note that I'm just echoing the array right
>> now; I'll move to inserting data into mySQL after):
>>
>> function dirList ($directory) {
>> =A0$results =3D array();
>> =A0$handler =3D opendir($directory);
>> =A0while ($file =3D readdir($handler)) {
>> =A0 =A0 =A0if ($file !=3D '.' && $file !=3D '..')
>> =A0 =A0 =A0 =A0 =A0$results[] =3D $file;
>> =A0}
>> =A0closedir($handler);
>> =A0return $results;
>> }
>>
>> $origin =3D "/Library/WebServer/Documents/folder1/folder2/images/";
>>
>> #load file listing into an array
>> $shell =3D shell_exec("du $origin");
>> $array  = array_reverse(explode("\n",$shell));
>> $contIdArr =3D array();
>>
>> $newArr =3D array();
>> foreach($array as $elem){
>> =A0$newDir =3D "";
>> =A0$pathArray =3D explode("/", $elem);
>> =A0$nodeDepth =3D count($pathArray);
>> =A0for($count=3D1; $count<$nodeDepth; $count++){
>> =A0 =A0 =A0$newDir =3D $newDir.$pathArray[$count].'/';
>> =A0}
>> =A0$newArr[] =3D '/'.$newDir;
>> }
>> sort($newArr);
>>
>> foreach($newArr as $dir){
>> =A0$pathArray =3D explode("/", $dir);
>> =A0$fileListArr =3D dirList($dir);
>>
>> =A0foreach($fileListArr as $file){
>> =A0 =A0 =A0 =A0echo $file."
";
>> =A0 =A0 =A0//Insert file($file) and current dir/path($dir) into db
>> =A0}
>> }
>>
>>
>> As an output ... i get a list of all the files in the "images" folder
>> preceeded by the all the list of root folders on my machine!! How do I
>> eliminate the list of root folders?
>
>
> Nevermind, I was looking at the wrong output. I got it!! I've got all my
> filenames in my $fileListArr[] array!!
> Now I just to get the values in a mySQL table.
>
>
> ---
> Rahul Sitaram Johari
> Founder, Internet Architects Group, Inc.
>
> [Email] sleepwalker@rahulsjohari.com
> [Web] =A0 http://www.rahulsjohari.com
>
>
>
>
>
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>


consider stacking the insert statements in sql to allow for a certain
number of inserts in one connect.

insert into my_table (field1, field2...fieldn)
values('field1','field2'...fieldn),('field1','field2'...fiel dn),('field1','=
field2'...fieldn),('field1','field2'...fieldn)...

keep to something like 100 to avoid buffer overflows and it should
make the inserts much faster

--=20

Bastien

Cat, the other other white meat

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Read directory; store filenames found in mySQL table?

Kenneth Sande wrote:
> I use the glob function in my little homemade "web cam" page, which can
> really swell up in memory when used against a large amount of files (in
> my case around 30k files).

+1 for glob()


--
Thanks!
-Shawn
http://www.spidean.com

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php