Recursive delete of directory - Directory not empty error

I'm trying to remove a directory and all its contents from within a
script. I wrote a recursive function to take care of it, but when I
run it I get random "Directory not empty" error messages.

I dropped some code in to echo out the name of the file that's about
to be rmdir()ed or unoink()ed (depending if it's a file or a
directory) to see if it was choking on a particular subdirectory, but
doing that causes all the error messages to stop appearing!

It looks to me as if at some stages of the script the delete commands
are being issued too rapidly and the result is that an attempt to
delete a parent is made before its children have all been deleted.
The code I added to echo out the directory name first probably
introduces just enough of a delay for this problem to go away.

Obviously just doing something in the function to cause a delay isn't
a good solution, is there a better way of handling this problem?

I'm running PHP 5 on a Windows development machine through Apache.
Directory listings are being retrieved from a database.

Re: Recursive delete of directory - Directory not empty error

Gordon wrote:
> I'm trying to remove a directory and all its contents from within a
> script. I wrote a recursive function to take care of it, but when I
> run it I get random "Directory not empty" error messages.
>
> I dropped some code in to echo out the name of the file that's about
> to be rmdir()ed or unoink()ed (depending if it's a file or a
> directory) to see if it was choking on a particular subdirectory, but
> doing that causes all the error messages to stop appearing!
>
> It looks to me as if at some stages of the script the delete commands
> are being issued too rapidly and the result is that an attempt to
> delete a parent is made before its children have all been deleted.
> The code I added to echo out the directory name first probably
> introduces just enough of a delay for this problem to go away.
>
> Obviously just doing something in the function to cause a delay isn't
> a good solution, is there a better way of handling this problem?
>
> I'm running PHP 5 on a Windows development machine through Apache.
> Directory listings are being retrieved from a database.

Damn it. I started looking at your problem and my crystal ball went down!

Maybe someone else's crystal ball is still working and they can see you
code.

Re: Recursive delete of directory - Directory not empty error

On Apr 14, 10:04 pm, "Paul Lautman"
wrote:
> Gordon wrote:
> > I'm trying to remove a directory and all its contents from within a
> > script. I wrote a recursive function to take care of it, but when I
> > run it I get random "Directory not empty" error messages.
>
> > I dropped some code in to echo out the name of the file that's about
> > to be rmdir()ed or unoink()ed (depending if it's a file or a
> > directory) to see if it was choking on a particular subdirectory, but
> > doing that causes all the error messages to stop appearing!
>
> > It looks to me as if at some stages of the script the delete commands
> > are being issued too rapidly and the result is that an attempt to
> > delete a parent is made before its children have all been deleted.
> > The code I added to echo out the directory name first probably
> > introduces just enough of a delay for this problem to go away.
>
> > Obviously just doing something in the function to cause a delay isn't
> > a good solution, is there a better way of handling this problem?
>
> > I'm running PHP 5 on a Windows development machine through Apache.
> > Directory listings are being retrieved from a database.
>
> Damn it. I started looking at your problem and my crystal ball went down!
>
> Maybe someone else's crystal ball is still working and they can see you
> code.

I am trying to trim the code down to what's pertinant, but there are a
few classes involved. Please bear with me a moment.

Re: Recursive delete of directory - Directory not empty error

On Apr 14, 10:04 pm, "Paul Lautman"
wrote:
> Gordon wrote:
> > I'm trying to remove a directory and all its contents from within a
> > script. I wrote a recursive function to take care of it, but when I
> > run it I get random "Directory not empty" error messages.
>
> > I dropped some code in to echo out the name of the file that's about
> > to be rmdir()ed or unoink()ed (depending if it's a file or a
> > directory) to see if it was choking on a particular subdirectory, but
> > doing that causes all the error messages to stop appearing!
>
> > It looks to me as if at some stages of the script the delete commands
> > are being issued too rapidly and the result is that an attempt to
> > delete a parent is made before its children have all been deleted.
> > The code I added to echo out the directory name first probably
> > introduces just enough of a delay for this problem to go away.
>
> > Obviously just doing something in the function to cause a delay isn't
> > a good solution, is there a better way of handling this problem?
>
> > I'm running PHP 5 on a Windows development machine through Apache.
> > Directory listings are being retrieved from a database.
>
> Damn it. I started looking at your problem and my crystal ball went down!
>
> Maybe someone else's crystal ball is still working and they can see you
> code.

On Apr 14, 10:04 pm, "Paul Lautman"
wrote:
> Gordon wrote:
> > I'm trying to remove a directory and all its contents from within a
> > script. I wrote a recursive function to take care of it, but when I
> > run it I get random "Directory not empty" error messages.
>
> > I dropped some code in to echo out the name of the file that's about
> > to be rmdir()ed or unoink()ed (depending if it's a file or a
> > directory) to see if it was choking on a particular subdirectory, but
> > doing that causes all the error messages to stop appearing!
>
> > It looks to me as if at some stages of the script the delete commands
> > are being issued too rapidly and the result is that an attempt to
> > delete a parent is made before its children have all been deleted.
> > The code I added to echo out the directory name first probably
> > introduces just enough of a delay for this problem to go away.
>
> > Obviously just doing something in the function to cause a delay isn't
> > a good solution, is there a better way of handling this problem?
>
> > I'm running PHP 5 on a Windows development machine through Apache.
> > Directory listings are being retrieved from a database.
>
> Damn it. I started looking at your problem and my crystal ball went down!
>
> Maybe someone else's crystal ball is still working and they can see you
> code.

I was hoping the problem would be generic enough (trying to
recursively delete a directory) that someone else would have already
run into this problem and found a solution, but no search on either
thr group or Google turned anything up, the only thing my code is
doing that's significantly different from the other recursive delete
examples I've found is that it's using a database to fetch subdirs
instead of filesystem functions.

abstract class CmsItem
{
const TYPE_SITE = 0x1;
const TYPE_DIR = 0x2;
const TYPE_DOC = 0x4;
const TYPE_ASSET = 0x8;
const TYPE_TEMPLATE = 0x40000000;
const TYPE_TRASH = -0x80000000; // Negative because postgress
doesn't have an unsigned int type

protected $database = NULL;
protected $user = NULL;

public $itemProps = array (
'itm_id' => 0,
'itm_type' => 0,
'itm_parent' => 0,
'tpl_id' => 0,
'itm_title' => '',
'itm_path' => '',
'itm_summary' => '',
'itm_keywords' => '',
'itm_notes' => ''
);

abstract protected function getItem ($id);
public static function factory ($className, $id, Database $dbHandle,
CmsUser $user = NULL)
{
// Create new item
$newItem = new $className ($id, $dbHandle, $user);
// Fail if an ID was specified but no item state was returned
if (($id) && (!($newItem -> itemProps ['itm_id'])))
{
$newItem = NULL;
}
return ($newItem);
}
protected function pathToUrl ($path = NULL, $type = NULL)
{
if ($path === NULL)
{
$path = $this -> itemProps ['itm_fullpath'];
}
if ($type === NULL)
{
$type = $this -> itemProps ['itm_type'];
}
$url = 'http://';
if ($type == self::TYPE_SITE)
{
// For sites the URL is the same as the fullpath
$url .= $path . '/';
}
else
{
// Insert the prefix into the full path value and use as the URL
$url
.= substr ($path, 0, strpos ($path, '/'))
. CFG_CONTENT_URL_PREFIX
. substr ($path, strpos ($path, '/'));
}
// Folders should get a trailing slash
if ($type == self::TYPE_DIR)
{
$url .= '/';
}
return ($url);
}
protected function spawn ($id, $type)
{
switch ($type)
{
case self::TYPE_SITE : $className = 'CmsSite'; break;
case self::TYPE_DIR : $className = 'CmsDir'; break;
case self::TYPE_DOC : $className = 'CmsDoc'; break;
case self::TYPE_ASSET : $className = 'CmsAsset'; break;
case self::TYPE_TEMPLATE : $className = 'CmsTemplate'; break;
//case self::TYPE_TRASH : $className = 'CmsTrash'; break;
default :
break;
}
//echo ('id: '.$id.' type: '.$type.' ClassName: '.$className."

\n");
if ($className)
{
return (self::factory ($className, $id, $this -> database, $this ->
user));
}
}
public function getChildren ($showSys = NULL, $showPub = NULL,
$showDel = 'FALSE')
{
// prepare query and arguments
$id = $this -> itemProps ['itm_id'];
$params [] = $id;
$query = 'SELECT *
FROM cms_v_items
WHERE itm_parent = ? ';
// Handle the case where the showSys string has been set
if ($showSys)
{
$params [] = $showSys;
$query .= 'AND itm_system = ? ';
}
// Handle the case where the showPub string has been set
if ($showPub)
{
$params [] = $showPub;
$query .= 'AND
(
itm_publish = ?
OR itm_type != ' . intval (self::TYPE_DOC) . '
)';
}
$query .= 'ORDER BY itm_system DESC, itm_type, itm_sort, itm_path;';
$preparedQuery = $this -> database -> prepare ($query);
// Get child items
//if ($preparedQuery -> execute (array ($id)))
if ($row = $preparedQuery -> ask ($params))
{
//while ($thisRow = $preparedQuery -> fetch (PDO::FETCH_ASSOC))
foreach ($row as $thisRow)
{
//print_r ($thisRow);
$thisRow ['itm_date_create'] = strtotime ($thisRow
['itm_date_create']);
$thisRow ['itm_date_modify'] = strtotime ($thisRow
['itm_date_modify']);
$thisRow ['itm_fullpath'] = $this -> itemProps ['itm_fullpath'] .
'/' . $thisRow ['itm_path'];
$thisRow ['itm_url'] = $this -> pathToUrl ($thisRow
['itm_fullpath'], $thisRow ['itm_type']);
$result [] = $thisRow;
}
return ($result);
}
}
public function getChildObjects ($showSys = NULL, $showPub = NULL,
$showDel = 'FALSE')
{
if ($childList = $this -> getChildren ($showSys, $showPub))
{
$objectArr = array ();
foreach ($childList as $thisChild)
{
$objectArr [] = $this -> spawn ($thisChild ['itm_id'], $thisChild
['itm_type']);
}
return ($objectArr);
}
}
public function deleteItem ()
{
// Following line added for debugging but it caused the problem to
disappear!
echo ($this -> itemProps ['itm_fullpath'].'
');
$query = 'DELETE FROM cms_items
WHERE itm_id = ?;';
$preparedQuery = $this -> database -> prepare ($query);
// Delete item
return (($preparedQuery -> tell (array ($this -> itemProps
['itm_id']))) == 1);
}
}


class CmsDir extends CmsItem
{
public function deleteItem ($deleteFile = true)
{
$error = false;
$this -> database -> beginTransaction ();
// Deal with any child nodes this directory might hold
if ($children = $this -> getChildObjects ())
{

foreach ($children as $thisChild)
{
if (!$thisChild -> deleteItem ($deleteFile))
{
$error = true;
break;
}
}
}
// If child nodes were successfully handled then remove the
directory itself
if (!$error)
{
if (parent::deleteItem ())
{
// Deal with the actual directory
$this -> uncache ();
if ((!deleteFile)
|| (!file_exists (CFG_CONTENT_ROOT . '/' . $this -> itemProps
['itm_fullpath']))
|| (rmdir (CFG_CONTENT_ROOT . '/' . $this -> itemProps
['itm_fullpath'])))
{
clearstatcache ();
$this -> database -> commit ();
return (true);
}
else
{
// Something went wrong removing the directory
$this -> database -> rollback ();
}
}
else
{
// Error occured deleting the database record
$this -> database -> rollback ();
}
}
else
{
// Error occured deleting children
$this -> database -> rollback ();
}
}
public function uncache ()
{
$path = CFG_CONTENT_ROOT . '/' . $this -> itemProps
['itm_fullpath'] . '/' . CFG_DIR_INDEXFILE;
$success = ((!file_exists ($path))
|| (unlink ($path)));
clearstatcache ();
return ($success);
}
}

Re: Recursive delete of directory - Directory not empty error

Gordon escribió:
> I'm trying to remove a directory and all its contents from within a
> script. I wrote a recursive function to take care of it, but when I
> run it I get random "Directory not empty" error messages.

In the manual page for unlink() there's a user comment with sample code
for recursive deletion. I haven't tried it but who knows:

http://php.net/unlink




--
-- http://alvaro.es - Álvaro G. Vicario - Burgos, Spain
-- Mi sitio sobre programación web: http://bits.demogracia.com
-- Mi web de humor al baño María: http://www.demogracia.com
--

Re: Recursive delete of directory - Directory not empty error

Álvaro G. Vicario wrote:
> Gordon escribió:
>> I'm trying to remove a directory and all its contents from within a
>> script. I wrote a recursive function to take care of it, but when I
>> run it I get random "Directory not empty" error messages.
>
> In the manual page for unlink() there's a user comment with sample code
> for recursive deletion. I haven't tried it but who knows:
>
> http://php.net/unlink
>
>
>
>
Normally thats because there are hidden or wrong permission files in it.

I suspect you need more subtle code, and if in Linux etc, some form of
attention to permissions if these are the problem.

Re: Recursive delete of directory - Directory not empty error

On Apr 15, 2:28 pm, The Natural Philosopher wrote:
> =C1lvaro G. Vicario wrote:
> > Gordon escribi=F3:
> >> I'm trying to remove a directory and all its contents from within a
> >> script. I wrote a recursive function to take care of it, but when I
> >> run it I get random "Directory not empty" error messages.
>
> > In the manual page for unlink() there's a user comment with sample code
> > for recursive deletion. I haven't tried it but who knows:
>
> >http://php.net/unlink
>
> Normally thats because there are hidden or wrong permission files in it.
>
> I suspect you need more subtle code, and if in Linux etc, some form of
> attention to permissions if these are the problem.

I don't think permissions are the problem, because the development
machine is Windows, and the error message being given is Directory Not
Empty. No Permission Denied errors are popping up in the output. I
also mentioned that the problem went away when I introduced a line of
code for debugging purposes into CmsItem::deleteItem(), which echos
the path being deleted to the output. My suspicion is that the calls
to rmdir are happening more rapidly than the filesystem can cope with
them. The debug code introduced just enough of a delay for the code
to work properly but without it attempts to delete a parent directory
can occur before the deletion of its children has completed, causing
the not empty error.

Re: Recursive delete of directory - Directory not empty error

Gordon wrote:
> On Apr 15, 2:28 pm, The Natural Philosopher wrote:
>> Álvaro G. Vicario wrote:
>>> Gordon escribió:
>>>> I'm trying to remove a directory and all its contents from within a
>>>> script. I wrote a recursive function to take care of it, but when I
>>>> run it I get random "Directory not empty" error messages.
>>> In the manual page for unlink() there's a user comment with sample code
>>> for recursive deletion. I haven't tried it but who knows:
>>> http://php.net/unlink
>> Normally thats because there are hidden or wrong permission files in it.
>>
>> I suspect you need more subtle code, and if in Linux etc, some form of
>> attention to permissions if these are the problem.
>
> I don't think permissions are the problem, because the development
> machine is Windows, and the error message being given is Directory Not
> Empty. No Permission Denied errors are popping up in the output. I
> also mentioned that the problem went away when I introduced a line of
> code for debugging purposes into CmsItem::deleteItem(), which echos
> the path being deleted to the output. My suspicion is that the calls
> to rmdir are happening more rapidly than the filesystem can cope with
> them. The debug code introduced just enough of a delay for the code
> to work properly but without it attempts to delete a parent directory
> can occur before the deletion of its children has completed, causing
> the not empty error.

Ah. Caching.

I've had a similar problem with PHP creating a file that didn't actually
exist on disk till PHP exited.


So your are suggesting that PHP doesn't actually delete the files until
after the call to unlink the directory is received?

I would NOT be surprised. PHP'S file handling seems to be a bit flaky in
this area.


Windows should be capable of processing the calls as fast as they can be
sent: IT *should* simply block if the directory is still busy.

My guess - and its only a guess - is that PHP itself is caching the file
deletes. But not the directory deletes.

Or spitting them out in random order.

Maybe its possible to force a buffer flush in PHP.

Re: Recursive delete of directory - Directory not empty error

On Apr 15, 3:27 pm, The Natural Philosopher wrote:
> Gordon wrote:
> > On Apr 15, 2:28 pm, The Natural Philosopher wrote:
> >> =C1lvaro G. Vicario wrote:
> >>> Gordon escribi=F3:
> >>>> I'm trying to remove a directory and all its contents from within a
> >>>> script. I wrote a recursive function to take care of it, but when I
> >>>> run it I get random "Directory not empty" error messages.
> >>> In the manual page for unlink() there's a user comment with sample cod=
e
> >>> for recursive deletion. I haven't tried it but who knows:
> >>>http://php.net/unlink
> >> Normally thats because there are hidden or wrong permission files in it=
..
>
> >> I suspect you need more subtle code, and if in Linux etc, some form of
> >> attention to permissions if these are the problem.
>
> > I don't think permissions are the problem, because the development
> > machine is Windows, and the error message being given is Directory Not
> > Empty. No Permission Denied errors are popping up in the output. I
> > also mentioned that the problem went away when I introduced a line of
> > code for debugging purposes into CmsItem::deleteItem(), which echos
> > the path being deleted to the output. My suspicion is that the calls
> > to rmdir are happening more rapidly than the filesystem can cope with
> > them. The debug code introduced just enough of a delay for the code
> > to work properly but without it attempts to delete a parent directory
> > can occur before the deletion of its children has completed, causing
> > the not empty error.
>
> Ah. Caching.
>
> I've had a similar problem with PHP creating a file that didn't actually
> exist on disk till PHP exited.
>
> So your are suggesting that PHP doesn't actually delete the files until
> after the call to unlink the directory is received?
>
> I would NOT be surprised. PHP'S file handling seems to be a bit flaky in
> this area.
>
> Windows should be capable of processing the calls as fast as they can be
> sent: IT *should* simply block if the directory is still busy.
>
> My guess - and its only a guess - is that PHP itself is caching the file
> deletes. But not the directory deletes.
>
> Or spitting them out in random order.
>
> Maybe its possible to force a buffer flush in PHP.

Yes, I'm caching database-stored content to disk as HTML in a
directory structure so only the first view of a page causes the actual
script to run, do the DB lookup, build the page and all the other time-
consuming jobs involved in presenting the content to the user (My 404
page is actually a PHP script that takes care of all this).

Like you said, I think that PHP is simply dispatching filesystem
commands to a queue of some sort rather than waiting for each one to
complete before moving on to the next, resulting in directories not
being empty when an unlink attempt is made on them, even though all
their contents has already also been unlinked.

I had thought of the possibility that there was a buffer problem
somewhere, and I tried adding clearstatcache calls in strategic places
in the code but that doesn't seem to have helped.