Advice on maintaining public and private files

I have a site I'm working on with some data that I want to be readable =
by anyone, but some files that I want to keep hidden from outside users. =
Here is an example of my file structure.

/products/data1/item_1/data.txt
/products/data2/item_2/data.txt

I would like everything in data1 to be available by anyone who visits =
the site, but I want to keep items in the data2 folder to only be =
accessible through certain web page which I hope to eventually require =
logins. Some of these items I'd like to not only display but also allow =
people to download.

My main concern is that I don't want people to be able to guess the =
names of the files and then be able to access the information on them. =
Every 'item' has an entry in a MySQL database which holds some =
information. I was thinking I could have randomly generated folder names =
to take the place of the things like 'item_2' such as

/products/data2/kl23j42i/data.txt

and then link the folder name through a database entry. But I'm not sure =
if there are more elegant or easier ways to deal with this. Plus someone =
could still just try randomly querying the site until they get a match. =
I'd first like to just create a web page where you can go to access the =
hidden files but would later like to add more control for other users =
using logins and passwords.

Most of my files are just text files and images. Any suggestions?

Thanks in advance!

Michael=

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Advice on maintaining public and private files

On Fri, Feb 19, 2010 at 1:19 PM, Michael Stroh wrote:
> I have a site I'm working on with some data that I want to be readable by=
anyone, but some files that I want to keep hidden from outside users. Here=
is an example of my file structure.
>
> /products/data1/item_1/data.txt
> /products/data2/item_2/data.txt
>
> I would like everything in data1 to be available by anyone who visits the=
site, but I want to keep items in the data2 folder to only be accessible t=
hrough certain web page which I hope to eventually require logins. Some of =
these items I'd like to not only display but also allow people to download.
>
> My main concern is that I don't want people to be able to guess the names=
of the files and then be able to access the information on them. Every 'it=
em' has an entry in a MySQL database which holds some information. I was th=
inking I could have randomly generated folder names to take the place of th=
e things like 'item_2' such as
>
> /products/data2/kl23j42i/data.txt
>
> and then link the folder name through a database entry. But I'm not sure =
if there are more elegant or easier ways to deal with this. Plus someone co=
uld still just try randomly querying the site until they get a match. I'd f=
irst like to just create a web page where you can go to access the hidden f=
iles but would later like to add more control for other users using logins =
and passwords.
>
> Most of my files are just text files and images. Any suggestions?
>
> Thanks in advance!
>
> Michael
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>


Place all those files above the web root, the use php to read in the
data from the files when display that data to the user.
--=20

Bastien

Cat, the other other white meat

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Advice on maintaining public and private files

the "proper way" i know of is not the easiest to implement..;

1) create a php script that accepts enough parameters to get at your data.
eg: /products/view.php?dataNr=3D1&itemNr=3D1
2) let that script compare the current user (visitor who's logged in)
to authentication data that tells which it if the user can access the
data requested. if it fails, you can route the user to a std page or
to a custom page (store in auth-data under "onFail")
3) use apache's RewriteRule in /products/.htaccess to point virtual
urls to the view script; /products/data1/item_1/data.txt =3D
/products/view.php?dataNr=3D1&itemNr=3D1&file=3Ddata.txt (or something like
that).

the main problem here is how to properly store authentication data.
how far to go depends on your (future) requirements.

for my cms i went all the way and copied the unix filesystem
permission architecture (incl the concept of users in groups) to work
from mysql on an object-cloud (mapped to any "path(s)" elsewhere).

but you can just as easilly just map userIDs to array records
containing the keys that view.php works on. sorta like:
global $permissions;
$permissions =3D array (
100 =3D> array(
array (
dataNr =3D> 1,
itemNr =3D> 1,
fileID =3D> 'data.txt',
mayRead =3D> true,
mayWrite =3D> false
),
(...other objects user 100 has permissions for...)
userID =3D> permissionsList
);

you could use username instead of userid even, but i recommend against
that if you're going to store user-definition records in a db, of
course.


On Fri, Feb 19, 2010 at 7:19 PM, Michael Stroh wrote:
> I have a site I'm working on with some data that I want to be readable by=
anyone, but some files that I want to keep hidden from outside users. Here=
is an example of my file structure.
>
> /products/data1/item_1/data.txt
> /products/data2/item_2/data.txt
>
> I would like everything in data1 to be available by anyone who visits the=
site, but I want to keep items in the data2 folder to only be accessible t=
hrough certain web page which I hope to eventually require logins. Some of =
these items I'd like to not only display but also allow people to download.
>
> My main concern is that I don't want people to be able to guess the names=
of the files and then be able to access the information on them. Every 'it=
em' has an entry in a MySQL database which holds some information. I was th=
inking I could have randomly generated folder names to take the place of th=
e things like 'item_2' such as
>
> /products/data2/kl23j42i/data.txt
>
> and then link the folder name through a database entry. But I'm not sure =
if there are more elegant or easier ways to deal with this. Plus someone co=
uld still just try randomly querying the site until they get a match. I'd f=
irst like to just create a web page where you can go to access the hidden f=
iles but would later like to add more control for other users using logins =
and passwords.
>
> Most of my files are just text files and images. Any suggestions?
>
> Thanks in advance!
>
> Michael
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Advice on maintaining public and private files

As far as storing the files, use a seperate subdirectory called
"rawData" or something, and place all your files in there, aim for 10
- 5000 files per directory, and keep it logical.
But since you want to stop guessers from accessing it, use a
randomID() function that you create to generate a random subdirectory
under "rawData".
You could also use just the YYYY-MM-DD HH-MM-SS of the
submit/upload-date for the file or the last-modification date of the
file.

Then create something that maps IDs (dataNr, itemNr, fileID) to the
relative path under "rawData".

Then let view.php readfile() and output the requested file, instead of
sending any link to your "rawData"-subdirectory-location to the
browser.

It should be airtight then.

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Advice on maintaining public and private files

1 more thing: doing this right isn't easy. at all.
it took me more than a year to "do it properly".

you may wanna look around on sf.net for any package that can do this for yo=
u.

On Fri, Feb 19, 2010 at 7:19 PM, Michael Stroh wrote:
> I have a site I'm working on with some data that I want to be readable by=
anyone, but some files that I want to keep hidden from outside users. Here=
is an example of my file structure.
>
> /products/data1/item_1/data.txt
> /products/data2/item_2/data.txt
>
> I would like everything in data1 to be available by anyone who visits the=
site, but I want to keep items in the data2 folder to only be accessible t=
hrough certain web page which I hope to eventually require logins. Some of =
these items I'd like to not only display but also allow people to download.
>
> My main concern is that I don't want people to be able to guess the names=
of the files and then be able to access the information on them. Every 'it=
em' has an entry in a MySQL database which holds some information. I was th=
inking I could have randomly generated folder names to take the place of th=
e things like 'item_2' such as
>
> /products/data2/kl23j42i/data.txt
>
> and then link the folder name through a database entry. But I'm not sure =
if there are more elegant or easier ways to deal with this. Plus someone co=
uld still just try randomly querying the site until they get a match. I'd f=
irst like to just create a web page where you can go to access the hidden f=
iles but would later like to add more control for other users using logins =
and passwords.
>
> Most of my files are just text files and images. Any suggestions?
>
> Thanks in advance!
>
> Michael
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

RE: Advice on maintaining public and private files

From: Rene Veerman

> the "proper way" i know of is not the easiest to implement..;
>=20
> 1) create a php script that accepts enough parameters to get at your
data.
> eg: /products/view.php?dataNr=3D1&itemNr=3D1
> 2) let that script compare the current user (visitor who's logged in)
> to authentication data that tells which it if the user can access the
> data requested. if it fails, you can route the user to a std page or
> to a custom page (store in auth-data under "onFail")
> 3) use apache's RewriteRule in /products/.htaccess to point virtual
> urls to the view script; /products/data1/item_1/data.txt =3D
> /products/view.php?dataNr=3D1&itemNr=3D1&file=3Ddata.txt (or something =
like
> that).
>=20
> the main problem here is how to properly store authentication data.
> how far to go depends on your (future) requirements.

There are some easier tricks, but still not simple. Only the wrapper
script should be in the webroot space. Everything else should be outside
of it, but accessible by the user that the web server runs under. The
wrapper also manages the session and any other access controls
necessary, such as connections to a DB server. Once you parse the
parameters from the URL, use require() or require_once() to link in the
specific pages you need from outside webroot. This way none of the files
or paths are exposed to the browser and nobody can get to those pages
without going through the authentication in the wrapper. You can even
pull in more than one, so there could be one file for the banner, one
for the menu tree on the left column, one for a header, one for the page
specific content and one for the footer. It makes global updates
relatively easy, but can be a pain to get started.

Bob McConnell

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Advice on maintaining public and private files

On Fri, 19 Feb 2010 13:19:50 -0500, stroh@astroh.org (Michael Stroh) wrote:

>I have a site I'm working on with some data that I want to be readable by anyone, but some files that I want to keep hidden from outside users. Here is an example of my file structure.
>
>/products/data1/item_1/data.txt
>/products/data2/item_2/data.txt
>
>I would like everything in data1 to be available by anyone who visits the site, but I want to keep items in the data2 folder to only be accessible through certain web page which I hope to eventually require logins. Some of these items I'd like to not only display but also allow people to download.
>
>My main concern is that I don't want people to be able to guess the names of the files and then be able to access the information on them. Every 'item' has an entry in a MySQL database which holds some information. I was thinking I could have randomly generated folder names to take the place of the things like 'item_2' such as
>
>/products/data2/kl23j42i/data.txt
>
>and then link the folder name through a database entry. But I'm not sure if there are more elegant or easier ways to deal with this. Plus someone could still just try randomly querying the site until they get a match. I'd first like to just create a web page where you can go to access the hidden files but would later like to add more control for other users using logins and passwords.
>
>Most of my files are just text files and images. Any suggestions?
>
>Thanks in advance!
>
>Michael

I have been working on a website engine for some time, and have recently been addressing
these problems. The website layout is specified by textbased data files, with a separate
entry for each item on the page. These may be links to subdirectories or even other
websites, links to further index pages or links to individual items.

Users are divided into groups, e.g. Guest, Admin, or Manager, and each data file has a
field specifying who is allowed to use it. Each entry has a similar field, and when a data
file is being loaded the loader checks that the current user has permission to access it
before allowing the file to be loaded, and then as it processes each item in the file it
checks if the user has permission to view this item, and if not skips it. This means that
the user only sees the items he is entitled to see. There is nothing to indicate that
anything is being hidden from him.

At present I only have one allowable group for each file or item, and permit individual
users to belong to multiple groups (as set up by the administrator). On reflection it
would probably be better to assign each user to a single group, and allow multiple groups
to be given access to the file. At first I simply assigned each user a privilege level; 0,
1, 2, .. , but this prevented giving some user groups access to some areas of the website,
and other user groups access to others.

Each website has one area containing data, and a separate one containing the engine (which
has all the code). The data area also contains a small file index.php, which sets up site
dependent parameters, and then hands access to the engine. I have several different
websites sharing the same engine, and this means both that the individual websites can
specify different configuration files and security requirements. One website can be fully
accessible, another only accessible after the user is logged in, and another can have some
areas only accessible through a hidden log in.

I use parameters to specify which page to be loaded, but I've recently realised that this
is a significant security hole, as the parameters are readily visible, and convey a lot of
information about the structure of the site. On second thoughts it would have been better
to specify the various directories and files by numbers.

The engine is in a separate directory which is not under the root, so it is not readily
accessible, but I wanted the photos to be able to be bookmarked, which meant that they had
to be under the root, and I put the data files with them for simplicity. However this
means that they can also be downloaded, so I will have to move them to a different
location, as some of them contain valuable information. Fortunately the way the engine is
designed makes this reasonably simple to do.

You can see a very simple demonstration website at

http://www.cydalba.com/?new=1.

At present this is set up so that part of the website is only accessible by hidden log in.
If you access it via

http://www.cydalba.com/?new=1&action=log_in

you will be asked to log in, which you can do as 'Guest', with password 'Mandy17'. Some
more of the website will then be accessible.

Clancy


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Advice on maintaining public and private files

Michael Stroh wrote on 19/02/2010 19:19:
> I have a site I'm working on with some data that I want to be
> readable by anyone, but some files that I want to keep hidden from
> outside users. Here is an example of my file structure.
>
> /products/data1/item_1/data.txt
> /products/data2/item_2/data.txt

since no one has suggested it then... if you're on an Apache webserver
use a .htaccess file in data2 which contains:

Deny from all
Allow from none

That will do the trick and PHP can still fetch the files in data2 and
serve it to the user.

--
Kind regards
Kim Emax - masterminds.dk

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Advice on maintaining public and private files

Kim Madsen wrote:
> Michael Stroh wrote on 19/02/2010 19:19:
>> I have a site I'm working on with some data that I want to be
>> readable by anyone, but some files that I want to keep hidden from
>> outside users. Here is an example of my file structure.
>>
>> /products/data1/item_1/data.txt
>> /products/data2/item_2/data.txt
>
> since no one has suggested it then... if you're on an Apache webserver
> use a .htaccess file in data2 which contains:
>
> Deny from all
> Allow from none
>
> That will do the trick and PHP can still fetch the files in data2 and
> serve it to the user.
>

Glad you said this; I'd been waiting to see if anybody would - certainly
there is no quicker or easier way to solve this particular problem.

Also worth adding that you can easily password protect the directories
too using HTTP authorisation [1] (and even hook it in to LDAP or
suchlike very simply).

It's the curse of the PHP developer to try and use PHP to solve every
problem - we all fall fowl of it often (I've wasted years doing things
in PHP that really should have been done with a different tech).

[1] http://httpd.apache.org/docs/2.0/howto/auth.html

Regards!

Nathan

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Advice on maintaining public and private files

I use Kim's solution and take it one step forward. Htacces files can get lost or
corrupted, so....


In my config file I have the text string.

//region******** htaccess file text ********
// Code writes to /db folder; Admin mode checks file existence and text;
replaces with this if different.

$htaccessText = << # Prevent Direct Access to MiniRegDB DB Files

Order Deny,Allow
Deny from all

hta;
//endregion

In my main control file I call this function

/**
* checkHTaccessFile()
*
* Checks and restores htaccess Prevent Direct Access to MiniRegDB Program Files
*
* @param mixed $htaccessText in config file
* @return
*/
function checkHTaccessFile($htaccessText)
{
if(file_exists(MINIREG_DATA_DIR . '.htaccess') &&
file_get_contents(MINIREG_DATA_DIR . '.htaccess') == $htaccessText) return true;

file_put_contents(MINIREG_DATA_DIR . '.htaccess', $htaccessText);
return true;
}


On 2/20/2010 4:05 AM, Kim Madsen wrote:
> Michael Stroh wrote on 19/02/2010 19:19:
>> I have a site I'm working on with some data that I want to be
>> readable by anyone, but some files that I want to keep hidden from
>> outside users. Here is an example of my file structure.
>>
>> /products/data1/item_1/data.txt
> > /products/data2/item_2/data.txt
>
> since no one has suggested it then... if you're on an Apache webserver
> use a .htaccess file in data2 which contains:
>
> Deny from all
> Allow from none
>
> That will do the trick and PHP can still fetch the files in data2 and
> serve it to the user.
>

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php