Comments on stability of SQLite for storage

Greetings.

I'm beginning a project where I have to store some stuff on the disk
somehow. At present, I'm using SQLite *only* for temporary storage. I
had planned to dump data ( which is basically config data, but is
absolutely critical in terms of keeping the project held together ) to
either XML or CSV, but now this is getting more cumbersome that I'd
expected.

Who has some anecdotal evidence about the reliability of SQLite files
for storing data? Should I drop the XML / CSV idea and just use SQLite?

--
Daniel Kasak
IT Developer
NUS Consulting Group
Level 5, 77 Pacific Highway
North Sydney, NSW, Australia 2060
T: (+61) 2 9922-7676 / F: (+61) 2 9922 7989
email: dkasak@nusconsulting.com.au
website: http://www.nusconsulting.com.au

Re: Comments on stability of SQLite for storage

dkasak@nusconsulting.com.au (Daniel Kasak) writes:

> Greetings.
>
> I'm beginning a project where I have to store some stuff on the disk
> somehow. At present, I'm using SQLite *only* for temporary storage. I
> had planned to dump data ( which is basically config data, but is
> absolutely critical in terms of keeping the project held together ) to
> either XML or CSV, but now this is getting more cumbersome that I'd
> expected.
>
> Who has some anecdotal evidence about the reliability of SQLite files
> for storing data? Should I drop the XML / CSV idea and just use SQLite?

They are reliable (sqlite files) as other files of a filesystem.
E.g. I use it for config & contents data in a production system without
any trouble.
See also
http://technology.guardian.co.uk/weekly/story/0,,2107239,00. html

--
Radek

Re: Comments on stability of SQLite for storage

Daniel Kasak wrote:
> Greetings.
>
> I'm beginning a project where I have to store some stuff on the disk
> somehow. At present, I'm using SQLite *only* for temporary storage. I
> had planned to dump data ( which is basically config data, but is
> absolutely critical in terms of keeping the project held together ) to
> either XML or CSV, but now this is getting more cumbersome that I'd
> expected.
>
> Who has some anecdotal evidence about the reliability of SQLite files
> for storing data? Should I drop the XML / CSV idea and just use SQLite?

I don't see this as DBI-related, but hey...

You obviously need some form of backups of your data. Whether you do
that by a copy of the SQLite database files or through a dump into
XML/CSV is entirely up to you.

If you have the desire to process or view the data through some form
other than SQLite (e.g. grep/sort), then using CSV or other textual
formats such as XML makes sense.

If you care about getting the data as it was at an arbitrary point in
time, you need a real database that does transactional logging and log
archiving. It sounds like you need less than that, in which case just
copying the SQLite files should be sufficient.

Whatever you do, make sure you test querying the backed-up files and
make sure you have some sort of script that shows the difference in
contents between archived files (in whatever form) and the current
state. If you archive files in non-SQLite format, make sure you can
recreate the database from the archived files - and test this for each
major release of your project.