Berkeley DB Installation

I posted a question 10 days ago and I've found that somehow or other it got
tagged onto the end of another thread so perhaps has been overlooked so I'm
trying again! Here's an amended version of that question.

I've been using a networked PC running NT4 Server to test some Perl CGI
scripts using the remote debugging features of Komodo, which is running on
an XP Pro PC. I've been experimenting with running the scripts on a virtual
machine (using MS Virtual PC) hosted by the XP Pro PC so that I only need to
have one PC to test these Perl scripts. This is working well except that I'm
having problems with some Perl scripts that use DB_File.

I don't seem to be able to read the same database files on the virtual
machine and I have discovered that although I thought I had the same set up
on both machines this is not the case which could account for this. The
standalone PC is running DB_File version 1.73 and Berkeley DB version
2.004010 while the virtual machine has DB_File version 1.806 and Berkeley DB
version 4.001025.

I came across a test to check how well Berkeley DB is working in conjunction
with Perl and I found that with the standalone machine it took 1 wallclock
second to use DB_File to write 10,000 records while with the virtual machine
it took 58. There was a note associated with this test to say that there was
a bug in version 4.1 of Berkeley DB which might not have been corrected and
which would have caused it to run very slowly. I'm wondering if this is
perhaps the problem although it could just be that this is because it's
running in a virtual machine and/or I've not allocated enough RAM to the
virtual machine.

I don't know how I've managed to have a different version of Berkeley DB on
the virtual machine. Am I correct in thinking that Berkeley DB comes with
the OS and is not part of the Perl installation? I see that the Service Pack
on the standalone PC is SP5 whereas the one on the virtual machine is SP6 so
perhaps this accounts for the difference.


_______________________________________________
ActivePerl mailing list
ActivePerl@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs

RE: Berkeley DB Installation

Hi Jim
Hi Jim

Thanks for your comments - the MS Virtual PC side of things doesn't add any
complications, my question can just be read as wanting to transfer the same
Perl/BerkeleyDB system from one machine to another.

It seems that there is a difference between how the BerkeleyDB engine is
installed as between a Windows and a Unix/Linux installation - see Brian
Raven's comments. I've been trying ProcessExplorer at someone else's
suggestion to see which happens when the DB_File module is loaded. I
debugged the small Perl script that was producing some test BerkeleyDB files
using Komodo and I found that the only change to the loaded processes when
the DB_File module was loaded was a DB_File.dll one. Also I found that the
properties for that dll had a string that referred to version 4.1.25 of
BerkeleyDB which matched the version of the BerkeleyDB engine that my test
script showed. I'm not sure whether this confirms that the BerkeleyDB engine
is controlled by the DB_File module for a Windows machine?

Unfortunately I'm testing Perl scripts that already use DB_File so I can't
change the database system. These were written so that they could be run on
servers, whether Windows or Unix/Linux, that didn't have any more
sophisticated database system, such as MySQL, Oracle, so using DB_File with
the BerkeleyDB engine was the best option (assuming the DB_File module was
available).

Eric Robertson

-----Original Message-----
From: Jim Schueler [mailto:jschueler@motorcityinteractive.com]
Sent: Monday, November 12, 2007 6:07 PM
To: Eric Robertson
Cc: activeperl@listserv.ActiveState.com
Subject: Re: Berkeley DB Installation

Hello.

I don't know anything about MS Virtual PC, so I don't understand your
question very well. I have about 15 years expertise with BerkeleyDB on
Unix and wrote the BerkeleyDB::Lite module. I only monitor this group to
answer BerkeleyDB questions, even though I don't think my module is
distributed by ActivePerl.

This doesn't seem to be a Perl problem, and even less likely to concern
ActivePerl. There is a BerkeleyDB Usenet group that is obstensibly
monitored by what used to be SleepyCat (now Oracle) engineers. Have you
posted there?

BerkeleyDB is an embedded database. It consists of a library that adds
database functionality to any application. Usually, database
functionality means persistent storage of significant volumes of data.
So on a diskless platform, a BerkeleyDB application at least requires a
virtualized file system. Additionally BerkeleyDB optimization performs
caching and swapping separate from the OS/FS. So a virtual file system
needs to be optimized specifically for BerkeleyDB to begin matching the
performance of native hard storage.

In short BerkeleyDB is a database engine instead of a database application
(although the BerkeleyDB distribution includes a primitive database server
to migrate up to a client/server model). In your case, why use BerkeleyDB
at all? The Perl ability to represent databases as tied objects was the
original appeal of BerkeleyDB back in the day. But now, DBI does the same
thing for any database model/application.

Jim Schueler
Motor City Interactive



On Sat, 10 Nov 2007, Eric Robertson wrote:

> I posted a question 10 days ago and I've found that somehow or other it
got
> tagged onto the end of another thread so perhaps has been overlooked so
I'm
> trying again! Here's an amended version of that question.
>
> I've been using a networked PC running NT4 Server to test some Perl CGI
> scripts using the remote debugging features of Komodo, which is running on
> an XP Pro PC. I've been experimenting with running the scripts on a
virtual
> machine (using MS Virtual PC) hosted by the XP Pro PC so that I only need
to
> have one PC to test these Perl scripts. This is working well except that
I'm
> having problems with some Perl scripts that use DB_File.
>
> I don't seem to be able to read the same database files on the virtual
> machine and I have discovered that although I thought I had the same set
up
> on both machines this is not the case which could account for this. The
> standalone PC is running DB_File version 1.73 and Berkeley DB version
> 2.004010 while the virtual machine has DB_File version 1.806 and Berkeley
DB
> version 4.001025.
>
> I came across a test to check how well Berkeley DB is working in
conjunction
> with Perl and I found that with the standalone machine it took 1 wallclock
> second to use DB_File to write 10,000 records while with the virtual
machine
> it took 58. There was a note associated with this test to say that there
was
> a bug in version 4.1 of Berkeley DB which might not have been corrected
and
> which would have caused it to run very slowly. I'm wondering if this is
> perhaps the problem although it could just be that this is because it's
> running in a virtual machine and/or I've not allocated enough RAM to the
> virtual machine.
>
> I don't know how I've managed to have a different version of Berkeley DB
on
> the virtual machine. Am I correct in thinking that Berkeley DB comes with
> the OS and is not part of the Perl installation? I see that the Service
Pack
> on the standalone PC is SP5 whereas the one on the virtual machine is SP6
so
> perhaps this accounts for the difference.
>
>
> _______________________________________________
> ActivePerl mailing list
> ActivePerl@listserv.ActiveState.com
> To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
>

--



_______________________________________________
ActivePerl mailing list
ActivePerl@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs

RE: Berkeley DB Installation

Jim

Note what you say about compatibility, thanks. I hadn't heard that DB_File
had been superseded but I'm stuck with using the former.

Many thanks for the offer of extracting the data from the BerkeleyDB
database but that's not really my problem. The BB software that I'm working
with on my second (virtual) machine first writes away each new posting to a
new text file and then adds the data to the BerkeleyDB database, where it's
available for extraction to produce all the postings according to thread. I
have all these text files on that machine, and it should be possible to
reconstitute the BerkeleyDB database from these files. However in view of
what you say I'll have to somehow change the version of the BerkeleyDB
engine that's on that machine.

Also I'm not sure that there isn't a compatibility problem between Windows
and Unix. Some time ago I was using DB_File on a Unix server and trying to
test the script on a Windows NT machine and I had problems. I may be wrong
but I think that someone told me that even with the same version of
BerkeleyDB on both machines the database files produced would be
incompatible.

Eric

-----Original Message-----
From: Jim Schueler [mailto:jschueler@motorcityinteractive.com]
Sent: Monday, November 12, 2007 10:22 PM
To: Eric Robertson
Cc: activeperl@listserv.ActiveState.com
Subject: RE: Berkeley DB Installation

According to CPAN, DB_File is only compatible with BerkeleyDB v1.85 and
below. Among the data files, anything after that version is incompatible
with anything before that version. I believe that you can still download
v1.85 for compatability.

DB_File has been superceded by BerkeleyDB, also written by Paul Marquess.

If your data file is 4.1.25, and you are simply trying to extract the
data. I can probably do your extraction as long as there is no
compatibility issue between Windows and Unix (I'm guessing there isn't).
It would probably only take 5 - 10 minutes of my time. Interested?

-Jim




On Mon, 12 Nov 2007, Eric Robertson wrote:

> Hi Jim
> Hi Jim
>
> Thanks for your comments - the MS Virtual PC side of things doesn't add
any
> complications, my question can just be read as wanting to transfer the
same
> Perl/BerkeleyDB system from one machine to another.
>
> It seems that there is a difference between how the BerkeleyDB engine is
> installed as between a Windows and a Unix/Linux installation - see Brian
> Raven's comments. I've been trying ProcessExplorer at someone else's
> suggestion to see which happens when the DB_File module is loaded. I
> debugged the small Perl script that was producing some test BerkeleyDB
files
> using Komodo and I found that the only change to the loaded processes when
> the DB_File module was loaded was a DB_File.dll one. Also I found that the
> properties for that dll had a string that referred to version 4.1.25 of
> BerkeleyDB which matched the version of the BerkeleyDB engine that my test
> script showed. I'm not sure whether this confirms that the BerkeleyDB
engine
> is controlled by the DB_File module for a Windows machine?
>
> Unfortunately I'm testing Perl scripts that already use DB_File so I can't
> change the database system. These were written so that they could be run
on
> servers, whether Windows or Unix/Linux, that didn't have any more
> sophisticated database system, such as MySQL, Oracle, so using DB_File
with
> the BerkeleyDB engine was the best option (assuming the DB_File module was
> available).
>
> Eric Robertson
>
> -----Original Message-----
> From: Jim Schueler [mailto:jschueler@motorcityinteractive.com]
> Sent: Monday, November 12, 2007 6:07 PM
> To: Eric Robertson
> Cc: activeperl@listserv.ActiveState.com
> Subject: Re: Berkeley DB Installation
>
> Hello.
>
> I don't know anything about MS Virtual PC, so I don't understand your
> question very well. I have about 15 years expertise with BerkeleyDB on
> Unix and wrote the BerkeleyDB::Lite module. I only monitor this group to
> answer BerkeleyDB questions, even though I don't think my module is
> distributed by ActivePerl.
>
> This doesn't seem to be a Perl problem, and even less likely to concern
> ActivePerl. There is a BerkeleyDB Usenet group that is obstensibly
> monitored by what used to be SleepyCat (now Oracle) engineers. Have you
> posted there?
>
> BerkeleyDB is an embedded database. It consists of a library that adds
> database functionality to any application. Usually, database
> functionality means persistent storage of significant volumes of data.
> So on a diskless platform, a BerkeleyDB application at least requires a
> virtualized file system. Additionally BerkeleyDB optimization performs
> caching and swapping separate from the OS/FS. So a virtual file system
> needs to be optimized specifically for BerkeleyDB to begin matching the
> performance of native hard storage.
>
> In short BerkeleyDB is a database engine instead of a database application
> (although the BerkeleyDB distribution includes a primitive database server
> to migrate up to a client/server model). In your case, why use BerkeleyDB
> at all? The Perl ability to represent databases as tied objects was the
> original appeal of BerkeleyDB back in the day. But now, DBI does the same
> thing for any database model/application.
>
> Jim Schueler
> Motor City Interactive
>
>
>
> On Sat, 10 Nov 2007, Eric Robertson wrote:
>
> > I posted a question 10 days ago and I've found that somehow or other it
> got
> > tagged onto the end of another thread so perhaps has been overlooked so
> I'm
> > trying again! Here's an amended version of that question.
> >
> > I've been using a networked PC running NT4 Server to test some Perl CGI
> > scripts using the remote debugging features of Komodo, which is running
on
> > an XP Pro PC. I've been experimenting with running the scripts on a
> virtual
> > machine (using MS Virtual PC) hosted by the XP Pro PC so that I only
need
> to
> > have one PC to test these Perl scripts. This is working well except that
> I'm
> > having problems with some Perl scripts that use DB_File.
> >
> > I don't seem to be able to read the same database files on the virtual
> > machine and I have discovered that although I thought I had the same set
> up
> > on both machines this is not the case which could account for this. The
> > standalone PC is running DB_File version 1.73 and Berkeley DB version
> > 2.004010 while the virtual machine has DB_File version 1.806 and
Berkeley
> DB
> > version 4.001025.
> >
> > I came across a test to check how well Berkeley DB is working in
> conjunction
> > with Perl and I found that with the standalone machine it took 1
wallclock
> > second to use DB_File to write 10,000 records while with the virtual
> machine
> > it took 58. There was a note associated with this test to say that there
> was
> > a bug in version 4.1 of Berkeley DB which might not have been corrected
> and
> > which would have caused it to run very slowly. I'm wondering if this is
> > perhaps the problem although it could just be that this is because it's
> > running in a virtual machine and/or I've not allocated enough RAM to the
> > virtual machine.
> >
> > I don't know how I've managed to have a different version of Berkeley DB
> on
> > the virtual machine. Am I correct in thinking that Berkeley DB comes
with
> > the OS and is not part of the Perl installation? I see that the Service
> Pack
> > on the standalone PC is SP5 whereas the one on the virtual machine is
SP6
> so
> > perhaps this accounts for the difference.
> >
> >
> > _______________________________________________
> > ActivePerl mailing list
> > ActivePerl@listserv.ActiveState.com
> > To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
> >
>
>

--



_______________________________________________
ActivePerl mailing list
ActivePerl@listserv.ActiveState.com
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs