How to persist and query a hash lookup in memory with Perl ?

Hi I have a web application that needs to on an adhoc basis query a
hash lookup which should be persisted in memory - I have seen this
module "Persistent::Memory - A Persistent Class implemented using
Memory (RAM)" but I wanted to see if there was a much simpler approach
to taking a hash map and making it 'callable' while persisted in
memory using standard features not requiring a special Perl module
install..

Any thoughts would be appreciated.
Thanks,
Jack

Re: How to persist and query a hash lookup in memory with Perl ?

Jack wrote:
> Hi I have a web application that needs to on an adhoc basis query a
> hash lookup which should be persisted in memory - I have seen this
> module "Persistent::Memory - A Persistent Class implemented using
> Memory (RAM)" but I wanted to see if there was a much simpler approach
> to taking a hash map and making it 'callable' while persisted in
> memory using standard features not requiring a special Perl module
> install..
>
> Any thoughts would be appreciated.

Why not a tied hash? See e.g. "perldoc DB_File".

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl

Re: How to persist and query a hash lookup in memory with Perl ?

On Nov 28, 6:37 pm, Gunnar Hjalmarsson wrote:
> Jack wrote:
> > Hi I have a web application that needs to on an adhoc basis query a
> > hash lookup which should be persisted in memory - I have seen this
> > module "Persistent::Memory - A Persistent Class implemented using
> > Memory (RAM)" but I wanted to see if there was a much simpler approach
> > to taking a hash map and making it 'callable' while persisted in
> > memory using standard features not requiring a special Perl module
> > install..
>
> > Any thoughts would be appreciated.
>
> Why not a tied hash? See e.g. "perldoc DB_File".
>
> --
> Gunnar Hjalmarsson
> Email:http://www.gunnar.cc/cgi-bin/contact.pl

Hi and thank you however that looks too complex - just trying to put a
hash in memory, why is that so hard to do ? Is there a simpler way ?

Re: How to persist and query a hash lookup in memory with Perl ?

jackposemsky@yahoo.com wrote in
news:7c5df670-d214-40c2-b59d-9ba1f5785618@s36g2000prg.google groups.com:

> On Nov 28, 6:37 pm, Gunnar Hjalmarsson wrote:
>> Jack wrote:
>> > Hi I have a web application that needs to on an adhoc basis query a
>> > hash lookup which should be persisted in memory - I have seen this
>> > module "Persistent::Memory - A Persistent Class implemented using
>> > Memory (RAM)" but I wanted to see if there was a much simpler
>> > approach to taking a hash map and making it 'callable' while
>> > persisted in memory using standard features not requiring a special
>> > Perl module install..
>>
>> > Any thoughts would be appreciated.
>>
>> Why not a tied hash? See e.g. "perldoc DB_File".
>>
>> --
>> Gunnar Hjalmarsson
>> Email:http://www.gunnar.cc/cgi-bin/contact.pl

Don't quote sigs.

> Hi and thank you however that looks too complex - just trying to put a
> hash in memory, why is that so hard to do ? Is there a simpler way ?

It looks like programming might be too hard for you.

Sinan

--
A. Sinan Unur <1usa@llenroc.ude.invalid>
(remove .invalid and reverse each component for email address)
clpmisc guidelines:

Re: How to persist and query a hash lookup in memory with Perl ?

On Wed, 28 Nov 2007 18:22:34 -0800 (PST) Jack wrote:

J> Hi I have a web application that needs to on an adhoc basis query a
J> hash lookup which should be persisted in memory - I have seen this
J> module "Persistent::Memory - A Persistent Class implemented using
J> Memory (RAM)" but I wanted to see if there was a much simpler approach
J> to taking a hash map and making it 'callable' while persisted in
J> memory using standard features not requiring a special Perl module
J> install..

Are you talking about sharing the data between several instances of your
web application, so whichever one handles a request can read or update
data?

Use a database. There are cases where this is not the right approach,
but it doesn't sound like it in yours.

Ted

Re: How to persist and query a hash lookup in memory with Perl ?

jackposemsky@yahoo.com wrote:
> On Nov 28, 6:37 pm, Gunnar Hjalmarsson wrote:
> > Jack wrote:
> > > Hi I have a web application that needs to on an adhoc basis query a
> > > hash lookup which should be persisted in memory - I have seen this
> > > module "Persistent::Memory - A Persistent Class implemented using
> > > Memory (RAM)" but I wanted to see if there was a much simpler
> > > approach to taking a hash map and making it 'callable' while
> > > persisted in memory using standard features not requiring a special
> > > Perl module install..
> >
> > > Any thoughts would be appreciated.
> >
> > Why not a tied hash? See e.g. "perldoc DB_File".
>
> Hi and thank you however that looks too complex - just trying to put a
> hash in memory, why is that so hard to do ? Is there a simpler way ?

If you aren't willing to install modules and you aren't willing to use the
ones that are (often) standard, then I guess the next option would be to
hire someone who is willing to do one of those things.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

Re: How to persist and query a hash lookup in memory with Perl ?

In article
<7c5df670-d214-40c2-b59d-9ba1f5785618@s36g2000prg.googlegroups.com>,
wrote:

> On Nov 28, 6:37 pm, Gunnar Hjalmarsson wrote:
> > Jack wrote:
> > > Hi I have a web application that needs to on an adhoc basis query a
> > > hash lookup which should be persisted in memory - I have seen this
> > > module "Persistent::Memory - A Persistent Class implemented using
> > > Memory (RAM)" but I wanted to see if there was a much simpler approach
> > > to taking a hash map and making it 'callable' while persisted in
> > > memory using standard features not requiring a special Perl module
> > > install..
> >
> > > Any thoughts would be appreciated.
> >
> > Why not a tied hash? See e.g. "perldoc DB_File".
> >
> > --
> > Gunnar Hjalmarsson
> > Email:http://www.gunnar.cc/cgi-bin/contact.pl
>
> Hi and thank you however that looks too complex - just trying to put a
> hash in memory, why is that so hard to do ? Is there a simpler way ?

Your question is confusing. All Perl variables, including hashes, are
in memory. The trick to achieving "persistence" between execution
instances of your web application is to have the data on a permanent
data store, such as a disk file. Memory variables disappear once your
Perl process terminates. Using a tied hash, per Gunnar's suggestion, is
one way to accomplish this. There are others, such as using the
Storable module to serialize your hash so you can write it to memory.

I have not use Persistent::Memory, but you might want to check out the
Persistent::File module instead. Both of those modules seem to be part
of the Persistent framework of Perl modules.

See
d>

--
Jim Gibson

Posted Via Usenet.com Premium Usenet Newsgroup Services
----------------------------------------------------------
** SPEED ** RETENTION ** COMPLETION ** ANONYMITY **
----------------------------------------------------------
http://www.usenet.com

Re: How to persist and query a hash lookup in memory with Perl ?

On Nov 29, 9:56 am, Jim Gibson wrote:
> In article
> <7c5df670-d214-40c2-b59d-9ba1f5785...@s36g2000prg.googlegroups.com>,
>
>
>
>
>
> wrote:
> > On Nov 28, 6:37 pm, Gunnar Hjalmarsson wrote:
> > > Jack wrote:
> > > > Hi I have a web application that needs to on an adhoc basis query a
> > > > hash lookup which should be persisted in memory - I have seen this
> > > > module "Persistent::Memory - A Persistent Class implemented using
> > > > Memory (RAM)" but I wanted to see if there was a much simpler approach
> > > > to taking a hash map and making it 'callable' while persisted in
> > > > memory using standard features not requiring a special Perl module
> > > > install..
>
> > > > Any thoughts would be appreciated.
>
> > > Why not a tied hash? See e.g. "perldoc DB_File".
>
> > > --
> > > Gunnar Hjalmarsson
> > > Email:http://www.gunnar.cc/cgi-bin/contact.pl
>
> > Hi and thank you however that looks too complex - just trying to put a
> > hash in memory, why is that so hard to do ? Is there a simpler way ?
>
> Your question is confusing. All Perl variables, including hashes, are
> in memory. The trick to achieving "persistence" between execution
> instances of your web application is to have the data on a permanent
> data store, such as a disk file. Memory variables disappear once your
> Perl process terminates. Using a tied hash, per Gunnar's suggestion, is
> one way to accomplish this. There are others, such as using the
> Storable module to serialize your hash so you can write it to memory.
>
> I have not use Persistent::Memory, but you might want to check out the
> Persistent::File module instead. Both of those modules seem to be part
> of the Persistent framework of Perl modules.
>
> See
> > d>
>
> --
> Jim Gibson
>
> Posted Via Usenet.com Premium Usenet Newsgroup Services
> ----------------------------------------------------------
> ** SPEED ** RETENTION ** COMPLETION ** ANONYMITY **
> ----------------------------------------------------------
> http://www.usenet.com- Hide quoted text -
>
> - Show quoted text -

Thanks I am willing to install Perl Modules, really wondering if there
is a native ability to do it out of the box in Perl. I got persistent
memory to work but its not clear how to "pin" a hash lookup into
memory since .pl files and the persistent memory code example itself
are only as good as the life of the execution of the script !! Still
not clear how to hold a has in Memory - even if you were to create a
windows service out of it with activestate PDK, not clear how to write
the script so the script doesnt stop execution but sits and waits for
a socket connection or something.. Do I just have script 1 generate
a hash with "sleep forever" and then script 2 can call that hash ??
I dont want to use another database for this. So, not using another
DB, how does one pin and persist and then query a hash in memory using
perl or a perl module.

Thanks for the help all !
Jack

Re: How to persist and query a hash lookup in memory with Perl ?

Quoth Jack :

>
> Thanks I am willing to install Perl Modules, really wondering if there
> is a native ability to do it out of the box in Perl.

Well, on systems that support it you can use SysV shared memory, for
which Perl natively has the raw shm* calls. This doesn't work on
Windows, though.

You could try IPC::Shareable for Unix and Win32::MMF::Shareable (which
has the same interface) for Win32, which provide an easier interface to
shared memory.

Another alternative would be mmap, using the Sys::MMap module. mmap is
substantially better behaved than SysV shm, and portable to Windows.

> I got persistent memory to work but its not clear how to "pin" a hash
> lookup into memory since .pl files and the persistent memory code
> example itself are only as good as the life of the execution of the
> script !!

In what sense then is this memory 'persistent'? I think you are rather
confused about what is and isn't possible under modern operating
systems: unless you take steps to prevent it, all memory allocated by a
process is released when that process exits. This is a feature :).

Ben

Re: How to persist and query a hash lookup in memory with Perl ?

Jack wrote:
[...]
> So, not using another DB

If you're already using a Database then just store the data there, why
would you use another Database to store it?

>how does one pin and persist and then query a hash in memory using
> perl or a perl module.

Whatever 'pin' a hash means.

Why do you want to do this? Maybe that's the real problem. You
think that's what you want to do, but it's really not needed.

You could have a daemon running, that reads/creates the hash and
then it would have the data in memory. You could then then connect
to it via a socket, but why the heck do you want to do all of
that? That's really not Perl 101 and I'm pretty sure you'd
end up spending a lot of time understanding it and trying to get it
working, however it'll take 20-seconds to write and will be very
fast, using a tied hash.

There are a lot of examples online and you can read the documentation
that comes with perl.

perldoc DB_File

If you want to go the whole daemon and IPC/socket route:

perldoc perlipc

Re: How to persist and query a hash lookup in memory with Perl ?

On Nov 29, 3:35 pm, Ben Morrow wrote:
> Quoth Jack :
>
>
>
>
> > Thanks I am willing to install Perl Modules, really wondering if there
> > is a native ability to do it out of the box in Perl.
>
> Well, on systems that support it you can use SysV shared memory, for
> which Perl natively has the raw shm* calls. This doesn't work on
> Windows, though.
>
> You could try IPC::Shareable for Unix and Win32::MMF::Shareable (which
> has the same interface) for Win32, which provide an easier interface to
> shared memory.
>
> Another alternative would be mmap, using the Sys::MMap module. mmap is
> substantially better behaved than SysV shm, and portable to Windows.
>
> > I got persistent memory to work but its not clear how to "pin" a hash
> > lookup into memory since .pl files and the persistent memory code
> > example itself are only as good as the life of the execution of the
> > script !!
>
> In what sense then is this memory 'persistent'? I think you are rather
> confused about what is and isn't possible under modern operating
> systems: unless you take steps to prevent it, all memory allocated by a
> process is released when that process exits. This is a feature :).
>
> Ben

ok.. lets talk with code - I got this to work with DB_File.. my
fundamental question is, once you create the tied hash like the below,
and the program is done executing, how do I access the hash and query
it from another Perl execution program ?? I tried using the API
interface afterwards as described here http://perldoc.perl.org/DB_File.html
, see the error at bottom:
### RUN PROGRAM 1 #####
use warnings ;
use strict ;
use DB_File ;
our (%h, $k, $v) ; unlink "fruit" ;
tie %h, "DB_File", "fruit", O_RDWR|O_CREAT, 0666, $DB_HASH
or die "Cannot open file 'fruit': $!\n"; # Add a few key/
value pairs to the file
$h{"apple"} = "red" ;
$h{"orange"} = "orange" ;
$h{"banana"} = "yellow" ;
$h{"tomato"} = "red" ; # Check for existence of a key
print "Banana Exists\n\n" if $h{"banana"} ; # Delete a key/
value pair.
delete $h{"apple"} ; # print the contents of the file
while (($k, $v) = each %h)
{ print "$k -> $v\n" }
untie %h ;

#####START SEPARATE PROGRAM 2 ###
use warnings ;
use strict ;
use DB_File ;
$db = tie %h, "DB_File", "testfile1" ;
foreach (keys %h)
{ print "$_\n" }
### PROGRAM 2 ERRORS:
Global symbol "$db" requires explicit package name at
test_dbfile1_query.pl .
Global symbol "%h" requires explicit package name at
test_dbfile1_query.pl .
Global symbol "%h" requires explicit package name at
test_dbfile1_query.pl 5.
Execution of test_dbfile1_query.pl aborted due to compilation errors.

Re: How to persist and query a hash lookup in memory with Perl ?

Jack wrote:
> once you create the tied hash like the below, and the program is done
> executing, how do I access the hash and query it from another Perl
> execution program ??

Like this, for instance:

use DB_File;
tie my %h, 'DB_File', 'fruit', O_RDONLY or die $!;
print "$_: $h{$_}\n" for keys %h;

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl

Re: How to persist and query a hash lookup in memory with Perl ?

On Nov 30, 2:15 am, Gunnar Hjalmarsson wrote:
> Jackwrote:
> > once you create the tiedhashlike the below, and the program is done
> > executing, how do I access thehashand query it from another Perl
> > execution program ??
>
> Like this, for instance:
>
> use DB_File;
> tie my %h, 'DB_File', 'fruit', O_RDONLY or die $!;
> print "$_: $h{$_}\n" for keys %h;
>
> --
> Gunnar Hjalmarsson
> Email:http://www.gunnar.cc/cgi-bin/contact.pl

Thanks that worked but cant believe I need berkeley db just to keep a
hash in memory after a program has executed. In C# and other
languages its a simple statement. Can someone who knows
persistent::memory provide a real working example of this, or other
approach.
thanks,
Jack

Re: How to persist and query a hash lookup in memory with Perl ?

Quoth Jack :
> On Nov 30, 2:15 am, Gunnar Hjalmarsson wrote:
> > Jackwrote:
> > > once you create the tiedhashlike the below, and the program is done
> > > executing, how do I access thehashand query it from another Perl
> > > execution program ??
> >
> > Like this, for instance:
> >
> > use DB_File;
> > tie my %h, 'DB_File', 'fruit', O_RDONLY or die $!;
> > print "$_: $h{$_}\n" for keys %h;
>
> Thanks that worked but cant believe I need berkeley db just to keep a
> hash in memory after a program has executed. In C# and other
> languages its a simple statement.

I don't believe you. Can you provide an example, and explain how it
works? If you want something to persist after your process exits you
have to put it somewhere, either in the filesystem or somewhere like a
SysV shm segment that *ought* to be in the filesystem. This is a feature
of your OS.

Ben

Re: How to persist and query a hash lookup in memory with Perl ?

Jack wrote:
> On Nov 30, 2:15 am, Gunnar Hjalmarsson wrote:
> > Jackwrote:
> > > once you create the tiedhashlike the below, and the program is done
> > > executing, how do I access thehashand query it from another Perl
> > > execution program ??
> >
> > Like this, for instance:
> >
> > use DB_File;
> > tie my %h, 'DB_File', 'fruit', O_RDONLY or die $!;
> > print "$_: $h{$_}\n" for keys %h;
> >
> > --
> > Gunnar Hjalmarsson
> > Email:http://www.gunnar.cc/cgi-bin/contact.pl
>
> Thanks that worked but cant believe I need berkeley db just to keep a
> hash in memory after a program has executed. In C# and other
> languages its a simple statement.

Which statement?

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

Re: How to persist and query a hash lookup in memory with Perl ?

On Mon, 3 Dec 2007 21:45:45 -0800 (PST), Jack wrote:
>
> Thanks that worked but cant believe I need berkeley db just to keep a
> hash in memory after a program has executed.

Why? Modern operating systems relase memory used by a process once
the process finishes (which is a Good Thing). Just leaving something
hanging around in memory isn't something simple.

> In C# and other languages its a simple statement.

Is it? Care to share?

> Can someone who knows persistent::memory provide a real working example of
> this, or other approach.

I think you mean Persistent::Memory (case is important in Perl).
I also think you misunderstand what Peristent::Memory does - from a quick
glance it appears to be part of the Persistent framework and a base for other
classes, not some magic that keeps data in memory between process invocations.

Re: How to persist and query a hash lookup in memory with Perl ?

Jack wrote:
> On Nov 30, 2:15 am, Gunnar Hjalmarsson wrote:
>> Jackwrote:
>>> once you create the tiedhashlike the below, and the program is done
>>> executing, how do I access thehashand query it from another Perl
>>> execution program ??
>> Like this, for instance:
>>
>> use DB_File;
>> tie my %h, 'DB_File', 'fruit', O_RDONLY or die $!;
>> print "$_: $h{$_}\n" for keys %h;
>
> Thanks that worked but cant believe I need berkeley db just to keep a
> hash in memory after a program has executed.

I didn't say you do. As others have mentioned, there are other options.
For instance, there are other, simpler dbm files. SDBM_File is available
in Perl on *nix and Windows.

C:\home>type test.pl
use SDBM_File;
use Fcntl;
tie my %h, 'SDBM_File', 'fruit', O_RDWR|O_CREAT, 0600 or die $!;
%h = ( banana => 'yellow', apple => 'green', tomato => 'red' );

C:\home>type test2.pl
use SDBM_File;
use Fcntl;
tie my %h, 'SDBM_File', 'fruit', O_RDONLY, 0400 or die $!;
print "$_ -> $h{$_}\n" for keys %h;

C:\home>test.pl

C:\home>test2.pl
banana -> yellow
apple -> green
tomato -> red

C:\home>

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl

Re: How to persist and query a hash lookup in memory with Perl ?

* Jack wrote:
>
> Hi I have a web application that needs to on an adhoc basis query a
> hash lookup which should be persisted in memory - I have seen this
> module "Persistent::Memory - A Persistent Class implemented using
> Memory (RAM)" but I wanted to see if there was a much simpler approach
> to taking a hash map and making it 'callable' while persisted in
> memory using standard features not requiring a special Perl module
> install..

Why don't you tell us a bit more about your particular use case?

For all we know, what you're really looking for is something
like memcached¹.

--
Lars Haugseth

"If anyone disagrees with anything I say, I am quite prepared not only to
retract it, but also to deny under oath that I ever said it." -Tom Lehrer

¹ http://www.danga.com/memcached/