Working with arrays of arrays

ok guys I have working code finally that does what I want it to do. It reads
in the data file to an array of arrays. It works with it and even adds data
just fine to it but always to the end of the top level array. The code sped
up the program just as I expected it would, but I already found a possible
speed improvement.

Right now all new data records are placed at the end of the top level array.
By record I mean a new array is being added to the array of arrays. However,
it is very likely that this record will be needed almost imeadiately, but it
is now at the end of the data array. This causes the program to parce over
the whole array to fined it again. It would be more efficient to put that
new record as well as any recently updated records at the top of the array
so it is likely to be one of the first found.

In other words over time records used the most often will rise to the top of
the data stack while dead wood will sink to the bottom.

Working code snips.

# put the player data file into an array of arrays to work with it from
system memory
open(INF, $playerfile) or print "Error: File Load Failed";
while () {
chomp;
push @loaded, [ split ];
}
close(INF);

# code for saveing updated record

# save the updated data to the array (working)
$loaded[$linecount][0] = $PlayerNscore[0];
$loaded[$linecount][1] = $PlayerNscore[1];
$loaded[$linecount][2] = $PlayerNscore[2];
$loaded[$linecount][3] = $PlayerNscore[3];
$loaded[$linecount][4] = $PlayerNscore[4];
$loaded[$linecount][5] = $PlayerNscore[5];
$loaded[$linecount][6] = $PlayerNscore[6];

# maybe a better way (but does not work) should @loaded be referenced
another way when working with splice and unshift?

# Remove the old data from the stack
# splice @loaded, $linecount,1;

# Place new updated data at the top of the stack
# unshift @loaded, $PlayerNscore[0], $PlayerNscore[1],
$PlayerNscore[2], $PlayerNscore[3], $PlayerNscore[4], $PlayerNscore[5],
$PlayerNscore[6];


# code for saveing a new record.

# save the new player to the array (working)
$addthisline = (@loaded + "1");
$loaded[$addthisline][0] = $player1[0];
$loaded[$addthisline][1] = $player1[1];
$loaded[$addthisline][2] = "1";
$loaded[$addthisline][3] = $Playerscore;
$loaded[$addthisline][4] = $add this1;
$loaded[$addthisline][5] = $add this2;
$loaded[$addthisline][6] = $add this3;

# maybe a better way (but does not work) should @loaded be referenced
another way when working with slice and unshift? same problem as above.

# Place new data at the top of the stack
# unshift @loaded, $player1[0], $player1[1], "1", $Playerscore, $add
this1, $add this2, $add this3;


I am new to working with arrays of arrays and have scoured my books for
answers and have managed to get this far, but I'm stuck stareing at a stone
wall.

Any help appriciated.

Bill

Re: Working with arrays of arrays

Bill wrote:
> ok guys I have working code finally that does what I want it to do. It reads
> in the data file to an array of arrays. It works with it and even adds data
> just fine to it but always to the end of the top level array. The code sped
> up the program just as I expected it would, but I already found a possible
> speed improvement.
>
> Right now all new data records are placed at the end of the top level array.
> By record I mean a new array is being added to the array of arrays. However,
> it is very likely that this record will be needed almost imeadiately, but it
> is now at the end of the data array. This causes the program to parce over
> the whole array to fined it again. It would be more efficient to put that
> new record as well as any recently updated records at the top of the array
> so it is likely to be one of the first found.

Sounds like it would be more efficient still to just use a hash of
records, rather than an array, since hash lookups are O(1), and there
is no need to iterate over anything to find what you need...

> In other words over time records used the most often will rise to the top of
> the data stack while dead wood will sink to the bottom.
>
> Working code snips.
>
> # put the player data file into an array of arrays to work with it from
> system memory
> open(INF, $playerfile) or print "Error: File Load Failed";

You should always include the $! variable in these error messages, to
tell the user *why* the open failed.

> while () {
> chomp;
> push @loaded, [ split ];
> }
> close(INF);
>
> # code for saveing updated record
>
> # save the updated data to the array (working)
> $loaded[$linecount][0] = $PlayerNscore[0];
> $loaded[$linecount][1] = $PlayerNscore[1];
> $loaded[$linecount][2] = $PlayerNscore[2];
> $loaded[$linecount][3] = $PlayerNscore[3];
> $loaded[$linecount][4] = $PlayerNscore[4];
> $loaded[$linecount][5] = $PlayerNscore[5];
> $loaded[$linecount][6] = $PlayerNscore[6];

Why are you going through each and every element individually just to
copy over the array? Just copy the array:
@{$loaded[$linecount]} = @PlayerNscore;

or, if you actually want only the first 7 elements copied:
@{$loaded[$linecount]}[0..6] = @PlayerNscore[0..6];

> # maybe a better way (but does not work)

"Does not work" is the worst of all possible error descriptions, and
should be a red flag the instant you type it. *How* does it not work?
What does it do that you don't want, or what doesn't it do that you do
want?

> should @loaded be referenced
> another way when working with splice and unshift?
>
> # Remove the old data from the stack
> # splice @loaded, $linecount,1;
>
> # Place new updated data at the top of the stack
> # unshift @loaded, $PlayerNscore[0], $PlayerNscore[1],
> $PlayerNscore[2], $PlayerNscore[3], $PlayerNscore[4], $PlayerNscore[5],
> $PlayerNscore[6];

Your problem is not in how you refer to @loaded, but in how you refer
to the elements you're adding to @loaded. @loaded is an array of array
references. Yet this code is attempting to add 7 individual elements
to the array. You should be trying to add just a single array
reference that contains those seven elements. Off the top of my head,
three different ways of doing that:

#please don't do this -- just showing you the immediate solution most
like your code:
unshift @loaded, [ $PlayerNscore[0], $PlayerNscore[1],
$PlayerNscore[2],
$PlayerNscore[3], $PlayerNscore[4],
$PlayerNscore[5],
$PlayerNscore[6]];

#better:
unshift @loaded, [ @PlayerNscore ];

#best (?):
unshift @loaded, \@PlayerNscore;

The "better" solution creates a reference to an anonymous array whose
values are duplicates of the @PlayerNscore array. If you have properly
scoped @PlayerNscore, however (so that it falls out of scope as soon as
it's no longer needed), you can save that extra copying and just add a
reference to @PlayerNscore itself (the "best" solution).

> # code for saveing a new record.
>
> # save the new player to the array (working)
> $addthisline = (@loaded + "1");
> $loaded[$addthisline][0] = $player1[0];
> $loaded[$addthisline][1] = $player1[1];
> $loaded[$addthisline][2] = "1";
> $loaded[$addthisline][3] = $Playerscore;
> $loaded[$addthisline][4] = $add this1;
> $loaded[$addthisline][5] = $add this2;
> $loaded[$addthisline][6] = $add this3;

Ew.

First of all, you're implicitly creating an undefined element in
@loaded. Say there are currently 3 elements. They are indexed at
positions 0, 1 and 2. That means that $addthisline gets set to 4 (the
size of @loaded plus 1). Then you add elements to the position
$loaded[4]. You've completedly skipped over $loaded[3], which Perl
handles by autovivifying an undefined element in its place.

Secondly, This is what the "push" function is for

push @loaded, [$player1[0], $player1[1], 1, $Playerscore, $add_this1,
$add_this2, $add_this3];

> # maybe a better way (but does not work) should @loaded be referenced
> another way when working with slice and unshift? same problem as above.
>
> # Place new data at the top of the stack
> # unshift @loaded, $player1[0], $player1[1], "1", $Playerscore, $add
> this1, $add this2, $add this3;

Okay, if you now want to add it to the top of the stack rather than the
bottom, then just change my line above from 'push' to 'unshift'. This
problem is the same one as in your previous snippet - you need to add a
reference to an array which contains the values, rather than adding the
values themselves.

> I am new to working with arrays of arrays and have scoured my books for
> answers and have managed to get this far, but I'm stuck stareing at a stone
> wall.

I still recommend abandoning this structure in favor of a hash of
arrays instead, as you've not shown any code which requires these
records to be in any particular order.

You would probably also do well to read some of:
perldoc perlref
perldoc perlreftut
perldoc perllol
perldoc perldsc

Paul Lalli

Re: Working with arrays of arrays

Paul,

First of all thanks a bunch for all the food for thought.

I am learning this new skill set and everything you said made 100% sense to
me now that someone pointed out the errors of my ways.

A hash of arrays certainly sounds like the logical solution. The current
first cell "username" easily could be the hash key.

However since I am trying to learn I will still make my version 2 of the
module work as best I can. Version 1 was full of a ton of file read writes.

I will start on version 3 imeadiately though. It will be based on a hash of
arrays.

I am 100% self taught and have been hacking canned scripts for 9 years now.
This is the most complex project I have ever undertaken. I have 5 books on
Perl but they are all at least 3-4 years old. I should buy some new
reference matierial.

Hey, thanks again for all the great pointers in the right direction.

Bill


"Paul Lalli" wrote in message
news:1131472451.077885.89980@g14g2000cwa.googlegroups.com...
> Bill wrote:
> > ok guys I have working code finally that does what I want it to do. It
reads
> > in the data file to an array of arrays. It works with it and even adds
data
> > just fine to it but always to the end of the top level array. The code
sped
> > up the program just as I expected it would, but I already found a
possible
> > speed improvement.
> >
> > Right now all new data records are placed at the end of the top level
array.
> > By record I mean a new array is being added to the array of arrays.
However,
> > it is very likely that this record will be needed almost imeadiately,
but it
> > is now at the end of the data array. This causes the program to parce
over
> > the whole array to fined it again. It would be more efficient to put
that
> > new record as well as any recently updated records at the top of the
array
> > so it is likely to be one of the first found.
>
> Sounds like it would be more efficient still to just use a hash of
> records, rather than an array, since hash lookups are O(1), and there
> is no need to iterate over anything to find what you need...
>
> > In other words over time records used the most often will rise to the
top of
> > the data stack while dead wood will sink to the bottom.
> >
> > Working code snips.
> >
> > # put the player data file into an array of arrays to work with it
from
> > system memory
> > open(INF, $playerfile) or print "Error: File Load Failed";
>
> You should always include the $! variable in these error messages, to
> tell the user *why* the open failed.
>
> > while () {
> > chomp;
> > push @loaded, [ split ];
> > }
> > close(INF);
> >
> > # code for saveing updated record
> >
> > # save the updated data to the array (working)
> > $loaded[$linecount][0] = $PlayerNscore[0];
> > $loaded[$linecount][1] = $PlayerNscore[1];
> > $loaded[$linecount][2] = $PlayerNscore[2];
> > $loaded[$linecount][3] = $PlayerNscore[3];
> > $loaded[$linecount][4] = $PlayerNscore[4];
> > $loaded[$linecount][5] = $PlayerNscore[5];
> > $loaded[$linecount][6] = $PlayerNscore[6];
>
> Why are you going through each and every element individually just to
> copy over the array? Just copy the array:
> @{$loaded[$linecount]} = @PlayerNscore;
>
> or, if you actually want only the first 7 elements copied:
> @{$loaded[$linecount]}[0..6] = @PlayerNscore[0..6];
>
> > # maybe a better way (but does not work)
>
> "Does not work" is the worst of all possible error descriptions, and
> should be a red flag the instant you type it. *How* does it not work?
> What does it do that you don't want, or what doesn't it do that you do
> want?
>
> > should @loaded be referenced
> > another way when working with splice and unshift?
> >
> > # Remove the old data from the stack
> > # splice @loaded, $linecount,1;
> >
> > # Place new updated data at the top of the stack
> > # unshift @loaded, $PlayerNscore[0], $PlayerNscore[1],
> > $PlayerNscore[2], $PlayerNscore[3], $PlayerNscore[4], $PlayerNscore[5],
> > $PlayerNscore[6];
>
> Your problem is not in how you refer to @loaded, but in how you refer
> to the elements you're adding to @loaded. @loaded is an array of array
> references. Yet this code is attempting to add 7 individual elements
> to the array. You should be trying to add just a single array
> reference that contains those seven elements. Off the top of my head,
> three different ways of doing that:
>
> #please don't do this -- just showing you the immediate solution most
> like your code:
> unshift @loaded, [ $PlayerNscore[0], $PlayerNscore[1],
> $PlayerNscore[2],
> $PlayerNscore[3], $PlayerNscore[4],
> $PlayerNscore[5],
> $PlayerNscore[6]];
>
> #better:
> unshift @loaded, [ @PlayerNscore ];
>
> #best (?):
> unshift @loaded, \@PlayerNscore;
>
> The "better" solution creates a reference to an anonymous array whose
> values are duplicates of the @PlayerNscore array. If you have properly
> scoped @PlayerNscore, however (so that it falls out of scope as soon as
> it's no longer needed), you can save that extra copying and just add a
> reference to @PlayerNscore itself (the "best" solution).
>
> > # code for saveing a new record.
> >
> > # save the new player to the array (working)
> > $addthisline = (@loaded + "1");
> > $loaded[$addthisline][0] = $player1[0];
> > $loaded[$addthisline][1] = $player1[1];
> > $loaded[$addthisline][2] = "1";
> > $loaded[$addthisline][3] = $Playerscore;
> > $loaded[$addthisline][4] = $add this1;
> > $loaded[$addthisline][5] = $add this2;
> > $loaded[$addthisline][6] = $add this3;
>
> Ew.
>
> First of all, you're implicitly creating an undefined element in
> @loaded. Say there are currently 3 elements. They are indexed at
> positions 0, 1 and 2. That means that $addthisline gets set to 4 (the
> size of @loaded plus 1). Then you add elements to the position
> $loaded[4]. You've completedly skipped over $loaded[3], which Perl
> handles by autovivifying an undefined element in its place.
>
> Secondly, This is what the "push" function is for
>
> push @loaded, [$player1[0], $player1[1], 1, $Playerscore, $add_this1,
> $add_this2, $add_this3];
>
> > # maybe a better way (but does not work) should @loaded be
referenced
> > another way when working with slice and unshift? same problem as above.
> >
> > # Place new data at the top of the stack
> > # unshift @loaded, $player1[0], $player1[1], "1", $Playerscore, $add
> > this1, $add this2, $add this3;
>
> Okay, if you now want to add it to the top of the stack rather than the
> bottom, then just change my line above from 'push' to 'unshift'. This
> problem is the same one as in your previous snippet - you need to add a
> reference to an array which contains the values, rather than adding the
> values themselves.
>
> > I am new to working with arrays of arrays and have scoured my books for
> > answers and have managed to get this far, but I'm stuck stareing at a
stone
> > wall.
>
> I still recommend abandoning this structure in favor of a hash of
> arrays instead, as you've not shown any code which requires these
> records to be in any particular order.
>
> You would probably also do well to read some of:
> perldoc perlref
> perldoc perlreftut
> perldoc perllol
> perldoc perldsc
>
> Paul Lalli
>