multidimensional array or hashes?

Which data type would be better for doing something like:

I have a file:

ID, TYPE
ID2,TYPE2
....

where there will be roughly ~6 TYPES but many hundreds or thousands of
IDs. I *think* I want to build a multidim array, but I want to
reference them by their respective TYPE names. ie.
@TYPE and @TYPE2 which starts me thinking on hashes and using the type
as the value and ID as the keys...

Ultimately I'm trying to use SQL:abstract to build a giant where clause
like:
(TYPE in (@TYPE) or TYPE2 in (@TYPE2)...) etc.

thanks for any ideas...

Re: multidimensional array or hashes?

ok... I decided on a hash of arrays.

TICKER: A AA AAP AAPL
CUSIP: 12345678 87654321
etc

Can someone help me do this:

my %where = (
date => '20061027',
eval {foreach $IDTYPE (keys %HoA ) {
$IDTYPE => { -in => @{ $HoA{$IDTYPE} }}};
}

correctly? Since I don't know how many keys there will be... the eval
fails, but I'm not clear why.

thanks


denap wrote:
> Which data type would be better for doing something like:
>
> I have a file:
>
> ID, TYPE
> ID2,TYPE2
> ...
>
> where there will be roughly ~6 TYPES but many hundreds or thousands of
> IDs. I *think* I want to build a multidim array, but I want to
> reference them by their respective TYPE names. ie.
> @TYPE and @TYPE2 which starts me thinking on hashes and using the type
> as the value and ID as the keys...
>
> Ultimately I'm trying to use SQL:abstract to build a giant where clause
> like:
> (TYPE in (@TYPE) or TYPE2 in (@TYPE2)...) etc.
>
> thanks for any ideas...

Re: multidimensional array or hashes?

On 10/31/2006 09:50 AM, denap wrote:
> Which data type would be better for doing something like:
>
> I have a file:
>
> ID, TYPE
> ID2,TYPE2
> ....
>
> where there will be roughly ~6 TYPES but many hundreds or thousands of
> IDs. I *think* I want to build a multidim array, but I want to
> reference them by their respective TYPE names. ie.
> @TYPE and @TYPE2 which starts me thinking on hashes and using the type
> as the value and ID as the keys...
>
> Ultimately I'm trying to use SQL:abstract to build a giant where clause
> like:
> (TYPE in (@TYPE) or TYPE2 in (@TYPE2)...) etc.
>
> thanks for any ideas...
>

I think a hash with the keys as types and the values as ids would be
best. Could you write a small program that demonstrates what you're
trying to do?


--
paduille.4060.mumia.w@earthlink.net

Re: multidimensional array or hashes?

Mumia W. (reading news) wrote:
> On 10/31/2006 09:50 AM, denap wrote:
> > Which data type would be better for doing something like:
> >
> > I have a file:
> >
> > ID, TYPE
> > ID2,TYPE2
> > ....
> >
> > where there will be roughly ~6 TYPES but many hundreds or thousands of
> > IDs. I *think* I want to build a multidim array, but I want to
> > reference them by their respective TYPE names. ie.
> > @TYPE and @TYPE2 which starts me thinking on hashes and using the type
> > as the value and ID as the keys...
> >
> > Ultimately I'm trying to use SQL:abstract to build a giant where clause
> > like:
> > (TYPE in (@TYPE) or TYPE2 in (@TYPE2)...) etc.
> >
> > thanks for any ideas...
> >
>
> I think a hash with the keys as types and the values as ids would be
> best. Could you write a small program that demonstrates what you're
> trying to do?
>
>
> --
> paduille.4060.mumia.w@earthlink.net

Sure... see below... my issue is the contents of the hash %where. In
the loop where I am attempting to assign TYPE => array of IDs, I am not
understanding how to confirm (see) the contents of the arrays. ie.
Dumper shows nothing but the initialization of the 'date' key.

many thanks

use Data::Dumper::Simple;
use SQL::Abstract;
use DBI;

%where = (
date => '20061027'
);
my $dsn = 'DBI:ODBC:BARNEY';
my $user = 'reports';
my $password = 'reports';

my @result;

# Generate a HASH OF ARRAYS
# reading from file: ID,TYPE
# forming a hash: ticker: a b c d
while ( <> ) {
# grab all lines that are formed correctly, case-insensitive: ID,TYPE
next unless s/^(.*?),(?i:(CUSIP|SEDOL|ISIN|TICKER))//;
push @{$HoA{uc $2}}, $1;
}
print "\nFile loaded...\n\n";

# stuff all of the IDTYPES and IDs into the hash to be used to create
the
# where clause
#
foreach $IDTYPE (keys %HoA ) {
# print "$IDTYPE: @{$HoA{$IDTYPE}} \n";
$where{$IDTYPE} => { -in => [@{$HoA{$IDTYPE} }] };
}
warn Dumper(%where);

Re: multidimensional array or hashes?

This is a multi-part message in MIME format.
--------------030900030809050105010305
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

On 10/31/2006 02:43 PM, denap wrote:
> Mumia W. (reading news) wrote:
>> On 10/31/2006 09:50 AM, denap wrote:
>>> Which data type would be better for doing something like:
>>>
>>> I have a file:
>>>
>>> ID, TYPE
>>> ID2,TYPE2
>>> ....
>>>
>>> where there will be roughly ~6 TYPES but many hundreds or thousands of
>>> IDs. I *think* I want to build a multidim array, but I want to
>>> reference them by their respective TYPE names. ie.
>>> @TYPE and @TYPE2 which starts me thinking on hashes and using the type
>>> as the value and ID as the keys...
>>>
>>> Ultimately I'm trying to use SQL:abstract to build a giant where clause
>>> like:
>>> (TYPE in (@TYPE) or TYPE2 in (@TYPE2)...) etc.
>>>
>>> thanks for any ideas...
>>>
>> I think a hash with the keys as types and the values as ids would be
>> best. Could you write a small program that demonstrates what you're
>> trying to do?
>>
>>
>> --
>> paduille.4060.mumia.w@earthlink.net
>
> Sure... see below... my issue is the contents of the hash %where. In
> the loop where I am attempting to assign TYPE => array of IDs, I am not
> understanding how to confirm (see) the contents of the arrays. ie.
> Dumper shows nothing but the initialization of the 'date' key.
>
> many thanks
>
> use Data::Dumper::Simple;
> use SQL::Abstract;
> use DBI;
>
> %where = (
> date => '20061027'
> );
> my $dsn = 'DBI:ODBC:BARNEY';
> my $user = 'reports';
> my $password = 'reports';
>
> my @result;
>
> # Generate a HASH OF ARRAYS
> # reading from file: ID,TYPE
> # forming a hash: ticker: a b c d
> while ( <> ) {
> # grab all lines that are formed correctly, case-insensitive: ID,TYPE
> next unless s/^(.*?),(?i:(CUSIP|SEDOL|ISIN|TICKER))//;

Why the substitution?

> push @{$HoA{uc $2}}, $1;
> }
> print "\nFile loaded...\n\n";
>
> # stuff all of the IDTYPES and IDs into the hash to be used to create
> the
> # where clause
> #
> foreach $IDTYPE (keys %HoA ) {
> # print "$IDTYPE: @{$HoA{$IDTYPE}} \n";
> $where{$IDTYPE} => { -in => [@{$HoA{$IDTYPE} }] };
> }
> warn Dumper(%where);
>

I don't know anything about SQL::Abstract, so I might consider doing
something like this:

use strict;
use warnings;
@ARGV = 'types-and-ids.txt';
my %hash;

while (<>) {
if (my ($id, $type) = m/(\d+), *(cwdb|purify|gdb|gprof|sundbg|tdb)/i) {
push @{ $hash{$type} }, $id;
}
}

while (my ($key, $value) = each %hash) {
local $" = ", ";
print "where $key in (@$value)\n";
}



__HTH__

--
paduille.4060.mumia.w@earthlink.net



--------------030900030809050105010305
Content-Type: text/plain;
name="types-and-ids.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="types-and-ids.txt"

1, cwdb
2, purify
3, gprof
4, purify
5, gprof
6, purify
7, cwdb
8, gprof
9, sundbg
10, tdb
11, tdb
12, purify
13, gprof
14, sundbg
15, tdb
16, purify
17, sundbg
18, cwdb
19, gdb
20, gdb
21, cwdb
22, purify
23, sundbg
24, gprof
25, gdb
26, purify
27, gdb
28, purify
29, purify
30, gdb
31, purify
32, purify
33, cwdb
35, gprof
36, gdb
37, sundbg
38, cwdb
39, purify
40, sundbg
41, purify
42, purify
43, tdb
44, cwdb
45, cwdb
46, tdb
47, tdb
48, gprof
49, gdb
50, gprof
51, cwdb
52, gprof
53, purify
54, gprof
55, cwdb
56, cwdb
57, gdb
58, sundbg
59, purify
60, purify
61, cwdb
62, tdb
63, gprof
64, gdb
65, cwdb
66, purify
67, tdb
68, cwdb
69, cwdb
70, tdb
71, sundbg
72, gprof
73, gprof
74, tdb
75, gdb
76, sundbg
77, gprof
78, gdb
79, sundbg
80, tdb
81, purify
82, purify
83, cwdb
84, cwdb
85, gprof
86, tdb
87, sundbg
88, sundbg
89, tdb
90, gdb
91, cwdb
92, gdb
93, gdb
94, sundbg
95, cwdb
96, gdb
97, purify
98, gprof
99, sundbg
100, tdb
101, cwdb
102, cwdb
103, gdb
104, tdb
105, gprof
106, tdb
107, gprof
108, tdb
109, purify
110, cwdb
111, gdb
112, tdb
113, tdb
114, gprof
115, gprof
116, cwdb
117, sundbg
118, gdb
119, cwdb
120, tdb
121, gdb



--------------030900030809050105010305--

Re: multidimensional array or hashes?

Mumia W. (reading news) wrote:
> On 10/31/2006 02:43 PM, denap wrote:
> > Mumia W. (reading news) wrote:
> >> On 10/31/2006 09:50 AM, denap wrote:
> >>> Which data type would be better for doing something like:
> >>>
> >>> I have a file:
> >>>
> >>> ID, TYPE
> >>> ID2,TYPE2
> >>> ....
> >>>
> >>> where there will be roughly ~6 TYPES but many hundreds or thousands of
> >>> IDs. I *think* I want to build a multidim array, but I want to
> >>> reference them by their respective TYPE names. ie.
> >>> @TYPE and @TYPE2 which starts me thinking on hashes and using the type
> >>> as the value and ID as the keys...
> >>>
> >>> Ultimately I'm trying to use SQL:abstract to build a giant where clause
> >>> like:
> >>> (TYPE in (@TYPE) or TYPE2 in (@TYPE2)...) etc.
> >>>
> >>> thanks for any ideas...
> >>>
> >> I think a hash with the keys as types and the values as ids would be
> >> best. Could you write a small program that demonstrates what you're
> >> trying to do?
> >>
> >>
> >> --
> >> paduille.4060.mumia.w@earthlink.net
> >
> > Sure... see below... my issue is the contents of the hash %where. In
> > the loop where I am attempting to assign TYPE => array of IDs, I am not
> > understanding how to confirm (see) the contents of the arrays. ie.
> > Dumper shows nothing but the initialization of the 'date' key.
> >
> > many thanks
> >
> > use Data::Dumper::Simple;
> > use SQL::Abstract;
> > use DBI;
> >
> > %where = (
> > date => '20061027'
> > );
> > my $dsn = 'DBI:ODBC:BARNEY';
> > my $user = 'reports';
> > my $password = 'reports';
> >
> > my @result;
> >
> > # Generate a HASH OF ARRAYS
> > # reading from file: ID,TYPE
> > # forming a hash: ticker: a b c d
> > while ( <> ) {
> > # grab all lines that are formed correctly, case-insensitive: ID,TYPE
> > next unless s/^(.*?),(?i:(CUSIP|SEDOL|ISIN|TICKER))//;
>
> Why the substitution?
>
> > push @{$HoA{uc $2}}, $1;
> > }
> > print "\nFile loaded...\n\n";
> >
> > # stuff all of the IDTYPES and IDs into the hash to be used to create
> > the
> > # where clause
> > #
> > foreach $IDTYPE (keys %HoA ) {
> > # print "$IDTYPE: @{$HoA{$IDTYPE}} \n";
> > $where{$IDTYPE} => { -in => [@{$HoA{$IDTYPE} }] };
> > }
> > warn Dumper(%where);
> >
>
> I don't know anything about SQL::Abstract, so I might consider doing
> something like this:
>
> use strict;
> use warnings;
> @ARGV = 'types-and-ids.txt';
> my %hash;
>
> while (<>) {
> if (my ($id, $type) = m/(\d+), *(cwdb|purify|gdb|gprof|sundbg|tdb)/i) {
> push @{ $hash{$type} }, $id;
> }
> }
>
> while (my ($key, $value) = each %hash) {
> local $" = ", ";
> print "where $key in (@$value)\n";
> }
>
>
>
> __HTH__
>
> --
> paduille.4060.mumia.w@earthlink.net
>
>
>
> --------------030900030809050105010305
> Content-Type: text/plain
> Content-Disposition: inline;
> filename="types-and-ids.txt"
> X-Google-AttachSize: 1151
>
> 1, cwdb
> 2, purify
> 3, gprof
> 4, purify
> 5, gprof
> 6, purify
> 7, cwdb
> 8, gprof
> 9, sundbg
> 10, tdb
> 11, tdb
> 12, purify
> 13, gprof
> 14, sundbg
> 15, tdb
> 16, purify
> 17, sundbg
> 18, cwdb
> 19, gdb
> 20, gdb
> 21, cwdb
> 22, purify
> 23, sundbg
> 24, gprof
> 25, gdb
> 26, purify
> 27, gdb
> 28, purify
> 29, purify
> 30, gdb
> 31, purify
> 32, purify
> 33, cwdb
> 35, gprof
> 36, gdb
> 37, sundbg
> 38, cwdb
> 39, purify
> 40, sundbg
> 41, purify
> 42, purify
> 43, tdb
> 44, cwdb
> 45, cwdb
> 46, tdb
> 47, tdb
> 48, gprof
> 49, gdb
> 50, gprof
> 51, cwdb
> 52, gprof
> 53, purify
> 54, gprof
> 55, cwdb
> 56, cwdb
> 57, gdb
> 58, sundbg
> 59, purify
> 60, purify
> 61, cwdb
> 62, tdb
> 63, gprof
> 64, gdb
> 65, cwdb
> 66, purify
> 67, tdb
> 68, cwdb
> 69, cwdb
> 70, tdb
> 71, sundbg
> 72, gprof
> 73, gprof
> 74, tdb
> 75, gdb
> 76, sundbg
> 77, gprof
> 78, gdb
> 79, sundbg
> 80, tdb
> 81, purify
> 82, purify
> 83, cwdb
> 84, cwdb
> 85, gprof
> 86, tdb
> 87, sundbg
> 88, sundbg
> 89, tdb
> 90, gdb
> 91, cwdb
> 92, gdb
> 93, gdb
> 94, sundbg
> 95, cwdb
> 96, gdb
> 97, purify
> 98, gprof
> 99, sundbg
> 100, tdb
> 101, cwdb
> 102, cwdb
> 103, gdb
> 104, tdb
> 105, gprof
> 106, tdb
> 107, gprof
> 108, tdb
> 109, purify
> 110, cwdb
> 111, gdb
> 112, tdb
> 113, tdb
> 114, gprof
> 115, gprof
> 116, cwdb
> 117, sundbg
> 118, gdb
> 119, cwdb
> 120, tdb
> 121, gdb
>
>
>
> --------------030900030809050105010305--
terrific! thanks for the help. To answer the question, "why the
substitution", good point, it's unecessary. I had come to this
conclusion in rethinking last night.

The area I'm fundamentally failing on though is my loop to assign to a
new hash %where a set of dynamically built key,value pairs. (your
print loop) I can see now that this is perhaps unecessay, but I don't
see why:
$where{$IDTYPE} => { -in =>
[@{$HoA{$IDTYPE} }] };
fails the assignment.

I'll continue figuring this out, thanks for the help!

-Tom