POSIX shell: no arrays? no RANDOM?

I'm trying to write some shell code that will run on any POSIX-
conformant shell. I'm looking at the IEEE 1003.1 POSIX standard,
which is http://www.unix.org/single_unix_specification/

POSIX doesn't mention arrays. Are they a vendor-provided extension?
If so they appear to be a universal one, albeit with non-universal
limits. I even downloaded the POSIX html and grepped through the
utilities seciton. The only mentions of "array" I could find were in
section 1.9 and they referred to bc. Nothing in 2.5.3 ("Shell
Variables"). I was under the impression that POSIX is a superset of
ksh88, but apparently not, since ksh88 features arrays.

I was hoping to find the limit for the number of elements in an
array...instead I seem to have discovered that arrays don't exist in
the POSIX shell.

The old ksh88 limit was 1024 elements. The limits as I've discovered
them are:

bash: unlimited (presumably 2^31 or 2^63 or whatever the addressable
size is)
Solaris /usr/xpg4/bin/sh: 4096
HP-UX 11.23 /usr/bin/sh: 1024 (/usr/bin/sh is really ksh88, according
to the man page)
ksh93: unlimited (presumably 2^31 or 2^63 or whatever the addressable
size is)

I could live with 4096, but HP-UX's allegedly POSIX-confirming /usr/
bin/sh doesn't take it. I could go to HP and yell at them, but I
can't find anything in the POSIX spec that says" arrays shall have at
least X elements", so...they're within their rights?

So therefore, any code using arrays can't claim to be POSIX-conforming
(or written for the "POSIX shell"), since another POSIX-compliant
shell might not implement arrays...?

RANDOM also appears to be a universal but non-standard extension. No
mention in the spec, however, bash, Solaris, HP-UX, and ksh88/93 all
implement it...?

It seems the only safe thing to do is either

(a) write code that requires a specific shell and version (e.g., ksh93
or bash version X), or
(b) write code that doesn't use arrays or RANDOM, which would be
impractical

....or perhaps someone here will say "you idiot, can't you read, look
at section X that you overlooked".

Re: POSIX shell: no arrays? no RANDOM?

2007-09-12, 22:02(-07), Andrew Fabbro:
> I'm trying to write some shell code that will run on any POSIX-
> conformant shell. I'm looking at the IEEE 1003.1 POSIX standard,
> which is http://www.unix.org/single_unix_specification/
[...]

No array, no RANDOM in POSIX shells. If you need arrays, chances
that you're after some programming language, not a shell.

For both random and arrays, you have awk.

You can't do
a[1]=foo b[foo]=bar
but you can do a_1=foo b_foo=bar
so you can implement arrays with functions like

aset() {
eval "$1_$2=\$3"
}
aget() {
eval "REPLY=\${$1_$2}"
}

aset a 1 foo # for a[1]=foo

ksh, bash and zsh are the only POSIX shells that support arrays
and they all implement them differently (though ksh's and bash's
are quite similar). zsh and some versions of ksh are the only
shells to support associative arrays, zsh being the only one
that support anything for the key, and having no limit.

--
Stéphane

Re: POSIX shell: no arrays? no RANDOM?

On 13 Sep., 07:02, Andrew Fabbro wrote:
> I'm trying to write some shell code that will run on any POSIX-
> conformant shell. I'm looking at the IEEE 1003.1 POSIX standard,
> which ishttp://www.unix.org/single_unix_specification/
>
> POSIX doesn't mention arrays. Are they a vendor-provided extension?
> If so they appear to be a universal one, albeit with non-universal
> limits. I even downloaded the POSIX html and grepped through the
> utilities seciton. The only mentions of "array" I could find were in
> section 1.9 and they referred to bc. Nothing in 2.5.3 ("Shell
> Variables"). I was under the impression that POSIX is a superset of
> ksh88, but apparently not, since ksh88 features arrays.

POSIX shells are (or claim to be) supersets WRT the POSIX standard.

> I was hoping to find the limit for the number of elements in an
> array...instead I seem to have discovered that arrays don't exist in
> the POSIX shell.

You mean, "don't exist in every existing POSIX shell", or, "don't
exist in the POSIX standard"??

> The old ksh88 limit was 1024 elements. The limits as I've discovered
> them are:
>
> bash: unlimited (presumably 2^31 or 2^63 or whatever the addressable
> size is)
> Solaris /usr/xpg4/bin/sh: 4096
> HP-UX 11.23 /usr/bin/sh: 1024 (/usr/bin/sh is really ksh88, according
> to the man page)
> ksh93: unlimited (presumably 2^31 or 2^63 or whatever the addressable
> size is)
>
> I could live with 4096, but HP-UX's allegedly POSIX-confirming /usr/
> bin/sh doesn't take it. I could go to HP and yell at them, but I
> can't find anything in the POSIX spec that says" arrays shall have at
> least X elements", so...they're within their rights?
>
> So therefore, any code using arrays can't claim to be POSIX-conforming

If you want to claim that a piece of shell code is running on all
POSIX conformant shells you have to restrict to the subset of features
defined by POSIX.

Limits on maximum array sizes are not even constant withing a specific
shell. AFAIR, you can only rely on ~1000 on ksh88 while on ksh93 you
have 4096 guaranteed (but the limit can be even higher depending on
the version).

> (or written for the "POSIX shell"), since another POSIX-compliant
> shell might not implement arrays...?

There's no *the* "POSIX shell". There's the POSIX standard and a
couple of shells that support the features defined by the standard.

> RANDOM also appears to be a universal but non-standard extension. No
> mention in the spec, however, bash, Solaris, HP-UX, and ksh88/93 all
> implement it...?
>
> It seems the only safe thing to do is either
>
> (a) write code that requires a specific shell and version (e.g., ksh93
> or bash version X), or
> (b) write code that doesn't use arrays or RANDOM, which would be
> impractical

Depends on what you want. What do you mean by safe? Run the same
program on any Unix system? On any "modern" Unix system?

Personally I don't want to restrict myself if the extensions to POSIX
are generally available in all the prominent shells (ksh, bash, zsh).
OTOH, if I don't need any of the "proprietary" extension of a specific
shell (or if I need some extended portability) I might write the code
the POSIX way.

POSIX "gives you" portability, not features.

If you need arrays you may have a look how you can use the "positional
parameters" for that purpose. (After all, even ksh88 arrays are not
very interseting to work with if you want arrays as provided by most
conventional programming language.)

For random numbers (or some type of data processing in arrays) you may
have a look at awk; use awk programs/utilities in your shell code.

> ...or perhaps someone here will say "you idiot, can't you read, look
> at section X that you overlooked".

No. It's only to realize that POSIX does not provide any/many
state-of-the-art features, and, e.g., 20 year old ksh88 supports
features not present in POSIX.

Janis