Announcement: bashcritic

Hi,

after using Perl::Critic for a while (see
http://search.cpan.org/~elliotjs/Perl-Critic-1.078/lib/Perl/ Critic.pm)
I had to idea to begin something similar for shell scripts. For
example to check for non-portable constructs and so on.

An initial prototype (in alpha stage) is available at:

https://trac.id.ethz.ch/projects/bashcritic/wiki/WikiStart

I began to add some tests for portability but I would really like some
feedback and some suggestions on what to check (I will in any case
monitor the group looking for possible candidates)

Thanks,

Matteo

Re: Announcement: bashcritic

2007-10-12, 08:07(-00), Teo:
> Hi,
>
> after using Perl::Critic for a while (see
> http://search.cpan.org/~elliotjs/Perl-Critic-1.078/lib/Perl/ Critic.pm)
> I had to idea to begin something similar for shell scripts. For
> example to check for non-portable constructs and so on.
>
> An initial prototype (in alpha stage) is available at:
>
> https://trac.id.ethz.ch/projects/bashcritic/wiki/WikiStart
>
> I began to add some tests for portability but I would really like some
> feedback and some suggestions on what to check (I will in any case
> monitor the group looking for possible candidates)
[...]

Some comments:

description => '"declare keyword" is not portable use
"typeset keyword" to define' . ' local variables (or
variables with special attributes).',

typeset is as non-standard as declare. typeset is ksh, declare
is bash. The only other Bourne-lie shell that supports anything
similar is zsh that supports both (plus "local").

In POSIX, you use subshells to have local variables ((...) or
$(...)).

description => 'Avoid here-strings (a special form of
the here-document)' . ' in portable scripts.',

Here strings are very recent in bash. So, on most "stable"
systems, even bash won't recognise it. The feature was borrowed
from ksh93 that borrowed it from zsh that borrowed it from rc.
It's of little use in scripts.

description => q{Though POSIX allows it, many shells
don' want the assignment} . ' and the exporting in one
command.',

No, there's only one Bourne-like shell that doesn't support
export var=value and that's the Bourne shell itself. The Bourne
shell is a non-POSIX shell from the old ages. You may want to
add a Bourne compatibility checking mode to your tool, but then,
there'll be many more things to check.

On the other hand, it should be noted that export behaves
differently depending on the shell.

For instance, in some shells (namely bash and the AT&T
implementations of ksh)
a="A B"
export b=$a
doesn't export the varibles b and B. Instead, it exports b with
as value "A B", which makes them non-POSIX conformant. So, here
even more than anywhere else, it's important to quote variable
expansions.

description => q{POSIX does't define an arithmetic
compund command, many shells} . q{don't know it. Using
the pseudo-command : and the arithmetic expansion} . q{
$(( )) is a kind of workaround here.},

However, some shells don't support assignments as in:
: $(( a = 1 + 1 ))
so I wouldn't give that advice.
a=$((1 + 1)) is prefered. $((...)) is not Bourne.

description => q{The Bashish test keyword "[[" is
reserved by POSIX, } . q{but not defined. Use the old
fashioned way with the test command} . q{ ("test" or
"[").},

As far as I know, bash didn't invent anything [[ is a kshism
also recognized by bash and zsh (with variations).

description => q{The &>FILE >&FILE redirection syntax is
short for >FILE 2>&1} . q{ and is derived from the
C-Shell. It's very old and not part of POSIX.},

No, csh's is >&, also found in zsh. It's not as old as most
features of bash that come from the Bourne shell.

description => q{`COMMANDS` is an older form of the
command substitution. } . q{The usage of the POSIX-form
$(COMMANDS) is preferred.} ,

....unless you want to be Bourne compatible ($(...) is not
Bourne).

description => q{The "function NAME { ...; }" form of
the function definition} . q{ is not recommended simply
use NAME() { ...; } to define a function.},

is a kshism. In some AT&T ksh, functions defined like that are
treated differently from the standardly defined ones. Note that
the { ...; } are not necessary. POSIX requires a compound
command. All shells but bash will allow any command.

description => q{"let MATH" is the classic form of the
arithmetic evaluation } . q{command. Bash has an own
compound command for that, which should be used} . q{ if
possible: ": $((MATH))".} ,

Again, "let" is a kshism. $((...)) is the POSIX form not bash's.
In bash, arithmetic expansion should be quoted as it's subject
to word splitting. So either

: "$((MATH))"
or use assignments
var=$((MATH))

description => 'Line is longer than 78 characters,
consider splitting it over' . " multiple lines to
increase readability.",

Why 78?

--
Stéphane

Re: Announcement: bashcritic

Stephane CHAZELAS wrote:

> Some comments:

Let me add some comments, too, and questions, because it seems that most
descriptions come from my site (corrections always welcome!).


typeset/declare:

> typeset is as non-standard as declare. typeset is ksh, declare
> is bash. The only other Bourne-lie shell that supports anything
> similar is zsh that supports both (plus "local").

In my sources it's extra marked as non-POSIX, but portable, for limited
values of "portability".


> However, some shells don't support assignments as in:
> : $(( a = 1 + 1 ))
> so I wouldn't give that advice.
> a=$((1 + 1)) is prefered. $((...)) is not Bourne.

Though it was meant as a 1:1 replacement option for the (( )) compund
command, I'll add that comment. I didn't think of assignments here :/


> description => q{The Bashish test keyword "[[" is
> reserved by POSIX, } . q{but not defined. Use the old
> fashioned way with the test command} . q{ ("test" or
> "[").},
>
> As far as I know, bash didn't invent anything [[ is a kshism
> also recognized by bash and zsh (with variations).

Of course, but it's a Bash-related site :-)
Do you know some documentation about what was derived from where for the
common shells, by the way?


> description => q{The "function NAME { ...; }" form of
> the function definition} . q{ is not recommended simply
> use NAME() { ...; } to define a function.},
>
> is a kshism. In some AT&T ksh, functions defined like that are
> treated differently from the standardly defined ones. Note that
> the { ...; } are not necessary. POSIX requires a compound
> command. All shells but bash will allow any command.

Bash also wants "only" a compound command there. In the table where that
description comes from, it was an optic decision. The syntax
descriptions are safe. My question here: Did Bash change the behaviour
over the years? I mean, did it only recognize the curly-brace-syntax in
the past?


(Sorry for mentioning Bash in all descriptions, I saw that made you a
bit upset: It's a Bash-related site. "Bash has a command for ..." isn't
intended to sound like "Bash invented the universe")


my 2ct,
J.

Re: Announcement: bashcritic

2007-10-16, 19:36(+02), Jan Schampera:
[...]
>> typeset is as non-standard as declare. typeset is ksh, declare
>> is bash. The only other Bourne-lie shell that supports anything
>> similar is zsh that supports both (plus "local").
>
> In my sources it's extra marked as non-POSIX, but portable, for limited
> values of "portability".

it's portable to ksh's and zsh, for what it's worth. It's not
specified in any standard.

f() (
local_var=1
)

is standard.

>> However, some shells don't support assignments as in:
>> : $(( a = 1 + 1 ))
>> so I wouldn't give that advice.
>> a=$((1 + 1)) is prefered. $((...)) is not Bourne.
>
> Though it was meant as a 1:1 replacement option for the (( )) compund
> command, I'll add that comment. I didn't think of assignments here :/

a 1:1 replacement would be

[ "$((...))" -ne 0 ]

quotes around $((...)) are not necessary as per POSIX but in
practive are because bash and AT&T ksh perform word splitting
upon arithmetic expansion.

>> description => q{The Bashish test keyword "[[" is
>> reserved by POSIX, } . q{but not defined. Use the old
>> fashioned way with the test command} . q{ ("test" or
>> "[").},
>>
>> As far as I know, bash didn't invent anything [[ is a kshism
>> also recognized by bash and zsh (with variations).
>
> Of course, but it's a Bash-related site :-)

Sure, but it's not a bashism, it's a kshism even in bash. That
is, it's a ksh feature implemented by bash. It's not a Bourne
feature nor a POSIX feature.

> Do you know some documentation about what was derived from where for the
> common shells, by the way?

No unfortunately.

>> description => q{The "function NAME { ...; }" form of
>> the function definition} . q{ is not recommended simply
>> use NAME() { ...; } to define a function.},
>>
>> is a kshism. In some AT&T ksh, functions defined like that are
>> treated differently from the standardly defined ones. Note that
>> the { ...; } are not necessary. POSIX requires a compound
>> command. All shells but bash will allow any command.
>
> Bash also wants "only" a compound command there. In the table where that
> description comes from, it was an optic decision. The syntax
> descriptions are safe. My question here: Did Bash change the behaviour
> over the years? I mean, did it only recognize the curly-brace-syntax in
> the past?

Yes it did. It was made to accept any compound command in
bash-2.05-alpha1 in order to be POSIX conformant. The Bourne
shell accepts any command. You can write:

f() echo "$@"

in any shell (Bourne shell, ash, ksh, pdksh, zsh...) but bash.

> (Sorry for mentioning Bash in all descriptions, I saw that made you a
> bit upset: It's a Bash-related site. "Bash has a command for ..." isn't
> intended to sound like "Bash invented the universe")
[...]

I'm not upset at all. It's a welcome initiative.

But it's true many people fail to remember that bash is an
unfinished clone of ksh with interactive featured borrowed from
tcsh.

--
Stéphane

Re: Announcement: bashcritic

Dear St=E9phane,

On Oct 15, 10:15 pm, Stephane CHAZELAS
wrote:
> 2007-10-12, 08:07(-00), Teo:> Hi,
>
> > after using Perl::Critic for a while (see
> >http://search.cpan.org/~elliotjs/Perl-Critic-1.078/lib/Perl /Critic.pm)
> > I had to idea to begin something similar for shell scripts. For
> > example to check for non-portable constructs and so on.
>
> > An initial prototype (in alpha stage) is available at:
>
> >https://trac.id.ethz.ch/projects/bashcritic/wiki/WikiStart
>
> > I began to add some tests for portability but I would really like some
> > feedback and some suggestions on what to check (I will in any case
> > monitor the group looking for possible candidates)
>
> [...]
>
> Some comments:

Many thanks for your valuable comments, I am pretty busy now but I'll
try to update bashcritic in the next days and come back here.

Many thanks again.

Matteo

Re: Announcement: bashcritic

2007-10-16, 19:36(+02), Jan Schampera:
> Stephane CHAZELAS wrote:
>
>> Some comments:
>
> Let me add some comments, too, and questions, because it seems that most
> descriptions come from my site (corrections always welcome!).
[...]

Ok, soryr I realise only now that you're not the OP and you're
referring to http://www.bash-hackers.org/. Note that I had no
knowledge of that site at the time I wrote my response to Teo, I
was quoting messages from bashcritic.

--
Stéphane

Re: Announcement: bashcritic

Stephane CHAZELAS wrote:

>>> Some comments:
>> Let me add some comments, too, and questions, because it seems that most
>> descriptions come from my site (corrections always welcome!).
> [...]
>
> Ok, soryr I realise only now that you're not the OP and you're
> referring to http://www.bash-hackers.org/. Note that I had no
> knowledge of that site at the time I wrote my response to Teo, I
> was quoting messages from bashcritic.

I didn't post because of somebody quoting messages from a specific site,
I was interested in how you clarified some contents. If it comes from my
site (or not) is not really the point in this context ;-)

Thanks for the portable arithmetic "command". I appreciate it (if I
wanted static documents, I'd put a PDF for download - heh).

J.

Re: Announcement: bashcritic

Dear St=E9phane,

first of all: thanks. You are a great source of help. I updated
bashcritic to the next alpha version (0.2.0).
I didn't have time to do much but I managed to include several
suggestions.

Some major code cleanup is still needed (as a simple example
bashcritic now complains for stuff written in comments ...)

In any case:

On Oct 15, 10:15 pm, Stephane CHAZELAS
wrote:
> Some comments:
>
> description =3D> '"declare keyword" is not portable use
> "typeset keyword" to define' . ' local variables (or
> variables with special attributes).',
>
> typeset is as non-standard as declare. typeset is ksh, declare
> is bash. The only other Bourne-lie shell that supports anything
> similar is zsh that supports both (plus "local").

I now give a warning for both.

> description =3D> 'Avoid here-strings (a special form of
> the here-document)' . ' in portable scripts.',
>
> Here strings are very recent in bash. So, on most "stable"
> systems, even bash won't recognise it. The feature was borrowed
> from ksh93 that borrowed it from zsh that borrowed it from rc.
> It's of little use in scripts.

Warn always

> description =3D> q{Though POSIX allows it, many shells
> don' want the assignment} . ' and the exporting in one
> command.',
>
> No, there's only one Bourne-like shell that doesn't support
> export var=3Dvalue and that's the Bourne shell itself. The Bourne
> shell is a non-POSIX shell from the old ages. You may want to
> add a Bourne compatibility checking mode to your tool, but then,
> there'll be many more things to check.

Corrected, you can now specify --bourne and it will give a warning. I
know that much more should be checked but let's leave it there as the
only bourne-specific check at the moment.

> On the other hand, it should be noted that export behaves
> differently depending on the shell.
>
> For instance, in some shells (namely bash and the AT&T
> implementations of ksh)
> a=3D"A B"
> export b=3D$a
> doesn't export the varibles b and B. Instead, it exports b with
> as value "A B", which makes them non-POSIX conformant. So, here
> even more than anywhere else, it's important to quote variable
> expansions.

I added a new rule suggesting to quote expansions to avoid the
problem.

> description =3D> q{POSIX does't define an arithmetic
> compund command, many shells} . q{don't know it. Using
> the pseudo-command : and the arithmetic expansion} . q{
> $(( )) is a kind of workaround here.},
>
> However, some shells don't support assignments as in:
> : $(( a =3D 1 + 1 ))
> so I wouldn't give that advice.
> a=3D$((1 + 1)) is prefered. $((...)) is not Bourne.

Removed

> description =3D> q{The Bashish test keyword "[[" is
> reserved by POSIX, } . q{but not defined. Use the old
> fashioned way with the test command} . q{ ("test" or
> "[").},
>
> As far as I know, bash didn't invent anything [[ is a kshism
> also recognized by bash and zsh (with variations).

Description adapted.

> description =3D> q{The &>FILE >&FILE redirection syntax is
> short for >FILE 2>&1} . q{ and is derived from the
> C-Shell. It's very old and not part of POSIX.},
>
> No, csh's is >&, also found in zsh. It's not as old as most
> features of bash that come from the Bourne shell.

Corrected

> description =3D> q{`COMMANDS` is an older form of the
> command substitution. } . q{The usage of the POSIX-form
> $(COMMANDS) is preferred.} ,
>
> ...unless you want to be Bourne compatible ($(...) is not
> Bourne).

I don't complain anymore for `` but I complain for $(COMMAND) if --
bourne is specified

[..]
> description =3D> 'Line is longer than 78 characters,
> consider splitting it over' . " multiple lines to
> increase readability.",
>
> Why 78?

Mmm this was just a test with a function that did something different
than a pattern match. 78 is 'my' standard. I changed to 80 (VT100) but
I don't know if I will really keep it :-)

Many thanks again, and as soon as I will find some time I'll try to
add some other checks. In any case I'm having fun and learning a lot
about shells and portability (I always only used bash ...)

Matteo

Re: Announcement: bashcritic

2007-10-17, 17:16(-07), Teo:
[...]
> Many thanks again, and as soon as I will find some time I'll try to
> add some other checks. In any case I'm having fun and learning a lot
> about shells and portability (I always only used bash ...)
[...]

Some more checks you may add:

---
cmd "$var"

should be

cmd -- "$var"
for cmds supporting it.
---
cmd -- $var
or
cmd -- $(cmd2)

unquoted var or cmdsubst means word splitting and filename
generation on the resulting words. So either it should be
quoted, or most probably filename generation should be disabled
(set -f), and IFS should be explicitely set to clarify where to
split.
---
echo

should be banned except maybe with non-variable arguments that
don't start with "-" and don't contain backslashes.

echo -n and echo -e are neither portable nor standard.
even with one same shell (especially for ksh93 and bash), echo's
behavior may vary depending on the environment.
---
check that the first argument to printf is not variable.

printf "$var"
should be
printf %s "$var"

printf "$pattern" "$@"
is probably OK, for more than one arg, it's hard to tell what
the programmer's intent was.
---
${var:x:x} is ksh93/bash specific
---
${var/../..} ${var//../..} are ksh93/zsh/bash specific
---
arrays are bash/ksh/zsh specific (and handled differently in
those shells).
---
$((...)) is POSIX but not Bourne
---
var=$*, var=$@ are not handled the same in all shells if the
first char of IFS is not " ".
var="$*" should be OK (except that the Bourn shell always joins
with space).
---
brace expansion is not Bourne ({a,b})
---
{a..b} is zsh/recent ksh93/recent bash specific (and differ
between shells)
---
tilde expansion is not Bourne
---
cd -- "$var"
should be
cd -P -- "$var" (except in the Bourne shell)
---
Most Bourne builtins don't support "--"
---
select is ksh/bash/zsh specific
and is of no use. as you can use printf and read.
---
$'...' is ksh93/bash/zsh specific (and handled differently)
use printf %b instead.
---
$"..." is ksh93/bash specific (and handled differently)
---
$PWD is POSIX but not Bourne (most shells are not POSIX in that
they don't ignore the value of the $PWD env var, you need to do
a pwd -P > /dev/null at the start of your script to fix its
value).
---
$RANDOM
$SECONDS are bash/ksh/zsh specific. Use awk's rand() for
randomness.
---
command, alias, readonly are not Bourne
---
command -v/-V are optionnal in POSIX
---
type is required by Unix but not POSIX and is not in every
Bourne shell and in every POSIX shell (see posh).
---
unset is not in every Bourne shell
---
double quotes should be escaped in double quoted backticks:
echo "`echo \"a b\"`"
---
^ should be quoted (as required by POSIX and it's special in
Bourne and zsh).
---
functions and comments were not supported in very early versions
of the Bourne shell
---
wait "$pid"
may block if the process of $pid pid had terminated before the
wait command was run in some shells (even though they shouldn't
as per latest version of POSIX).
---


.... and so on. I'll try and give more later on.


--
Stéphane

Re: Announcement: bashcritic

Dear St=E8phane,

On Oct 18, 10:06 am, Stephane CHAZELAS
wrote:
> 2007-10-17, 17:16(-07), Teo:
> [...]> Many thanks again, and as soon as I will find some time I'll try to
> > add some other checks. In any case I'm having fun and learning a lot
> > about shells and portability (I always only used bash ...)
>
> [...]
>
> Some more checks you may add:
[..]
>
> ... and so on. I'll try and give more later on.

Wow, thanks. I am really grateful. I'll try to include them as soon as
possible but it will take some time (in this period of time it seems
as new deadlines are growing out of nowhere :-)

In any case I created some tickets and a deadline:

https://trac.id.ethz.ch/projects/bashcritic/milestone/Beta

on the trac project page so that you can follow the progess (if you
are interested of course :-)

Thanks again,

Matteo

Re: Announcement: bashcritic

Stephane CHAZELAS wrote:

> a 1:1 replacement would be
>
> [ "$((...))" -ne 0 ]
>
> quotes around $((...)) are not necessary as per POSIX but in
> practive are because bash and AT&T ksh perform word splitting
> upon arithmetic expansion.

Huh? POSIX requires field splitting and pathname expansion to
be done after arithmetic expansion. See XCU 2.6 Word Expansions.

Perhaps you are being misled by the statement in 2.6.4, "The
expression shall be treated as if it were in double-quotes"
I'm fairly sure that is only supposed to affect how the expression
is treated before the substitution is done (i.e. what happens when
doing parameter expansion and command substitution on the expression
before evaluating its numeric value).

If the intention was for field splitting and pathname expansion not
to be done on the results of arithmetic expansion, then 2.6 would
be worded differently.

--
Geoff Clare

Re: Announcement: bashcritic

2007-10-18, 14:05(+01), Geoff Clare:
> Stephane CHAZELAS wrote:
>
>> a 1:1 replacement would be
>>
>> [ "$((...))" -ne 0 ]
>>
>> quotes around $((...)) are not necessary as per POSIX but in
>> practive are because bash and AT&T ksh perform word splitting
>> upon arithmetic expansion.
>
> Huh? POSIX requires field splitting and pathname expansion to
> be done after arithmetic expansion. See XCU 2.6 Word Expansions.
[...]
> If the intention was for field splitting and pathname expansion not
> to be done on the results of arithmetic expansion, then 2.6 would
> be worded differently.

Could you please quote the exact text that makes you think so?
Then, would it apply to tilde expansion as well.

AFAICS, only bash and AT&T ksh split upon arithmetic expansion.
ash, pdksh, mksh, posh, zsh don't.

And only ash (well debian ash at least) splits upon tilde
expansion, all the other ones don't.

And only bash performs filename generation upon tilde expansion,
all the other ones including ash don't.

All in all Wether your interpret the spec correctly or not the
end result is that you need to quote those expansions, as in
effect, if you don't, the behavior is not certain.

And for tilde expansion, given that quoting prevents the
expansion, you need to use an intermediary variable.

So you can't do:

cd -P -- ~/tmp

unless you do a set -f and IFS= before

You should do:

dir=~/tmp
cd -P -- "$dir"

--
Stéphane

Re: Announcement: bashcritic

Stephane CHAZELAS wrote:

> 2007-10-18, 14:05(+01), Geoff Clare:
>> Stephane CHAZELAS wrote:
>>
>>> a 1:1 replacement would be
>>>
>>> [ "$((...))" -ne 0 ]
>>>
>>> quotes around $((...)) are not necessary as per POSIX but in
>>> practive are because bash and AT&T ksh perform word splitting
>>> upon arithmetic expansion.
>>
>> Huh? POSIX requires field splitting and pathname expansion to
>> be done after arithmetic expansion. See XCU 2.6 Word Expansions.
> [...]
>> If the intention was for field splitting and pathname expansion not
>> to be done on the results of arithmetic expansion, then 2.6 would
>> be worded differently.
>
> Could you please quote the exact text that makes you think so?

"The order of word expansion shall be as follows:

1. Tilde expansion (see Tilde Expansion ), parameter expansion
(see Parameter Expansion ), command substitution (see Command
Substitution ), and arithmetic expansion (see Arithmetic
Expansion ) shall be performed, beginning to end. See item 5 in
Token Recognition.

2. Field splitting (see Field Splitting ) shall be performed on the
portions of the fields generated by step 1, unless IFS is null.

3. Pathname expansion (see Pathname Expansion ) shall be performed,
unless set -f is in effect.

4. Quote removal (see Quote Removal ) shall always be performed last."

So arithmetic expansion is done as part of item 1. Then in item 2
field splitting is done on the results of the item 2 expansions.
Then pathname expansion is performed on each field.

> Then, would it apply to tilde expansion as well.

Yes.

> AFAICS, only bash and AT&T ksh split upon arithmetic expansion.
> ash, pdksh, mksh, posh, zsh don't.
>
> And only ash (well debian ash at least) splits upon tilde
> expansion, all the other ones don't.
>
> And only bash performs filename generation upon tilde expansion,
> all the other ones including ash don't.

If ksh88 doesn't do these last two, then it is possible the requirements
relating to tilde expansion in POSIX are accidental.

> All in all Wether your interpret the spec correctly or not the
> end result is that you need to quote those expansions, as in
> effect, if you don't, the behavior is not certain.
>
> And for tilde expansion, given that quoting prevents the
> expansion, you need to use an intermediary variable.
>
> So you can't do:
>
> cd -P -- ~/tmp
>
> unless you do a set -f and IFS= before
>
> You should do:
>
> dir=~/tmp
> cd -P -- "$dir"

Personally I always use "$HOME" in scripts anyway. I only use ~ as
shorthand in interactive shells.

--
Geoff Clare

Re: Announcement: bashcritic

2007-10-19, 13:48(+01), Geoff Clare:
[...]
>> AFAICS, only bash and AT&T ksh split upon arithmetic expansion.
>> ash, pdksh, mksh, posh, zsh don't.
>>
>> And only ash (well debian ash at least) splits upon tilde
>> expansion, all the other ones don't.
>>
>> And only bash performs filename generation upon tilde expansion,
>> all the other ones including ash don't.
>
> If ksh88 doesn't do these last two, then it is possible the requirements
> relating to tilde expansion in POSIX are accidental.
[...]

I did more test on a Solaris 7 machine. ksh88i there does
perform word splitting and filename generation upon tilde
and arithmetic expansions. dtksh (ksh93d) performs only filename
generation (not word splitting).

>> So you can't do:
>>
>> cd -P -- ~/tmp
>>
>> unless you do a set -f and IFS= before
>>
>> You should do:
>>
>> dir=~/tmp
>> cd -P -- "$dir"
>
> Personally I always use "$HOME" in scripts anyway. I only use ~ as
> shorthand in interactive shells.

That's OK for ~, but there's no such equivalent for ~user, given
that (AFAIK) POSIX doesn't provide with a standard command to
retrieve the home directory of a given user (like GNU/Solaris
getent).

--
Stéphane

Re: Announcement: bashcritic

Geoff Clare wrote:

> So arithmetic expansion is done as part of item 1. Then in item 2
> field splitting is done on the results of the item 2 expansions.

Silly typo. Obviously the second "item 2" should have been "item 1".

--
Geoff Clare

bashcritic 0.3.0 (Was: Announcement: bashcritic)

Hi,


On Oct 18, 1:55 pm, Teo wrote:
> > Some more checks you may add:
> [...]
>
> > ... and so on. I'll try and give more later on.
>
> Wow, thanks. I am really grateful. I'll try to include them as soon as
> possible but it will take some time (in this period of time it seems
> as new deadlines are growing out of nowhere :-)
>
> In any case I created some tickets and a deadline:
>
> https://trac.id.ethz.ch/projects/bashcritic/milestone/Beta
>
> on the trac project page so that you can follow the progess (if you
> are interested of course :-)

A couple ot train trips gave me the possibility to implement some more
rules and to begin to fix the script itself (e.g. stuff in comments
and strings is not checked). I know that a string could be executed at
a later point but checking them should be optional.

In any case version 0.3.0 is available for download:
https://trac.id.ethz.ch/projects/bashcritic/downloads

List of supported rules:
https://trac.id.ethz.ch/projects/bashcritic/wiki#Tests

Thanks again,

Matteo

Re: Announcement: bashcritic

Stephane CHAZELAS wrote:
> ...
> is a kshism. In some AT&T ksh, functions defined like that are
> treated differently from the standardly defined ones. Note that
> the { ...; } are not necessary. POSIX requires a compound
> command. All shells but bash will allow any command.

Not true. Kornshell never has supported and has in B&K officially
deprecated this Bourne shell's syntax expressing optional following
grouping keywords to define that function declaration.
E.g.:

foo(): # no braces: only works in Bourne shell
foo(){ :;} -or- function foo { :; } # Kornshell usage

But then again, much of the functionality of Kornshell was merely
undocumented in the Bourne shell, e.g. "<>".

Did you know that Bourne shell (and presumably Kornshell) can
use grouping keywords for the compound statements following
"while", "for", and "until"? IIRC, only "case" couldn't accept this
usage. E.g.:

while true
{ # Is it C? C++? Algol?? ;)
cmd1
cmd2
}

I can't currently test this verify my memory of this syntactical
variant.

=Brian

Re: Announcement: bashcritic

bsh wrote:
> Not true. Kornshell never has supported and has in B&K officially
> deprecated this Bourne shell's syntax expressing optional following
> grouping keywords to define that function declaration.
> E.g.:
>
> foo(): # no braces: only works in Bourne shell

$ ksh
$ bar () :
$ bar
$ foo () echo hi
$ foo
hi

Version: Version M-11/16/88i



--
pgas
SDF Public Access UNIX System - http://sdf.lonestar.org

Re: Announcement: bashcritic

2007-10-21, 16:53(-07), bsh:
> Stephane CHAZELAS wrote:
>> ...
>> is a kshism. In some AT&T ksh, functions defined like that are
>> treated differently from the standardly defined ones. Note that
>> the { ...; } are not necessary. POSIX requires a compound
>> command. All shells but bash will allow any command.
>
> Not true. Kornshell never has supported and has in B&K officially
> deprecated this Bourne shell's syntax expressing optional following
> grouping keywords to define that function declaration.
> E.g.:
>
> foo(): # no braces: only works in Bourne shell
[...]

Not true, works in ksh88, ksh93, Bourne shell, ash, pdksh, posh,
mksh, zsh and their derivatives (at least).

And foo()(:), foo() for i do echo "$i"; done (that is, with a
compound command) are POSIX, and in addition to the above, are
supported by bash as well (hence my note about bash being
pedantic here in not supporting simple commands like every other
shell).

--
Stéphane

Re: Announcement: bashcritic

Stephane CHAZELAS wrote:
> bsh:
> > Stephane CHAZELAS wrote:
> > > ...
> Not true, works in ksh88, ksh93, Bourne shell, ash, pdksh, posh,
> mksh, zsh and their derivatives (at least). And foo()(:), foo() for i
> do echo "$i"; done (that is, with a compound command) are
> POSIX, and in addition to the above, are supported by bash as
> well (hence my note about bash being pedantic here in not
> supporting simple commands like every other shell).

I'm not surprised that the clone shells (including bash) support
the Bourne function variant syntax -- but POSIX?! May I ask
where exactly? POSIX is usually so conservative and minimalistic
that this is exactly the kind of thing that would have been "left to
the discrimination of the vendor." What are they thinking!?

Okay, I stand corrected. I didn't think I had been confusing it with
the new kornshell function syntax that required the grouping
keywords around its function body; however, I need to consult
B&K again because I could swear that (1) The ksh(1) downward-
compatibility mode Bourne function variant syntax is officially
deprecated, and (2) My recollection is trying this usage under
my SunOS4 ksh88d years ago and determining that it was as
described above. I wonder if this could this be a port-specific
issue? I'll look in the source code....

(Good news and bad news: by necessity, as I no longer have
access to a command prompt of the canonical shells, I rely on
having virtually memorized B&K. So a quote: "In theory, practise
and theory are the same, but in practise they're not.")

=Brian

Re: Announcement: bashcritic

2007-10-22, 17:30(-07), bsh:
> Stephane CHAZELAS wrote:
>> bsh:
>> > Stephane CHAZELAS wrote:
>> > > ...
>> Not true, works in ksh88, ksh93, Bourne shell, ash, pdksh, posh,
>> mksh, zsh and their derivatives (at least). And foo()(:), foo() for i
>> do echo "$i"; done (that is, with a compound command) are
>> POSIX, and in addition to the above, are supported by bash as
>> well (hence my note about bash being pedantic here in not
>> supporting simple commands like every other shell).
>
> I'm not surprised that the clone shells (including bash) support
> the Bourne function variant syntax -- but POSIX?! May I ask
> where exactly?

See the grammar:

http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_ chap02.html#tag_02_10_02

> POSIX is usually so conservative and minimalistic
> that this is exactly the kind of thing that would have been "left to
> the discrimination of the vendor." What are they thinking!?

Well, in Bourne and every other shell (but bash), creating a
function is stick foo() in front of a command.

The canonical syntax is
foo() {
...
}

Since {...} is the most obvious choice for grouping several
commands.

We've been using that canonical form so much what we forgot it
was only one way of defining a function.


> Okay, I stand corrected. I didn't think I had been confusing it with
> the new kornshell function syntax that required the grouping
> keywords around its function body; however, I need to consult
> B&K again because I could swear that (1) The ksh(1) downward-
> compatibility mode Bourne function variant syntax is officially
> deprecated, and (2) My recollection is trying this usage under
> my SunOS4 ksh88d years ago and determining that it was as
> described above. I wonder if this could this be a port-specific
> issue? I'll look in the source code....
[...]

Maybe you're confusing with the Korn vs Bourne syntax as in

foo() { ...; } vs function foo { ...; }

As far as I know, the ksh syntax predates the Bourne syntax
(The Bourne shell didn't have functions initially). Then, ksh
added support for the Bourne syntax for Bourne compatibility,
but both syntaxes are treated differently by ksh in some subtle
ways. So it may be why Korn advises to use the ksh syntax
instead in the same way as he advises to use "[[" instead of
"[".

Personnaly, I write POSIX scripts, so I use the POSIX syntax. I
wouldn't write ksh scripts nor bash scripts so that "advice"
from Korn doesn't apply to me.

--
Stéphane

Re: Announcement: bashcritic

Stephane CHAZELAS wrote:
> bsh:
> > Stephane CHAZELAS wrote:
> > > bsh:
> > > > Stephane CHAZELAS wrote:
> > > > > ...

> http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_ chap02.ht...
> The canonical syntax is foo() { ...; }
> since {...} is the most obvious choice for grouping several commands.

I don't agree that it was "obvious". The syntax of sh(1)
is liberally adopted from Stephen R. Bourne's experience
with Algol -- indeed, he was coauthor of Algol68C. On the
other hand, the idea of using braces for code blocks was
inherited from BCPL / B, the ancestor of C. I surmise that
braces were added as grouping keywords (not implemented in
old sh(1)!) and function body delimiters when those with
knowledge of C began to maintain the shell later on. An
undocumented feature of sh(1) is that braces may be substituted
for "do..done" in "for", "while", and "until" compound
statements -- as I had already mentioned in this thread --
and I suspect but cannot corroborate that this was a later
modification at the same time catering to the same "audience".

"Bourne Shell": http://en.wikipedia.org/wiki/Bourne_shell

http://en.wikipedia.org/wiki/ALGOL_68C

But in regard to bash(1), your advice to read the BNF in
the above document does indeed explicate a few things that
I haven't noticed before:

(1) The Bourne shell at least utilized a modified recursive
descent parser with lookahead. Kornshell retains this basic
code. It is _not_ a LR grammar nor is it single-pass. See:

http://groups.google.com/group/comp.unix.shell/msg/8378d15b1 1134de6

This leads me to think that the BNF grammar you point me to
is reverse-engineered from observation of behavior.

(2) I must have at least once known that a grouping construct
was a compound command... but it took this grammar to remind
me! _This_ is the key to understanding POSIX'es decision: they
didn't want to break the downward-compatibility _already_ built
into ksh(1) and other shells -- which is typical indeed of POSIX.
One (re)learns/recalls something new every day....

According to this BNF/yacc subset grammar:

function_definition : fname '(' ')' linebreak function_body ;
function_body : compound_command | compound_command redirect_list ;
compound_command : brace_group | subshell | ... ;
brace_group : Lbrace compound_list Rbrace ;
compound_list : term | ... ;
term : term separator and_or | and_or ;
and_or : pipeline | and_or AND_IF linebreak pipeline | and_or OR_IF
linebreak pipeline ;
pipeline : pipe_sequence | ... ;
pipe_sequence : command | pipe_sequence '|' linebreak command ;
command : simple_command | compound_command | function_definition
| ... ;
simple_command : cmd_name | ... ;

By inspection, I determine that the token stream "foo" "("
")" "{" cmds "}" (I elide the semicolon) has the parse tree:

function_definition
+--fname = "foo"
+--"("
+--")"
+--function_body
| +--compound_command
| | +--brace_group
| | | +--Lbrace = "{"
| | | | +--compound_list
| | | | | +--term
| | | | | | +--and_or
| | | | | | | +--pipeline
| | | | | | | | +--pipe_sequence
| | | | | | | | | +--command
| | | | | | | | | | +--simple_command
| | | | | | | | | | | +--cmd_name = "cmds"
| | | +--Rbrace = "}"

(2) But there is a problem. According this grammar that
you cite as the de facto justification for the variant
bourne function syntax, the token stream "foo" "(" ")"
"cmds" has no production! It is broken grammar.

function_body : compound_command | compound_command redirect_list ;
compound_command : brace_group | subshell | ... ;
subshell : '(' compound_list ')' ;

function_definition
+--fname = "foo"
+--"("
+--")"
+--function_body
| +--compound_command
| | +--subshell
error: "(" expected!

If derivation rule for compound_command had been instead:

compound_command : brace_group | term | subshell | ... ;

.... it should have the expected behavior.

I use "term", although I cannot see why the more appropriate
"complete_command" shouldn't be used, nor why there are
identical derivation rules for "term" and "list"....

(BTW, I wonder what "rule 9" for functions is, anyway....)

(3) Another problems with this grammar: according to the
derivation rule:

command : simple_command | compound_command | function_definition
| ... ;

Allows the legal production:

cmd1 | cmd2 | cmd3 () { cmd4; }

Hmmm.

(4) Also, I'm wondering how this can be a formal standard or
even a meta-description. It won't parse the construct for left-
associative redirection for subshells. E.g.: " although via some hackery it simulates left-association and/or
right-associative redirection bindings for commands and
[compound] command groupings.

> We've been using that canonical form so much what we forgot it
> was only one way of defining a function.
> Maybe you're confusing with the Korn vs Bourne syntax as in
> foo() { ...; } vs. function foo { ...; }

No. I addressed this in the second sentence of the second
paragraph. You couldn't really think I would not know the
difference, did you?

> As far as I know, the ksh syntax predates the Bourne syntax
> (The Bourne shell didn't have functions initially).

No. Although ksh had a version 1982 or 1984 (records vary),
it wasn't released outside Bell Labs / AT&T until 1988
within SVR4. Bourne shell, extant circa 1976, and released
1979, first implemented functions in as the shell in SVR2
(1984). See:

http://groups.google.com/group/alt.folklore.computers/msg/a1 76a30b54f92c99

"The Traditional Bourne Shell Family":
http://www.in-ulm.de/~mascheck/bourne/

"Unix Command Shell Graph & Timeline":
http://www.aisee.com/manual/unix/img31lrg.gif

> Then, ksh
> added support for the Bourne syntax for Bourne compatibility,
> but both syntaxes are treated differently by ksh in some subtle
> ways.

No. ksh has always been built upon the Bourne shell codebase,
updated from the original "Bournalgol" to the K&R dialect, and
currently ANSI C. See:

"ksh: An Extensible High Level Language: chapter 2: HISTORY":
http://www.in-ulm.de/~mascheck/bourne/korn.html

http://minnie.tuhs.org/UnixTree/V7/usr/src/cmd/sh/mac.h.html

> So it may be why Korn advises to use the ksh syntax
> instead in the same way as he advises to use "[[" instead of
> "[".

But, this unfortunate ksh neologism was specifically
created for the _very reason_ that he was trying to
both conform to new POSIX "test" (utilizing the disambiguation
method of counting argv) as well as to avoid breaking legacy
code! The same is true for having "invented" $'...' quoting
to allow ANSI encodings, and having "invented" print/printf
to allow ANSI printf(3) emulation, and the new function
syntax -- all added to be formally upward-compatible. See:

http://kornshell.com/doc/faq.html # "III.Q5, III.Q10, III.Q12"

> Personally, I write POSIX scripts, so I use the POSIX
> syntax. I wouldn't write ksh scripts nor bash scripts so that
> "advice" from Korn doesn't apply to me.

This is a naive statement, Stephane. I am aware of your
preference for bash/zsh, as well as your statements
deprecating ksh in past posts (why, I wonder?) but ksh
_is_ POSIX, warts and all [1]. Kornshell 1988 complies/
conforms to IEEE P1003.2-1992 / ISO 9945.2-1993, IEEE
POSIX 1003.1, and ANSI-C proposed standards [2]. The
1993 version is a superset of the POSIX and ISO/IEC 1992
shell standards [3], and had passed the X/OPEN test
suite [4].

[1] https://mailman.research.att.com/pipermail/ast-users/2006q4/ 001472.html
[2] http://www.cs.princeton.edu/~jlk/lj/korns1.html
[3] http://www.in-ulm.de/~mascheck/bourne/korn.html
[4] http://kornshell.com/doc/faq.html

On the other hand, I remember emailing Chet Ramey some
years ago, complaining that bash was breaking my new
interactive debugger script because it wasn't following
documented _and_ POSIX behavior. He gave as his reply a
mistaken understanding of the rationale for special
builtin commands, called me "disingenuous," and suggested
using the "--posix" command line option! So, you were
saying?.... ;)

Now, regarding Matteo Corti and his script bashcritic:

After all this exegesis, I can make a definitive,
though discouraging, suggestion: the only _real_
solution to a static checker is to implement it using
a parser library -- preferably one already written ;)

Alas, there is no general purpose parser written in
perl that I can find in 15 minutes of searching in
CPAN. *Sheesh!* Even javascript has more than one!

For this particular solution, _I_ would investigate PGE
(Parser Grammar Engine) in perl6 used to emit rules for
parsing perl itself. Your program could be as simple as
applications programming, given the proper software tool.

And talking about the correct tool... I already have
written a special-purpose sh/ksh/bash-subset scanner/
parser toolset! While it has the underlying limits of
the sed (1000 LOC) and awk (1000 LOC) that it's
written in, it is a formal parser and cannot be
spoofed -- but here's the catch: I cannot get it off
my old SparcStation. The more I think about it, the
more it seems like the perfect expedient, insofar as
there even exists a software tool to convert sed scripts
into perl (with sed2pl). You could apply all your
concentration to context-free pattern matching,
instead of accomodating contextual "syntactic sugar".
Are you interested enough for me to go to the effort
of "prying the hood" of my dead SS?

You should know that for the last year I have been
surveying the practicality of writing a LR or RD parser
for ksh88/93 -- other shells are either too slow or
"fray at the edges" when attempting non-trivial
programming. Trivial examples have already been done,
but they are mere toys:

"ply.bash": "Primitive Lex Yacc"
http://sf.net/projects/ply/

I once thought to write a new /usr/lib/yaccpar and
yyparse.h for yacc(1) to emit shell code, and I have now
determined that a PEG (Parsing Expression Grammar) is the
most practical -- exploiting the powerful new extended
pattern matching in the latest versions of ksh93, but what
I will probably end up doing is port to ksh a pre-existing
frontend to the open-source parser GOLD, a la:

"pygold.py": stubs for utilizing the GOLD parser
http://sf.net/projects/pygold/

"Calitha Engine.cs": Parser toolset utilizing GOLD
http://calitha.com
http://www.xs4all.nl/~rvanloen/

GOLD: embeddable parser engine, multihost
http://www.devincook.com/goldparser/

I hope this helps. Good luck!

=Brian

Re: Announcement: bashcritic

2007-10-24, 18:38(-07), bsh:
> Stephane CHAZELAS wrote:
>> bsh:
>> > Stephane CHAZELAS wrote:
>> > > bsh:
>> > > > Stephane CHAZELAS wrote:
>> > > > > ...
>
>> http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_ chap02.ht...
>> The canonical syntax is foo() { ...; }
>> since {...} is the most obvious choice for grouping several commands.
>
> I don't agree that it was "obvious". The syntax of sh(1)
> is liberally adopted from Stephen R. Bourne's experience
> with Algol -- indeed, he was coauthor of Algol68C. On the
> other hand, the idea of using braces for code blocks was
> inherited from BCPL / B, the ancestor of C. I surmise that
> braces were added as grouping keywords (not implemented in
> old sh(1)!) and function body delimiters when those with
> knowledge of C began to maintain the shell later on. An
> undocumented feature of sh(1) is that braces may be substituted
> for "do..done" in "for", "while", and "until" compound
> statements -- as I had already mentioned in this thread --
> and I suspect but cannot corroborate that this was a later
> modification at the same time catering to the same "audience".
>
> "Bourne Shell": http://en.wikipedia.org/wiki/Bourne_shell

That's not what I meant. I meant people got used to do

foo() {
...
}

and not use the other forms, because in a function you generally
want to put several commands, and { ... ;} is the (most obvious)
way to group commands in the Bourne shell.

[...]
> According to this BNF/yacc subset grammar:
>
> function_definition : fname '(' ')' linebreak function_body ;
> function_body : compound_command | compound_command redirect_list ;
> compound_command : brace_group | subshell | ... ;
> brace_group : Lbrace compound_list Rbrace ;
> compound_list : term | ... ;
> term : term separator and_or | and_or ;
> and_or : pipeline | and_or AND_IF linebreak pipeline | and_or OR_IF
> linebreak pipeline ;
> pipeline : pipe_sequence | ... ;
> pipe_sequence : command | pipe_sequence '|' linebreak command ;
> command : simple_command | compound_command | function_definition
>| ... ;
> simple_command : cmd_name | ... ;
[...]
> (2) But there is a problem. According this grammar that
> you cite as the de facto justification for the variant
> bourne function syntax, the token stream "foo" "(" ")"
> "cmds" has no production! It is broken grammar.
>
> function_body : compound_command | compound_command redirect_list ;
> compound_command : brace_group | subshell | ... ;
> subshell : '(' compound_list ')' ;
>
> function_definition
> +--fname = "foo"
> +--"("
> +--")"
> +--function_body
>| +--compound_command
>| | +--subshell
> error: "(" expected!
>
> If derivation rule for compound_command had been instead:
>
> compound_command : brace_group | term | subshell | ... ;
>
> ... it should have the expected behavior.
>
> I use "term", although I cannot see why the more appropriate
> "complete_command" shouldn't be used, nor why there are
> identical derivation rules for "term" and "list"....
[...]

The more appropriate term is "simple command". That's exactly
what I said. POSIX doesn't require the
foo() echo bar
Bourne/ksh/zsh/ash syntax to be recognised. That's why I said
bash was being pedantic in applying POSIX strictly.

>
> (BTW, I wonder what "rule 9" for functions is, anyway....)
>
> (3) Another problems with this grammar: according to the
> derivation rule:
>
> command : simple_command | compound_command | function_definition
>| ... ;
>
> Allows the legal production:
>
> cmd1 | cmd2 | cmd3 () { cmd4; }

So what?

You're also allowed to do
a=1 | b=2 | c=3

Whether you want to do it is another matter.

[...]
>> As far as I know, the ksh syntax predates the Bourne syntax
>> (The Bourne shell didn't have functions initially).
>
> No. Although ksh had a version 1982 or 1984 (records vary),
> it wasn't released outside Bell Labs / AT&T until 1988
> within SVR4. Bourne shell, extant circa 1976, and released
> 1979, first implemented functions in as the shell in SVR2
> (1984). See:

Does that contradict what I said?

[...]
>> Then, ksh
>> added support for the Bourne syntax for Bourne compatibility,
>> but both syntaxes are treated differently by ksh in some subtle
>> ways.
>
> No. ksh has always been built upon the Bourne shell codebase,
> updated from the original "Bournalgol" to the K&R dialect, and
> currently ANSI C. See:

You're contradicting yourself. If ksh was being developped in
1982 and based on the Bourne shell, and functions were
introduced in the Bourne shell in 84, then ksh had no reason to
have the Bourne shell functions initially.

[...]
>> So it may be why Korn advises to use the ksh syntax
>> instead in the same way as he advises to use "[[" instead of
>> "[".
>
> But, this unfortunate ksh neologism was specifically
> created for the _very reason_ that he was trying to
> both conform to new POSIX "test" (utilizing the disambiguation
> method of counting argv) as well as to avoid breaking legacy
> code!

I'm pretty sure I've read from Korn that the function foo {
syntax predates the Bourne foo() { syntax, so that would
contradict your above statement. I'll try to find sources when I
have a moment.

> http://kornshell.com/doc/faq.html # "III.Q5, III.Q10, III.Q12"
>
>> Personally, I write POSIX scripts, so I use the POSIX
>> syntax. I wouldn't write ksh scripts nor bash scripts so that
>> "advice" from Korn doesn't apply to me.
>
> This is a naive statement, Stephane. I am aware of your
> preference for bash/zsh, as well as your statements
> deprecating ksh in past posts (why, I wonder?) but ksh
> _is_ POSIX, warts and all [1]. Kornshell 1988 complies/
> conforms to IEEE P1003.2-1992 / ISO 9945.2-1993, IEEE
> POSIX 1003.1, and ANSI-C proposed standards [2]. The
> 1993 version is a superset of the POSIX and ISO/IEC 1992
> shell standards [3], and had passed the X/OPEN test
> suite [4].

You missed my point. I don't write ksh scripts, I write POSIX
scripts. I don't care what the interpreter is. Generally (as most
commercial Unix POSIX sh are based on a ksh) it is actually a
ksh and that's fine with me (though sometimes my script have to
work around some limitations/conformance bug of this or that
shell).

I ranted on ksh idiosyncrasies, but bash is no better being an
unfinished clone of it. I use zsh as my interactive shell as
that's by far the shell that makes me most productive for my
everyday work, but that's a different matter.

--
Stéphane

Re: Announcement: bashcritic

bsh wrote:

> (4) Also, I'm wondering how this can be a formal standard or
> even a meta-description. It won't parse the construct for left-
> associative redirection for subshells. E.g.: "
That's intentional. POSIX doesn't require shells to support that
syntax (and bash doesn't support it).

--
Geoff Clare

Re: Announcement: bashcritic

Stephane CHAZELAS wrote:
> bsh:
> > Stephane CHAZELAS wrote:
> > > bsh:
> > > > Stephane CHAZELAS wrote:
> > > > > bsh:
> > > > > > Stephane CHAZELAS wrote:
> > > > > > > ...

> > I don't agree that it was "obvious" [for grouping several commands].
> That's not what I meant. I meant people got used to do "foo() { ... }"
> ...

Oh.

> > I use "term", although I cannot see why the more appropriate
> > "complete_command"....
> The more appropriate term is "simple command".

No. "complete_command" (and for that matter, "simple_command")
are formal derivation rules that refer to the BNF grammar. As
you had specifically stated yourself, "foo()... with a compound
command... is POSIX," the braceless function body code would have
to be a parsable construct with optional redirection, called in
the POSIX BNF an "AND_OR" or (as I understand it) a
"complete_command".
E.g.:

foo () cmd1 | cmd2 >file

is syntactically equivalent to:

foo () { cmd1 | cmd2 >file; }

That is, the above would/should not be parsed as:

{ foo () cmd1; } | cmd2 >file

> POSIX doesn't require the [bourne function variant syntax]
> to be recognised.

It either does -- and the included POSIX grammar is incomplete
or incorrect -- or it doesn't -- and this very sentence contradicts
the entire thesis of your post!

> > [The grammar] allows the legal production:
> > cmd1 | cmd2 | cmd3 () { cmd4; }
> So what?

I was strengthening my argument that the BNF grammar included
in this international standard is utterly inadequate to serve
in that capacity. An emphatic email should be written to the
POSIX committee!

> You're also allowed to do "a=1 | b=2 | c=3"

This is not a suitable counterexample, and ironic insofar as I
implicitly addressed the special case of variable assignment lists
in my pointer to my previous post. In the first of three passes,
the parser "strips out" VALs before command parsing is ever done --
canonical sh(1) does not even see them in the command-line -- so
your example is sytactically equivalent to ": | : | :", an assignment
being equivalent to "var=value :". VALs do not take stdin/stdout/
stderr, but the null statement slash ":" command _does_. This behavior
is understandably counterintuitive for modern programmers, and is the
rationale for the weird "set -k" shell option: yet another case
of backward-compatibility being preserved with the undicriminating
parsing of VALS in old legacy sh(1).

> > > Then, ksh
> > > added support for the Bourne syntax for Bourne compatibility,
> > > but both syntaxes are treated differently by ksh in some subtle
> > > ways.
> > No. ksh has always been built upon the Bourne shell codebase,
> > updated from the original "Bournalgol" to the K&R dialect, and
> > currently ANSI C. See:
> You're contradicting yourself. If ksh was being developped in
> 1982 and based on the Bourne shell, and functions were
> introduced in the Bourne shell in 84, then ksh had no reason to
> have the Bourne shell functions initially.

No. I could have alternately said: if ksh(1) had implemented functions
first (by whatever syntax), then sh(1) (without any kind of
functions yet) would have been back-patched with the syntax of
functions that ksh(1) had. As this is not so, it is the logical
assumption that ksh(1) functions were implemented after sh(1),
especially insofar as the POSIX-functions in ksh88(1) were
implemented incorrectly with ksh-style lexical scoping. All else
is surmise.

> I'm pretty sure I've read from Korn that the function foo {
> syntax predates the Bourne foo() { syntax, so that would
> contradict your above statement. I'll try to find sources when I
> have a moment.

I'd be interested in your findings. Maybe Sven Mascheck knows.
Perhaps even an email to DGK himself would be in order?

> > http://kornshell.com/doc/faq.html# "III.Q5, III.Q10, III.Q12"
> You missed my point. I don't write ksh scripts, I write POSIX
> scripts. I don't care what the interpreter is. Generally (as most
> commercial Unix POSIX sh are based on a ksh) it is actually a
> ksh and that's fine with me (though sometimes my script have to
> work around some limitations/conformance bug of this or that
> shell).
> I ranted on ksh idiosyncrasies, but bash is no better being an
> unfinished clone of it.

I now understand your statement better. Like the PC platform
being still being able to execute 30-year old software without
modification, but suffering for the layer upon layer of vision-less
extensions to accomodate each generation of functionality, ksh(1)
suffers a little bit for its attempt (sometimes incorrectly
implemented) of backward-compatibility, but at least DGK has always
tried to protect legacy code by contriving new syntax when he knew
its behavior would conflict with the old. On the other hand,
bash(1) is the fresh product of a post-standards world, but suffers
a bit by having to play "catch-up" simultaneous to a belated penchant
for indiscriminate "feature-itis".

> I use zsh as my interactive shell as
> that's by far the shell that makes me most productive for my
> everyday work, but that's a different matter.

Yes, zsh is a nice shell... because it knows how to turn off or
reconfigure its non-standard extensions and behavior at the will of
the user.

=Brian

Re: Announcement: bashcritic

Stephane CHAZELAS wrote:

> $((...)) is POSIX but not Bourne

As you mentioned also, considering traditional bourne compatibility
is different from POSIX. Numerous inspirations can be found on
Paul Jarc's list of suspicious or nonportable constructs
http://code.dogmap.org./lintsh/


> type [...]

> unset is not in every Bourne shell

> functions and comments were not supported in very early versions
> of the Bourne shell

All these features were introduced with the SVR2 shell.
You can rely on them, because earlier variants can only
be of historical interest. (But, if you find a still running
"Ultrix", you'll even be able to experience a V7-like /bin/sh.)

[ The autoconf doc is famous for the above items but it's old and
the intention was to be as robust as possible in install scripts.
It's very interesting to test for such old shells, but scripting
and using external commands is something different then. ]


BTW, the most important changes after SVR2 probably were
- modern handling of "$@" (SVR3)
- positional parameters robust about function calls (SVR3)
- read -r (SVR4.2)

Re: Announcement: bashcritic

Teo wrote:

> I began to add some tests for portability but I would really like some
> feedback and some suggestions on what to check (I will in any case
> monitor the group looking for possible candidates)

Maybe this is interesting for you, too: http://code.dogmap.org./lintsh/
(less lintsh itself, more the list at the bottom).

J.