Doesn"t xargs support regular expression?

Folks,

I met a puzzle here. Suppose I have a file named a.txt, which has
below lines:
/etc/system*
/etc/{dir1, dir2}

/* Above directories are all existing ones */

I wish to get all of the files listed in a.txt and pass them via xarg
to a comand like "ls -l", like this:
cat a.txt | xargs --verbose ls -l

But it seems xargs doesn't parse the regular expression correctly,
which just think it as a simple string. So I get below error:
ls -l /etc/system*
ls: /etc/system*: No such file or directory

How could I get over the problem or let xargs know its an regular
expression instead of a string.

Thanks,

Raymond

Re: Doesn"t xargs support regular expression?

On Wed, 14 Nov 2007 03:59:51 -0000, Raymond wrote:
| Folks,
|
| I met a puzzle here. Suppose I have a file named a.txt, which has
| below lines:
| /etc/system*
| /etc/{dir1, dir2}
|
| /* Above directories are all existing ones */
|
| I wish to get all of the files listed in a.txt and pass them via xarg
| to a comand like "ls -l", like this:
| cat a.txt | xargs --verbose ls -l
|
| But it seems xargs doesn't parse the regular expression correctly,
| which just think it as a simple string. So I get below error:
| ls -l /etc/system*
| ls: /etc/system*: No such file or directory
|
| How could I get over the problem or let xargs know its an regular
| expression instead of a string.


Under unix, most programs don't understand REs. What you have above
looks like shell file expansion syntax, not a regular expression.

Normally, the shell will expand any wildcards before handing the command
line to the program, so programs don't bother trying to do this again.

I haven't tried this, but you might get the behaviour you are expecting
if you let a shell see the strings first. Try something like this:

cat a.txt | xargs --verbose /bin/bash ls -l

Or use /bin/ksh. From memory (ICBR) /bin/sh does not understand the
/etc/{dir1, dir2} syntax.


--
Reverend Paul Colquhoun, ULC. http://andor.dropbear.id.au/~paulcol
Asking for technical help in newsgroups? Read this first:
http://catb.org/~esr/faqs/smart-questions.html#intro

Re: Doesn"t xargs support regular expression?

Raymond wrote:
> Folks,
>
> I met a puzzle here. Suppose I have a file named a.txt, which has
> below lines:
> /etc/system*
> /etc/{dir1, dir2}
>
> /* Above directories are all existing ones */
>
> I wish to get all of the files listed in a.txt and pass them via xarg
> to a comand like "ls -l", like this:
> cat a.txt | xargs --verbose ls -l

UUOC; the above is equivalent to

xargs --verbose ls -l < a.txt

>
> But it seems xargs doesn't parse the regular expression correctly,

What do you mean by "it doesn't parse *correctly*"?
Where is it specified that xargs expands shell patterns?
In Unix the _shell_ expands the "wildcards".

> which just think it as a simple string. So I get below error:
> ls -l /etc/system*
> ls: /etc/system*: No such file or directory
>
> How could I get over the problem or let xargs know its an regular
> expression instead of a string.

You can let the shell expand it...

printf "'%s'\n" $(< a.txt) | xargs ls -l


Janis

>
> Thanks,
>
> Raymond
>

Re: Doesn"t xargs support regular expression?

On 11 14 , 1 06 , Janis Papanagnou
wrote:
> Raymond wrote:
> > Folks,
>
> > I met a puzzle here. Suppose I have a file named a.txt, which has
> > below lines:
> > /etc/system*
> > /etc/{dir1, dir2}
>
> > /* Above directories are all existing ones */
>
> > I wish to get all of the files listed in a.txt and pass them via xarg
> > to a comand like "ls -l", like this:
> > cat a.txt | xargs --verbose ls -l
>
> UUOC; the above is equivalent to
>
> xargs --verbose ls -l < a.txt
>
>
>
> > But it seems xargs doesn't parse the regular expression correctly,
>
> What do you mean by "it doesn't parse *correctly*"?
> Where is it specified that xargs expands shell patterns?
> In Unix the _shell_ expands the "wildcards".
>
I understand that better with Paul's previous mail. Sorry for confuse
if any.

> > which just think it as a simple string. So I get below error:
> > ls -l /etc/system*
> > ls: /etc/system*: No such file or directory
>
> > How could I get over the problem or let xargs know its an regular
> > expression instead of a string.
>
> You can let the shell expand it...
>
> printf "'%s'\n" $(< a.txt) | xargs ls -l
>
This works, thanks


> Janis
>
>
>
>
>
> > Thanks,
>
> > Raymond- -
>
> - -

Re: Doesn"t xargs support regular expression?

On 11 14 , 1 06 , Janis Papanagnou
wrote:
> > How could I get over the problem or let xargs know its an regular
> > expression instead of a string.
>
> You can let the shell expand it...
>
> printf "'%s'\n" $(< a.txt) | xargs ls -l
>
Opps, just found this works with things like /etc/a.*, but dont' work
with braket {}. That is it doesn't work with things like "/etc/
{dir1,dir2}"

Any advise?

> Janis
>
>
>
>
>
> > Thanks,
>
> > Raymond- -
>
> - -

Re: Doesn"t xargs support regular expression?

in message <1195019242.854913.5090@o38g2000hse.googlegroups.com>,
wrote Raymond ...

> On 11 14 , 1 06 , Janis Papanagnou
> wrote:
>> > How could I get over the problem or let xargs know its an
>> > regular expression instead of a string.
>>
>> You can let the shell expand it...
>>
>> printf "'%s'\n" $(< a.txt) | xargs ls -l
>>
> Opps, just found this works with things like /etc/a.*, but dont'
> work with braket {}. That is it doesn't work with things like
> "/etc/ {dir1,dir2}"

The suggested pipeline is failing in both cases, a* & a{b,c}, in zsh
4.3.4 here on FreeBSD 6-STABLE ...

# cat p
/etc/rc{,.conf}
/etc/rc.s*

# printf "'%s'\n" $(< p) | xargs ls -l
ls: /etc/rc.s*: No such file or directory
ls: /etc/rc{,.conf}: No such file or directory


.... but either of these work (with possibly other ways too) ...

# xargs -n 1 printf "ls -d -1 %s\n" < p | zsh -s
# xargs -I % zsh -c 'ls -d -1 %' < p
/etc/rc
/etc/rc.conf
/etc/rc.sendmail
/etc/rc.shutdown
/etc/rc.subr
/etc/rc.suspend


.... mind that if ls(1) command is not quoted in the second pipeline
as above, then listing of the current directory is printed.

Speaking of which ... reading "INVOCATION OPTIONS" section of zsh(1)
man page ...

-c Take the first argument as a command to execute,
rather than reading commands from a script or standard input.
If any further arguments are given, the first one is assigned
to $0, rather than being used as a positional parameter.


.... led to me to believe that if one of ...

# xargs -I % zsh -c ls '-d -1 %' < p
# xargs -I % zsh -c ls -d -1 % < p


.... is executed, then $0 would be 'ls' and rest would end up in $@.
As indirectly noted above, the '-d -1 %' part seems to be missing
from arguments to ls(1) (why is that?).

What is wrong in my thinking (perhaps I fail to parse the -c option
description)?


- parv

--
Email address is broken.

Re: Doesn"t xargs support regular expression?

Finally I found below script works, which is to have shell parse the
filename expansion.

cat a.txt | while read line;do
/bin/ksh -c "ls ${line}";

Raymond

On 11ÔÂ14ÈÕ, ÏÂÎç5ʱ05·Ö, parv ..com> wrote:
> in message <1195019242.854913.5...@o38g2000hse.googlegroups.com>,
> wrote Raymond ...
>
> > On 11 14 , 1 06 , Janis Papanagnou
> > wrote:
> >> > How could I get over the problem or let xargs know its an
> >> > regular expression instead of a string.
>
> >> You can let the shell expand it...
>
> >> printf "'%s'\n" $(< a.txt) | xargs ls -l
>
> > Opps, just found this works with things like /etc/a.*, but dont'
> > work with braket {}. That is it doesn't work with things like
> > "/etc/ {dir1,dir2}"
>
> The suggested pipeline is failing in both cases, a* & a{b,c}, in zsh
> 4.3.4 here on FreeBSD 6-STABLE ...
>
> # cat p
> /etc/rc{,.conf}
> /etc/rc.s*
>
> # printf "'%s'\n" $(< p) | xargs ls -l
> ls: /etc/rc.s*: No such file or directory
> ls: /etc/rc{,.conf}: No such file or directory
>
> ... but either of these work (with possibly other ways too) ...
>
> # xargs -n 1 printf "ls -d -1 %s\n" < p | zsh -s
> # xargs -I % zsh -c 'ls -d -1 %' < p
> /etc/rc
> /etc/rc.conf
> /etc/rc.sendmail
> /etc/rc.shutdown
> /etc/rc.subr
> /etc/rc.suspend
>
> ... mind that if ls(1) command is not quoted in the second pipeline
> as above, then listing of the current directory is printed.
>
> Speaking of which ... reading "INVOCATION OPTIONS" section of zsh(1)
> man page ...
>
> -c Take the first argument as a command to execute,
> rather than reading commands from a script or standard input.
> If any further arguments are given, the first one is assigned
> to $0, rather than being used as a positional parameter.
>
> ... led to me to believe that if one of ...
>
> # xargs -I % zsh -c ls '-d -1 %' < p
> # xargs -I % zsh -c ls -d -1 % < p
>
> ... is executed, then $0 would be 'ls' and rest would end up in $@.
> As indirectly noted above, the '-d -1 %' part seems to be missing
> from arguments to ls(1) (why is that?).
>
> What is wrong in my thinking (perhaps I fail to parse the -c option
> description)?
>
> - parv
>
> --
> Email address is broken.

Re: Doesn"t xargs support regular expression?

Raymond writes:

> Finally I found below script works, which is to have shell parse the
> filename expansion.
>
> cat a.txt | while read line;do
> /bin/ksh -c "ls ${line}";


You can use echo instead of ls - echo is built-in (faster).

Re: Doesn"t xargs support regular expression?

I think its a very interesting question after all. You have to think a
lot about the order in witch the shell does brace expansion, pathname
expansion etc, you have to remark that eval is a builtin and thus
can't be invoked by xargs, you have to understand the difference
between globbing and regex search, etc etc. I am currently learning
shell scripting, and I learned a lot pondering about this exercise.

The last solution by Raymond, i.e

cat a.txt | while read line;do
/bin/ksh -c "ls ${line}";

has still the problem that
a) if one line in a.txt expands to a lot of arguments, it may exceed
ARG_MAX and thus ls could not be called in this case.
b) the shell is invoked once for each line instead only once, imposing
a performance penalty.


How about the following?

cat a.txt | perl -pe 's/(.*)/echo $1/' | bash | xargs ls -l

Sorry for using perl in this group, but I don't know the equivalent
sed or awk script.

Greetings

Flo