Bookmarks

Yahoo Gmail Google Facebook Delicious Twitter Reddit Stumpleupon Myspace Digg

Search queries



Links

Issociate
Impressum

#1: sed replace by enviroment var content

Posted on 2005-12-07 12:32:11 by Ruud de Jong

I want to use sed to read a piece of text and replace some strings by
the value of that in the environment.
The input file looks like this:

blabla${PATH}blabla

Then sed needs to replace "${PATH}" by the value of "PATH" in the
environment

PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin

so sed should return

blabla/etc:/usr/sbin:/usr/bin/X11:/sbin:/usr/local/binblabla

(this is just an example to make it simple)
If I use:

sed -e "s|\${\(PATH\)}|${PATH}|g" < input.txt

then is will do the job.
but... (now comes the hard part)...
It needs to be generic for multiple environment variables.
So the "replace" part of the sed needs to use "\1" so it can use the
matching input.
(After this, I need to change the "search" part to search for other
names as well, but that is no problem)
So if I do:

sed -e "s|\${\(PATH\)}|${\1}|g" < input.txt

I get: "bad substitution"
But when I use:

sed -e "s|\${\(PATH\)}|\1|g" < input.txt

It DOES use the 1st parameter.
This means the "\1" and "${ }" are interfering with each other.
I've tried lots of different single and double quotes, but it just
doesn't seem to work as I want it to.

The shell should get the value of the parameter, but sed FIRST has to
put in the correct name.

I'm trying this on AIX, but the problem is probably similar on Linux.

Report this message

Mr Ad

Google

#2: Re: sed replace by enviroment var content

Posted on 2005-12-07 15:04:05 by Ed Morton

Ruud de Jong wrote:
> I want to use sed to read a piece of text and replace some strings by
> the value of that in the environment.
> The input file looks like this:
>
> blabla${PATH}blabla
>
> Then sed needs to replace "${PATH}" by the value of "PATH" in the
> environment
>
> PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
>
> so sed should return
>
> blabla/etc:/usr/sbin:/usr/bin/X11:/sbin:/usr/local/binblabla
>
> (this is just an example to make it simple)
> If I use:
>
> sed -e "s|\${\(PATH\)}|${PATH}|g" < input.txt
>
> then is will do the job.
> but... (now comes the hard part)...
> It needs to be generic for multiple environment variables.
> So the "replace" part of the sed needs to use "\1" so it can use the
> matching input.
> (After this, I need to change the "search" part to search for other
> names as well, but that is no problem)
> So if I do:
>
> sed -e "s|\${\(PATH\)}|${\1}|g" < input.txt
>
> I get: "bad substitution"
> But when I use:
>
> sed -e "s|\${\(PATH\)}|\1|g" < input.txt
>
> It DOES use the 1st parameter.
> This means the "\1" and "${ }" are interfering with each other.
> I've tried lots of different single and double quotes, but it just
> doesn't seem to work as I want it to.
>
> The shell should get the value of the parameter, but sed FIRST has to
> put in the correct name.
>
> I'm trying this on AIX, but the problem is probably similar on Linux.

Maybe you'll get a sed solution from someone, but here's how to do it in
awk for contrast:

$ cat file
blabla${PATH}blabla
foofoo${HOME}foofoo
$ PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
$ HOME=/home/morton
awk
'match($0,"[$]{.*}"){var=substr($0,RSTART+2,RLENGTH-3);gsub( "[$]{"var"}",ENVIRON[var])}1'
file
blabla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/u sr/local/binblabla
foofoo/home/mortonfoofoo

The only caveat is that the variables (PATH, HOME, etc.) must be
exported or set on the awk command line (HOME="$HOME" etc. awk ...} for
ENVIRON to work.

Ed.

Report this message

#3: Re: sed replace by enviroment var content

Posted on 2005-12-07 15:20:52 by cfajohnson

On 2005-12-07, Ruud de Jong wrote:
> I want to use sed to read a piece of text and replace some strings by
> the value of that in the environment.
> The input file looks like this:
>
> blabla${PATH}blabla
>
> Then sed needs to replace "${PATH}" by the value of "PATH" in the
> environment
>
> PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
>
> so sed should return
>
> blabla/etc:/usr/sbin:/usr/bin/X11:/sbin:/usr/local/binblabla
>
> (this is just an example to make it simple)
> If I use:
>
> sed -e "s|\${\(PATH\)}|${PATH}|g" < input.txt
>
> then is will do the job.
> but... (now comes the hard part)...
> It needs to be generic for multiple environment variables.
> So the "replace" part of the sed needs to use "\1" so it can use the
> matching input.
> (After this, I need to change the "search" part to search for other
> names as well, but that is no problem)
> So if I do:
>
> sed -e "s|\${\(PATH\)}|${\1}|g" < input.txt
>
> I get: "bad substitution"
> But when I use:
>
> sed -e "s|\${\(PATH\)}|\1|g" < input.txt
>
> It DOES use the 1st parameter.
> This means the "\1" and "${ }" are interfering with each other.
> I've tried lots of different single and double quotes, but it just
> doesn't seem to work as I want it to.
>
> The shell should get the value of the parameter, but sed FIRST has to
> put in the correct name.
>
> I'm trying this on AIX, but the problem is probably similar on Linux.

It's not sed, but this should work with bash3:

eval "cat <<< \"$(< "$filename")\" "


--
Chris F.A. Johnson, author |
Shell Scripting Recipes: | My code in this post, if any,
A Problem-Solution Approach | is released under the
2005, Apress | GNU General Public Licence

Report this message

#4: Re: sed replace by enviroment var content

Posted on 2005-12-07 15:30:07 by Ruud de Jong

Ed Morton wrote:
> Ruud de Jong wrote:
>
>> I want to use sed to read a piece of text and replace some strings by
>> the value of that in the environment.
>> The input file looks like this:
>>
>> blabla${PATH}blabla
>>
>> Then sed needs to replace "${PATH}" by the value of "PATH" in the
>> environment
>>
>> PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
>>
>> so sed should return
>>
>> blabla/etc:/usr/sbin:/usr/bin/X11:/sbin:/usr/local/binblabla
>>
>> (this is just an example to make it simple)
>> If I use:
>>
>> sed -e "s|\${\(PATH\)}|${PATH}|g" < input.txt
>>
>> then is will do the job.
>> but... (now comes the hard part)...
>> It needs to be generic for multiple environment variables.
>> So the "replace" part of the sed needs to use "\1" so it can use the
>> matching input.
>> (After this, I need to change the "search" part to search for other
>> names as well, but that is no problem)
>> So if I do:
>>
>> sed -e "s|\${\(PATH\)}|${\1}|g" < input.txt
>>
>> I get: "bad substitution"
>> But when I use:
>>
>> sed -e "s|\${\(PATH\)}|\1|g" < input.txt
>>
>> It DOES use the 1st parameter.
>> This means the "\1" and "${ }" are interfering with each other.
>> I've tried lots of different single and double quotes, but it just
>> doesn't seem to work as I want it to.
>>
>> The shell should get the value of the parameter, but sed FIRST has to
>> put in the correct name.
>>
>> I'm trying this on AIX, but the problem is probably similar on Linux.
>
>
> Maybe you'll get a sed solution from someone, but here's how to do it in
> awk for contrast:
>
> $ cat file
> blabla${PATH}blabla
> foofoo${HOME}foofoo
> $ PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
> $ HOME=/home/morton
> awk
> 'match($0,"[$]{.*}"){var=substr($0,RSTART+2,RLENGTH-3);gsub( "[$]{"var"}",ENVIRON[var])}1'
> file
> blabla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/u sr/local/binblabla
>
> foofoo/home/mortonfoofoo
>
> The only caveat is that the variables (PATH, HOME, etc.) must be
> exported or set on the awk command line (HOME="$HOME" etc. awk ...} for
> ENVIRON to work.
>
> Ed.
This works, but only for one variable per line. With 2 or more vars on
one line, it does not work.
The input I have, consists of one big line with multiple vars like:

$ cat file
blabla${PATH}blabla foofoo${HOME}foofoo

The caveat you mention, is not a problem (in my case).

Report this message

#5: Re: sed replace by enviroment var content

Posted on 2005-12-07 15:40:04 by RolandRB

Chris F.A. Johnson wrote:
> On 2005-12-07, Ruud de Jong wrote:
> > I want to use sed to read a piece of text and replace some strings by
> > the value of that in the environment.
> > The input file looks like this:
> >
> > blabla${PATH}blabla
> >
> > Then sed needs to replace "${PATH}" by the value of "PATH" in the
> > environment
> >
> > PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
> >
> > so sed should return
> >
> > blabla/etc:/usr/sbin:/usr/bin/X11:/sbin:/usr/local/binblabla
> >
> > (this is just an example to make it simple)
> > If I use:
> >
> > sed -e "s|\${\(PATH\)}|${PATH}|g" < input.txt
> >
> > then is will do the job.
> > but... (now comes the hard part)...
> > It needs to be generic for multiple environment variables.
> > So the "replace" part of the sed needs to use "\1" so it can use the
> > matching input.
> > (After this, I need to change the "search" part to search for other
> > names as well, but that is no problem)
> > So if I do:
> >
> > sed -e "s|\${\(PATH\)}|${\1}|g" < input.txt
> >
> > I get: "bad substitution"
> > But when I use:
> >
> > sed -e "s|\${\(PATH\)}|\1|g" < input.txt
> >
> > It DOES use the 1st parameter.
> > This means the "\1" and "${ }" are interfering with each other.
> > I've tried lots of different single and double quotes, but it just
> > doesn't seem to work as I want it to.
> >
> > The shell should get the value of the parameter, but sed FIRST has to
> > put in the correct name.
> >
> > I'm trying this on AIX, but the problem is probably similar on Linux.
>
> It's not sed, but this should work with bash3:
>
> eval "cat <<< \"$(< "$filename")\" "

This should work for bash

$ while read line; do eval echo $line; done < file

Report this message

#6: Re: sed replace by enviroment var content

Posted on 2005-12-07 15:51:55 by RolandRB

Ruud de Jong wrote:
> I want to use sed to read a piece of text and replace some strings by
> the value of that in the environment.
> The input file looks like this:
>
> blabla${PATH}blabla
>
> Then sed needs to replace "${PATH}" by the value of "PATH" in the
> environment

Don't use sed for this. Use this method instead (for bash) to convert
all these environment variable references (any any other it comes
across) and write to a new file:

$ while read line; do eval echo $line; done < file > newfile

Report this message

#7: Re: sed replace by enviroment var content

Posted on 2005-12-07 15:54:44 by RolandRB

RolandRB wrote:
> Ruud de Jong wrote:
> > I want to use sed to read a piece of text and replace some strings by
> > the value of that in the environment.
> > The input file looks like this:
> >
> > blabla${PATH}blabla
> >
> > Then sed needs to replace "${PATH}" by the value of "PATH" in the
> > environment
>
> Don't use sed for this. Use this method instead (for bash) to convert
> all these environment variable references (any any other it comes
> across) and write to a new file:
>
> $ while read line; do eval echo $line; done < file > newfile

Echoing $line in double quotes is better.

$ while read line; do eval echo "$line"; done < file > newfile

Report this message

#8: Re: sed replace by enviroment var content

Posted on 2005-12-07 16:20:22 by Ruud de Jong

RolandRB wrote:
> RolandRB wrote:
>
>>Ruud de Jong wrote:
>>
>>>I want to use sed to read a piece of text and replace some strings by
>>>the value of that in the environment.
>>>The input file looks like this:
>>>
>>> blabla${PATH}blabla
>>>
>>>Then sed needs to replace "${PATH}" by the value of "PATH" in the
>>>environment
>>
>>Don't use sed for this. Use this method instead (for bash) to convert
>>all these environment variable references (any any other it comes
>>across) and write to a new file:
>>
>>$ while read line; do eval echo $line; done < file > newfile
>
>
> Echoing $line in double quotes is better.
>
> $ while read line; do eval echo "$line"; done < file > newfile
>
Thanks. This works. GREAT!

Report this message

#9: Re: sed replace by enviroment var content

Posted on 2005-12-07 16:37:53 by Ed Morton

Ruud de Jong wrote:
> Ed Morton wrote:
>
>>Ruud de Jong wrote:
>>
>>
>>>I want to use sed to read a piece of text and replace some strings by
>>>the value of that in the environment.
>>>The input file looks like this:
>>>
>>> blabla${PATH}blabla
>>>
>>>Then sed needs to replace "${PATH}" by the value of "PATH" in the
>>>environment
>>>
>>>
PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
>>>

>>Maybe you'll get a sed solution from someone, but here's how to do it in
>>awk for contrast:
>>
>>$ cat file
>>blabla${PATH}blabla
>>foofoo${HOME}foofoo
>>$ PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/us r/local/bin
>>$ HOME=/home/morton
>>awk
>>'match($0,"[$]{.*}"){var=substr($0,RSTART+2,RLENGTH-3);gsu b("[$]{"var"}",ENVIRON[var])}1'
>>file
>>blabla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin: /usr/local/binblabla
>>
>>foofoo/home/mortonfoofoo
>>
>>The only caveat is that the variables (PATH, HOME, etc.) must be
>>exported or set on the awk command line (HOME="$HOME" etc. awk ...} for
>>ENVIRON to work.
>>
>> Ed.
>
> This works, but only for one variable per line. With 2 or more vars on
> one line, it does not work.

Then just put it inside a while () loop and replace "." with "[^}]" to
get avoid greedy pattern matching:

$ cat file
blabla${PATH}blabla foofoo${HOME}foofoo
bla${PATH}bla${HOME}bla foo${HOME}foo
$ awk '{
while(match($0,"[$]{[^}]*}")){var=substr($0,RSTART+2,RLENGTH -3);gsub("[$]{"var"}",ENVIRON[var])}}1'
file
blabla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/u sr/local/binblabla
foofoo/home/mortonfoofoo
bla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/ local/binbla/home/mortonbla
foo/home/mortonfoo

> The input I have, consists of one big line with multiple vars like:
>
> $ cat file
> blabla${PATH}blabla foofoo${HOME}foofoo

Then that's the input you should've posted in the first place.

> The caveat you mention, is not a problem (in my case).

Good.

Ed.

Report this message

#10: Re: sed replace by enviroment var content

Posted on 2005-12-07 16:43:35 by Ed Morton

Ruud de Jong wrote:

> RolandRB wrote:
>
>>RolandRB wrote:
>>
>>
>>>Ruud de Jong wrote:
>>>
>>>
>>>>I want to use sed to read a piece of text and replace some strings by
>>>>the value of that in the environment.
>>>>The input file looks like this:
>>>>
>>>> blabla${PATH}blabla
>>>>
>>>>Then sed needs to replace "${PATH}" by the value of "PATH" in the
>>>>environment
>>>
>>>Don't use sed for this. Use this method instead (for bash) to convert
>>>all these environment variable references (any any other it comes
>>>across) and write to a new file:
>>>
>>>$ while read line; do eval echo $line; done < file > newfile
>>
>>
>>Echoing $line in double quotes is better.
>>
>>$ while read line; do eval echo "$line"; done < file > newfile
>>
>
> Thanks. This works. GREAT!

No, it doesn't. Look (and watch what happens to the "\t"):

$ cat file
blabla${PATH}bla\tbla
$ while read line; do eval echo "$line"; done < file
blabla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/u sr/local/binblatbla
$ awk '{
while(match($0,"[$]{[^}]*}")){var=substr($0,RSTART+2,RLENGTH -3);gsub("[$]{"var"}",ENVIRON[var])}}1'
file
blabla/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/u sr/local/binbla\tbla

Notice that the eval solution changed "\t" to "t".

Ed.

Report this message