Hello all
I'm trying to edit a file with sed in the following way:
sed 's/OldText/NewText/g' < myfile > myfile
using the same file for output as the input actually deletes the
content of myfile
the only way I do this is like so:
sed 's/OldText/NewText/g' < oldfile > newfile
cat newfile > oldfile
is there a way around this (using only sed)?
Thanks
On Oct 9, 11:40 am, hyperboogie
> Hello all
>
> I'm trying to edit a file with sed in the following way:
>
> sed 's/OldText/NewText/g' < myfile > myfile
>
> using the same file for output as the input actually deletes the
> content of myfile
>
> the only way I do this is like so:
>
> sed 's/OldText/NewText/g' < oldfile > newfile
> cat newfile > oldfile
>
> is there a way around this (using only sed)?
> Thanks
gnu sed has the "-i" or --in-place option which does this. others may
as well.
On Tue, 09 Oct 2007 15:40:25 +0000, hyperboogie wrote:
> Hello all
>
> I'm trying to edit a file with sed in the following way:
>
> sed 's/OldText/NewText/g' < myfile > myfile
>
> using the same file for output as the input actually deletes the content
> of myfile
>
> the only way I do this is like so:
>
> sed 's/OldText/NewText/g' < oldfile > newfile cat newfile > oldfile
>
> is there a way around this (using only sed)? Thanks
As others have pointed out GNU sed has a "-i" option.
The problem is that it is the shell that has opened stdout and stdin for
you long before the sed is started.
You can do
( rm -f myfile ; sed 's/../../' > myfile) < myfile
which works as follows.
A subshell is started with its standard input connected to myfile.
The subshell runs the rm command (it happens to have its stdin connected
to myfile, but this doesn't matter as rm does not read any input).
A new "myfile" is created and attached to stdout of sed. Sed then has the
old myfile on its stdin, and the new one on stdout, and it can then do
its work.
Note that if the processes are interrupted then you will probably have
lost the contents of "myfile".
2007-10-09, 15:40(-00), hyperboogie:
> Hello all
>
> I'm trying to edit a file with sed in the following way:
>
> sed 's/OldText/NewText/g' < myfile > myfile
>
> using the same file for output as the input actually deletes the
> content of myfile
>
> the only way I do this is like so:
>
> sed 's/OldText/NewText/g' < oldfile > newfile
> cat newfile > oldfile
>
> is there a way around this (using only sed)?
[...]
If the replacement text is exactly the same size as the replaced
text, you can do:
sed 's/OldText/NewText/g' < oldfile 1<> oldfile
FreeBSD and GNU seds have the -i option.
For portability, best is probably to use perl instead:
perl -pi -e 's/OldText/NewText/g' -- oldfile
or ed or ex, though those don't work in a stream fashion.
--
Stéphane
Stephane CHAZELAS wrote:
> If the replacement text is exactly the same size as the replaced
> text, you can do:
>
> sed 's/OldText/NewText/g' < oldfile 1<> oldfile
It works also if the replacement string is larger...
can you explain your above statement?
--
Jens
2007-10-09, 23:54(+02), Jens Binnewies:
> Stephane CHAZELAS wrote:
>
>> If the replacement text is exactly the same size as the replaced
>> text, you can do:
>>
>> sed 's/OldText/NewText/g' < oldfile 1<> oldfile
> It works also if the replacement string is larger...
> can you explain your above statement?
[...]
If it's larger, it may work but will generally not.
In the above, sed reads and writes from the same file. sed
generally uses stdio or any sort of buffered IO. It will read a
block of data, process the lines in that data, and the converted
lines will be written as a block.
If the oldfile is less than one block size (depends on the
system), then it will be OK. If not, then it won't because when
the first block is written, it will overwrite the next block to
be read.
Example, let's say the block size is 4 and you want to replace
"a"'s with "ab"'s in a file containing "a\na\na\na\na\na\n".
sed will read one block "a\na\n", process those two lines, and
thus write "ab\nab\n" to stdout. Therefore, the file will now
contain:
ab\nab\na\na\na\n, then it will read the second block of 4
bytes: "b\na\n"... you see what I mean?
--
Stéphane
Thank you all
I think I'll stick to the perl workaround ...
I was hoping there was a "standard" sed way of doing this
oh well ...
Thanks anyhow
:-)
On 2007-10-10, 16:21 GMT, hyperboogie wrote:
> I was hoping there was a "standard" sed way of doing this
> oh well ...
How much standard you need? -i parameter of (at least) GNU Sed is
not good enough, or did I miss some conditions you need to
satisfy?
MatÄj
On 2007-10-10, Matej Cepl wrote:
>
>
> On 2007-10-10, 16:21 GMT, hyperboogie wrote:
>> I was hoping there was a "standard" sed way of doing this
>> oh well ...
>
> How much standard you need? -i parameter of (at least) GNU Sed is
> not good enough, or did I miss some conditions you need to
> satisfy?
The -i option is not standard. AFAIK, only the GNU and FreeBSD
versions have it.
What's wrong with the traditional UNIX method of redirecting to a
different file and moving or copying it back over the original
(after making sure that it is correct)? That is what I recommend,
especially for someone who has to ask for help with the command.
--
Chris F.A. Johnson, author
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
===== My code in this post, if any, assumes the POSIX locale
===== and is released under the GNU General Public Licence
Chris F.A. Johnson
> On 2007-10-10, Matej Cepl wrote:
>>
>>
>> On 2007-10-10, 16:21 GMT, hyperboogie wrote:
>>> I was hoping there was a "standard" sed way of doing this
>>> oh well ...
>>
>> How much standard you need? -i parameter of (at least) GNU Sed is
>> not good enough, or did I miss some conditions you need to
>> satisfy?
>
> The -i option is not standard. AFAIK, only the GNU and FreeBSD
> versions have it.
And neither performs real in-place editing. Each of them rather
uses a temporary file, which is renamed to match the original file
name. AFAIK, only perl does in-place editing with the -i option.
> What's wrong with the traditional UNIX method of redirecting to a
> different file and moving or copying it back over the original
> (after making sure that it is correct)? That is what I recommend,
> especially for someone who has to ask for help with the command.
As mentioned above, what sed does is essentially the same, so the
recommendation is fully justified.
--
Kenan Kalajdzic
>
>
> How much standard you need? -i parameter of (at least) GNU Sed is
> not good enough, or did I miss some conditions you need to
> satisfy?
as stated by others it's not standard ... for example, most of the
machines I work on run Solaris on which sed does not include the i
option.
As I said, I think I'll stick to the perl workaround, which I've use
on several occasions before.
ps. - St=E9phane and Icarus Sparry - your exlanations were very
interesting and illuminating
Thanks :-)
On Oct 9, 9:39 pm, Icarus Sparry
> On Tue, 09 Oct 2007 15:40:25 +0000, hyperboogie wrote:
> > Hello all
>
> > I'm trying to edit a file with sed in the following way:
>
> > sed 's/OldText/NewText/g' < myfile > myfile
>
> > using the same file for output as the input actually deletes the content
> > of myfile
>
> > the only way I do this is like so:
>
> > sed 's/OldText/NewText/g' < oldfile > newfile cat newfile > oldfile
>
> > is there a way around this (using only sed)? Thanks
>
> As others have pointed out GNU sed has a "-i" option.
>
> The problem is that it is the shell that has opened stdout and stdin for
> you long before the sed is started.
>
> You can do
>
> ( rm -f myfile ; sed 's/../../' > myfile) < myfile
>
> which works as follows.
> A subshell is started with its standard input connected to myfile.
> The subshell runs the rm command (it happens to have its stdin connected
> to myfile, but this doesn't matter as rm does not read any input).
> A new "myfile" is created and attached to stdout of sed. Sed then has the
> old myfile on its stdin, and the new one on stdout, and it can then do
> its work.
>
> Note that if the processes are interrupted then you will probably have
> lost the contents of "myfile".
2007-10-11, 01:09(+02), Kenan Kalajdzic:
> Chris F.A. Johnson
>> On 2007-10-10, Matej Cepl wrote:
>>>
>>>
>>> On 2007-10-10, 16:21 GMT, hyperboogie wrote:
>>>> I was hoping there was a "standard" sed way of doing this
>>>> oh well ...
>>>
>>> How much standard you need? -i parameter of (at least) GNU Sed is
>>> not good enough, or did I miss some conditions you need to
>>> satisfy?
>>
>> The -i option is not standard. AFAIK, only the GNU and FreeBSD
>> versions have it.
>
> And neither performs real in-place editing. Each of them rather
> uses a temporary file, which is renamed to match the original file
> name. AFAIK, only perl does in-place editing with the -i option.
[...]
No, perl is the same. It writes a new file by the same name as
the original one's. It does something similar as:
{ rm file; sed 's/.../.../g' > file; } < file
except that it also tries to dupplicate the file ownership and
permissions.
--
Stéphane
Stephane CHAZELAS wrote:
> 2007-10-11, 01:09(+02), Kenan Kalajdzic:
>
>>Chris F.A. Johnson
>>
>>>On 2007-10-10, Matej Cepl wrote:
>>>
>>>>
>>>>On 2007-10-10, 16:21 GMT, hyperboogie wrote:
>>>>
>>>>>I was hoping there was a "standard" sed way of doing this
>>>>>oh well ...
>>>>
>>>>How much standard you need? -i parameter of (at least) GNU Sed is
>>>>not good enough, or did I miss some conditions you need to
>>>>satisfy?
>>>
>>> The -i option is not standard. AFAIK, only the GNU and FreeBSD
>>> versions have it.
>>
>>And neither performs real in-place editing. Each of them rather
>>uses a temporary file, which is renamed to match the original file
>>name. AFAIK, only perl does in-place editing with the -i option.
>
> [...]
>
> No, perl is the same. It writes a new file by the same name as
> the original one's. It does something similar as:
>
> { rm file; sed 's/.../.../g' > file; } < file
I hope that "similar" does not mean "as fatal as the above shell
construct is to a file" in case, e.g., the file system is nearly
full.
Janis
>
> except that it also tries to dupplicate the file ownership and
> permissions.
>
2007-10-16, 00:19(+02), Janis Papanagnou:
[...]
>> { rm file && sed 's/.../.../g' > file; } < file
>
> I hope that "similar" does not mean "as fatal as the above shell
> construct is to a file" in case, e.g., the file system is nearly
> full.
[...]
/mnt$ df -k .
Filesystem 1K-blocks Used Available Use% Mounted on
/home/chazelas/a 487 15 447 4% /mnt
/mnt$ yes | head -200000 > a
/mnt$ wc a
200000 200000 400000 a
/mnt$ df -k .
Filesystem 1K-blocks Used Available Use% Mounted on
/home/chazelas/a 487 409 53 89% /mnt
/mnt$ perl -pi -e 's/./../g' a
/mnt$ wc a
17749 17750 53248 a
It's worse, there's not even a warning.
--
Stéphane