Stripping ANSI escape characters from output

Hi
I have a program, let's call it 'toto', that generates text with color
(ansi escape codes)
I want to keep a log of toto's execution, and display its output with
the colors in real time, so at the moment, I do (in tcsh) :

toto |& tee execution.log

But when I read the log with nedit for example, the color codes
present in the log alter the readability of the file.
I could make a script that strips the escape characters in the log
file after execution, but I would prefer if the escape sequences were
never written in the file, so that no user intervention is needed.

I have an idea : we could pipe the output of toto to sed, but I don't
know if sed is able to print the input data both on the screen
(unaltered) and to a file (with escape chars removed)
Do you have a little trick that could solve my problem ?

Thanks to everyone and have a nice day !
Philippe

Re: Stripping ANSI escape characters from output

pmaire@gmail.com wrote:
>
> I have a program, let's call it 'toto', that generates text with color
> (ansi escape codes)
> I want to keep a log of toto's execution, and display its output with
> the colors in real time, so at the moment, I do (in tcsh) :
>
> toto |& tee execution.log
>
> But when I read the log with nedit for example, the color codes
> present in the log alter the readability of the file.
> I could make a script that strips the escape characters in the log
> file after execution, but I would prefer if the escape sequences were
> never written in the file, so that no user intervention is needed.
>
> I have an idea : we could pipe the output of toto to sed, but I don't
> know if sed is able to print the input data both on the screen
> (unaltered) and to a file (with escape chars removed)
> Do you have a little trick that could solve my problem ?

[bash & GNU sed]

echo 123hello | sed -ne "p;s/123//w/tmp/output.txt"
123hello

$ cat /tmp/output.txt
hello

--
Best regards | Be nice to America or they'll bring democracy to
Cyrus | your country.

Re: Stripping ANSI escape characters from output

pmaire@gmail.com wrote:
>
> I have a program, let's call it 'toto', that generates text with color
> (ansi escape codes)
> I want to keep a log of toto's execution, and display its output with
> the colors in real time, so at the moment, I do (in tcsh) :
>
> toto |& tee execution.log
>
> But when I read the log with nedit for example, the color codes
> present in the log alter the readability of the file.
> I could make a script that strips the escape characters in the log
> file after execution, but I would prefer if the escape sequences were
> never written in the file, so that no user intervention is needed.
>
> I have an idea : we could pipe the output of toto to sed, but I don't
> know if sed is able to print the input data both on the screen
> (unaltered) and to a file (with escape chars removed)
> Do you have a little trick that could solve my problem ?

[bash & GNU sed]

$ echo 123hello | sed -ne "p;s/123//w/tmp/output.txt"
123hello

$ cat /tmp/output.txt
hello

--
Best regards | Be nice to America or they'll bring democracy to
Cyrus | your country.

Re: Stripping ANSI escape characters from output

pmaire@gmail.com wrote:
> Hi
> I have a program, let's call it 'toto', that generates text with color
> (ansi escape codes)
> I want to keep a log of toto's execution, and display its output with
> the colors in real time, so at the moment, I do (in tcsh) :
>
> toto |& tee execution.log
>
> But when I read the log with nedit for example, the color codes
> present in the log alter the readability of the file.
> I could make a script that strips the escape characters in the log
> file after execution, but I would prefer if the escape sequences were
> never written in the file, so that no user intervention is needed.
>
> I have an idea : we could pipe the output of toto to sed, but I don't
> know if sed is able to print the input data both on the screen
> (unaltered) and to a file (with escape chars removed)
> Do you have a little trick that could solve my problem ?
>
> Thanks to everyone and have a nice day !
> Philippe
>

toto | awk '{print;gsub(/[[:control:]]/,"");print > "log";system("")}'

system() flushes the buffer. In GNU awk you could use fflush().

Ed.

Re: Stripping ANSI escape characters from output

Cyrus Kriticos wrote:
>
> [bash & GNU sed]
>
> $ echo 123hello | sed -ne "p;s/123//w/tmp/output.txt"
> 123hello
>
> $ cat /tmp/output.txt
> hello

sorry, does not work correct.

--
Best regards | Be nice to America or they'll bring democracy to
Cyrus | your country.

Re: Stripping ANSI escape characters from output

On 2007-11-06, pmaire@gmail.com wrote:
> Hi
> I have a program, let's call it 'toto', that generates text with color
> (ansi escape codes)
> I want to keep a log of toto's execution, and display its output with
> the colors in real time, so at the moment, I do (in tcsh) :
>
> toto |& tee execution.log
>
> But when I read the log with nedit for example, the color codes
> present in the log alter the readability of the file.
> I could make a script that strips the escape characters in the log
> file after execution, but I would prefer if the escape sequences were
> never written in the file, so that no user intervention is needed.
>
> I have an idea : we could pipe the output of toto to sed, but I don't
> know if sed is able to print the input data both on the screen
> (unaltered) and to a file (with escape chars removed)
> Do you have a little trick that could solve my problem ?
>
> Thanks to everyone and have a nice day !
> Philippe
>
toto |& tee /dev/tty | col -b > execution.log

Re: Stripping ANSI escape characters from output

On 6 nov, 14:12, Ed Morton wrote:
> pma...@gmail.com wrote:
> > Hi
> > I have a program, let's call it 'toto', that generates text with color
> > (ansi escape codes)
> > I want to keep a log of toto's execution, and display its output with
> > the colors in real time, so at the moment, I do (in tcsh) :
>
> > toto |& tee execution.log
>
> > But when I read the log with nedit for example, the color codes
> > present in the log alter the readability of the file.
> > I could make a script that strips the escape characters in the log
> > file after execution, but I would prefer if the escape sequences were
> > never written in the file, so that no user intervention is needed.
>
> > I have an idea : we could pipe the output of toto to sed, but I don't
> > know if sed is able to print the input data both on the screen
> > (unaltered) and to a file (with escape chars removed)
> > Do you have a little trick that could solve my problem ?
>
> > Thanks to everyone and have a nice day !
> > Philippe
>
> toto | awk '{print;gsub(/[[:control:]]/,"");print > "log";system("")}'
>
> system() flushes the buffer. In GNU awk you could use fflush().
>
> Ed.- Masquer le texte des messages pr=E9c=E9dents -
>
> - Afficher le texte des messages pr=E9c=E9dents -
Hi, thanks for your quick answer ! However, the command does not work:

awk: syntax error near line 1
awk: illegal statement near line 1

Philippe

Re: Stripping ANSI escape characters from output

On 6 nov, 16:42, Bill Marcum wrote:
> On 2007-11-06, pma...@gmail.com wrote:
>
>
>
> > Hi
> > I have a program, let's call it 'toto', that generates text with color
> > (ansi escape codes)
> > I want to keep a log of toto's execution, and display its output with
> > the colors in real time, so at the moment, I do (in tcsh) :
>
> > toto |& tee execution.log
>
> > But when I read the log with nedit for example, the color codes
> > present in the log alter the readability of the file.
> > I could make a script that strips the escape characters in the log
> > file after execution, but I would prefer if the escape sequences were
> > never written in the file, so that no user intervention is needed.
>
> > I have an idea : we could pipe the output of toto to sed, but I don't
> > know if sed is able to print the input data both on the screen
> > (unaltered) and to a file (with escape chars removed)
> > Do you have a little trick that could solve my problem ?
>
> > Thanks to everyone and have a nice day !
> > Philippe
>
> toto |& tee /dev/tty | col -b > execution.log- Masquer le texte des messa=
ges pr=E9c=E9dents -
>
> - Afficher le texte des messages pr=E9c=E9dents -

Hi, thanks for your answer.
col does not seem to work well : in my log file, I get
-----
34m1mInfo: test
0m35m1mWarning: tttiti
0m31m1mError: myerror
0mtest2
-----
However, the tee to /dev/tty is a good idea. I will try to use sed
instead of col in order to strip the escape sequences.
Regards
Philippe

Re: Stripping ANSI escape characters from output

pmaire@gmail.com wrote:
> On 6 nov, 14:12, Ed Morton wrote:
>
>>pma...@gmail.com wrote:
>>
>>>Hi
>>>I have a program, let's call it 'toto', that generates text with color
>>>(ansi escape codes)
>>>I want to keep a log of toto's execution, and display its output with
>>>the colors in real time, so at the moment, I do (in tcsh) :
>>
>>>toto |& tee execution.log
>>
>>>But when I read the log with nedit for example, the color codes
>>>present in the log alter the readability of the file.
>>>I could make a script that strips the escape characters in the log
>>>file after execution, but I would prefer if the escape sequences were
>>>never written in the file, so that no user intervention is needed.
>>
>>>I have an idea : we could pipe the output of toto to sed, but I don't
>>>know if sed is able to print the input data both on the screen
>>>(unaltered) and to a file (with escape chars removed)
>>>Do you have a little trick that could solve my problem ?
>>
>>>Thanks to everyone and have a nice day !
>>>Philippe
>>
>>toto | awk '{print;gsub(/[[:control:]]/,"");print > "log";system("")}'
>>
>>system() flushes the buffer. In GNU awk you could use fflush().
>>
>> Ed.- Masquer le texte des messages précédents -
>>
>>- Afficher le texte des messages précédents -
>
> Hi, thanks for your quick answer ! However, the command does not work:
>
> awk: syntax error near line 1
> awk: illegal statement near line 1

[[:cntrl:]]

Janis

>
> Philippe
>

Re: Stripping ANSI escape characters from output

On 6 nov, 17:26, "pma...@gmail.com" wrote:
> On 6 nov, 16:42, Bill Marcum wrote:
>
>
>
>
>
> > On 2007-11-06, pma...@gmail.com wrote:
>
> > > Hi
> > > I have a program, let's call it 'toto', that generates text with color
> > > (ansi escape codes)
> > > I want to keep a log of toto's execution, and display its output with
> > > the colors in real time, so at the moment, I do (in tcsh) :
>
> > > toto |& tee execution.log
>
> > > But when I read the log with nedit for example, the color codes
> > > present in the log alter the readability of the file.
> > > I could make a script that strips the escape characters in the log
> > > file after execution, but I would prefer if the escape sequences were
> > > never written in the file, so that no user intervention is needed.
>
> > > I have an idea : we could pipe the output of toto to sed, but I don't
> > > know if sed is able to print the input data both on the screen
> > > (unaltered) and to a file (with escape chars removed)
> > > Do you have a little trick that could solve my problem ?
>
> > > Thanks to everyone and have a nice day !
> > > Philippe
>
> > toto |& tee /dev/tty | col -b > execution.log- Masquer le texte des mes=
sages pr=E9c=E9dents -
>
> > - Afficher le texte des messages pr=E9c=E9dents -
>
> Hi, thanks for your answer.
> col does not seem to work well : in my log file, I get
> -----
> 34m1mInfo: test
> 0m35m1mWarning: tttiti
> 0m31m1mError: myerror
> 0mtest2
> -----
> However, the tee to /dev/tty is a good idea. I will try to use sed
> instead of col in order to strip the escape sequences.
> Regards
> Philippe- Masquer le texte des messages pr=E9c=E9dents -
>
> - Afficher le texte des messages pr=E9c=E9dents -

It works fine with
toto |& tee /dev/tty | sed 's/^[.*m//g' > test.log

There is still a problem...
My actual toto program can take few hours to run. If during the run, I
kill toto with Ctrl+C, the log file is empty, whereas with my previous
solution "toto |& tee test.log " the log file was written
continuously.
Is it a trouble with sed that cannot output continuously ?
Philippe

Re: Stripping ANSI escape characters from output

Janis Papanagnou wrote:
> pmaire@gmail.com wrote:
>
>> On 6 nov, 14:12, Ed Morton wrote:
>>
>>> pma...@gmail.com wrote:
>>>
>>>> Hi
>>>> I have a program, let's call it 'toto', that generates text with color
>>>> (ansi escape codes)
>>>> I want to keep a log of toto's execution, and display its output with
>>>> the colors in real time, so at the moment, I do (in tcsh) :
>>>
>>>
>>>> toto |& tee execution.log
>>>
>>>
>>>> But when I read the log with nedit for example, the color codes
>>>> present in the log alter the readability of the file.
>>>> I could make a script that strips the escape characters in the log
>>>> file after execution, but I would prefer if the escape sequences were
>>>> never written in the file, so that no user intervention is needed.
>>>
>>>
>>>> I have an idea : we could pipe the output of toto to sed, but I don't
>>>> know if sed is able to print the input data both on the screen
>>>> (unaltered) and to a file (with escape chars removed)
>>>> Do you have a little trick that could solve my problem ?
>>>
>>>
>>>> Thanks to everyone and have a nice day !
>>>> Philippe
>>>
>>>
>>> toto | awk '{print;gsub(/[[:control:]]/,"");print > "log";system("")}'
>>>
>>> system() flushes the buffer. In GNU awk you could use fflush().
>>>
>>> Ed.- Masquer le texte des messages précédents -
>>>
>>> - Afficher le texte des messages précédents -
>>
>>
>> Hi, thanks for your quick answer ! However, the command does not work:
>>
>> awk: syntax error near line 1
>> awk: illegal statement near line 1
>
>
> [[:cntrl:]]
>

If that's not it, you're probably using old, broken awk. Use GNU awk
(gawk), New awk (nawk), or /usr/xpg4/bin/awk on Solaris.

Ed.

Re: Stripping ANSI escape characters from output

> It works fine with
> toto |& tee /dev/tty | sed 's/^[.*m//g' > test.log
>
> There is still a problem...
> My actual toto program can take few hours to run. If during the run, I
> kill toto with Ctrl+C, the log file is empty, whereas with my previous
> solution "toto |& tee test.log " the log file was written
> continuously.
> Is it a trouble with sed that cannot output continuously ?
> Philippe- Masquer le texte des messages pr=E9c=E9dents -
>
> - Afficher le texte des messages pr=E9c=E9dents -

All right, I finally got my solution :
toto |& tee /dev/tty | tclsh stripcolor.tcl > test.log

and in stripcolor.tcl :
while {1} {
gets stdin data
regsub -all -line {\[[0-9][0-9]*m} $data "" data
puts stdout $data
flush stdout
if {[eof stdin]} { exit 0 }
}

Thanks to all for your help!

Re: Stripping ANSI escape characters from output

2007-11-07, 01:19(-08), pmaire@gmail.com:
>
>> It works fine with
>> toto |& tee /dev/tty | sed 's/^[.*m//g' > test.log
>>
>> There is still a problem...
>> My actual toto program can take few hours to run. If during the run, I
>> kill toto with Ctrl+C, the log file is empty, whereas with my previous
>> solution "toto |& tee test.log " the log file was written
>> continuously.
>> Is it a trouble with sed that cannot output continuously ?
>> Philippe- Masquer le texte des messages précédents -
>>
>> - Afficher le texte des messages précédents -
>
> All right, I finally got my solution :
> toto |& tee /dev/tty | tclsh stripcolor.tcl > test.log
>
> and in stripcolor.tcl :
> while {1} {
> gets stdin data
> regsub -all -line {\[[0-9][0-9]*m} $data "" data
> puts stdout $data
> flush stdout
> if {[eof stdin]} { exit 0 }
> }
>
> Thanks to all for your help!

Or sed 's/^[\[[0-9;]*m//g'

(where ^[ is the character obtained by typing )

Or:

esc=$(printf '\033')
sed "s/$esc\[[0-9;]*m//g"

--
Stéphane