Search & replace, but increment with every replace

Hi All,

I have a file that contains 100's of such lines:

insert into data (1,'test');
insert into data (1,'hj');
insert into data (1,'tejkdsst');
insert into data (1,'hjsd');
insert into data (1,'tdfgest');
insert into data (1,'dsffshj');

Now I want to replace all the 1's with a sequential number so that the
final output would be like this:

insert into data (1,'test');
insert into data (2,'hj');
insert into data (3,'tejkdsst');
insert into data (4,'hjsd');
insert into data (5,'tdfgest');
insert into data (6,'dsffshj');


Can this be achieved using sed in a single search replace operation -
I know I can write a shell script that goes line by line and changes &
increments the number.

Also - what would the %s command (if possible) I can give in vi (or
Vim) that would achieve this.

Thanks.

Re: Search & replace, but increment with every replace

Anoop wrote:
[...]
> insert into data (1,'test');
> insert into data (1,'hj');
> insert into data (1,'tejkdsst');
> insert into data (1,'hjsd');
> insert into data (1,'tdfgest');
> insert into data (1,'dsffshj');
>
> Now I want to replace all the 1's with a sequential number so that the
> final output would be like this:
>
> insert into data (1,'test');
> insert into data (2,'hj');
> insert into data (3,'tejkdsst');
> insert into data (4,'hjsd');
> insert into data (5,'tdfgest');
> insert into data (6,'dsffshj');
[...]

If you can use awk:


awk 'sub(/1/,++c)' filename


Use nawk or /usr/xpg4/bin/awk on Solaris.


P.S. You may need to adjust the pattern
if you have more than one 1 in the record.


Dimitre

Re: Search & replace, but increment with every replace

On 2007-10-27, Anoop wrote:
> Hi All,
>
> I have a file that contains 100's of such lines:
>
> insert into data (1,'test');
> insert into data (1,'hj');
> insert into data (1,'tejkdsst');
> insert into data (1,'hjsd');
> insert into data (1,'tdfgest');
> insert into data (1,'dsffshj');
>
> Now I want to replace all the 1's with a sequential number so that the
> final output would be like this:
>
> insert into data (1,'test');
> insert into data (2,'hj');
> insert into data (3,'tejkdsst');
> insert into data (4,'hjsd');
> insert into data (5,'tdfgest');
> insert into data (6,'dsffshj');
>
>
> Can this be achieved using sed in a single search replace operation -
> I know I can write a shell script that goes line by line and changes &
> increments the number.
>
I would use awk for a search and replace that involves arithmetic.
#!/bin/awk -f
BEGIN{num=1}
/insert into data/{sub(/1,/,num++ ",")}
{print}

> Also - what would the %s command (if possible) I can give in vi (or
> Vim) that would achieve this.
>
> Thanks.
>

Re: Search & replace, but increment with every replace

On Oct 27, 3:04 pm, Bill Marcum wrote:
> On 2007-10-27, Anoop wrote:
>
> > Hi All,
>
> > I have a file that contains 100's of such lines:
>
> > insert into data (1,'test');
> > insert into data (1,'hj');
> > insert into data (1,'tejkdsst');
> > insert into data (1,'hjsd');
> > insert into data (1,'tdfgest');
> > insert into data (1,'dsffshj');
>
> > Now I want to replace all the 1's with a sequential number so that the
> > final output would be like this:
>
> > insert into data (1,'test');
> > insert into data (2,'hj');
> > insert into data (3,'tejkdsst');
> > insert into data (4,'hjsd');
> > insert into data (5,'tdfgest');
> > insert into data (6,'dsffshj');
>
> > Can this be achieved using sed in a single search replace operation -
> > I know I can write a shell script that goes line by line and changes &
> > increments the number.
>
> I would use awk for a search and replace that involves arithmetic.
> #!/bin/awk -f
> BEGIN{num=1}
> /insert into data/{sub(/1,/,num++ ",")}
> {print}
>
> > Also - what would the %s command (if possible) I can give in vi (or
> > Vim) that would achieve this.
>
> > Thanks.

Yes - I could use awk - but it seems it is possible using sed.

Doing a google search I found this link:
http://skywayradio.com/tech/linux/sed.php

with this content:

# number each line left alignment. (See note on '\t'.)
sed = #1 | sed "N;s/\n/\t/" >##1

# number each line (right-aligned)
sed = #1 | sed "N;s/^/ /;s/ *\(.\{6,\}\)\n/\1 /" >##2

# number each not blank line
sed "/./=" #1 | sed "/./N;s/\n/ /" >##3

#1 ##1 ##2 ##3
=== ----------- ----------- -----
aaa 1 aaa 1 aaa 1 aaa
bbb 2 bbb 2 bbb 2 bbb
3 3
ccc 4 ccc 4 ccc 4 ccc

But the solution only replaces the leading value, I am trying to
explore if I can modify this to help in my case..

thanks for the solution - I am trying it now.

Thanks!

Re: Search & replace, but increment with every replace

On 10/27/2007 3:19 PM, Anoop wrote:
> On Oct 27, 3:04 pm, Bill Marcum wrote:
>
>>On 2007-10-27, Anoop wrote:
>>
>>
>>>Hi All,
>>
>>>I have a file that contains 100's of such lines:
>>
>>>insert into data (1,'test');
>>>insert into data (1,'hj');
>>>insert into data (1,'tejkdsst');
>>>insert into data (1,'hjsd');
>>>insert into data (1,'tdfgest');
>>>insert into data (1,'dsffshj');
>>
>>>Now I want to replace all the 1's with a sequential number so that the
>>>final output would be like this:
>>
>>>insert into data (1,'test');
>>>insert into data (2,'hj');
>>>insert into data (3,'tejkdsst');
>>>insert into data (4,'hjsd');
>>>insert into data (5,'tdfgest');
>>>insert into data (6,'dsffshj');
>>
>>>Can this be achieved using sed in a single search replace operation -
>>>I know I can write a shell script that goes line by line and changes &
>>>increments the number.
>>
>>I would use awk for a search and replace that involves arithmetic.
>>#!/bin/awk -f
>>BEGIN{num=1}
>>/insert into data/{sub(/1,/,num++ ",")}
>>{print}
>>
>>
>>>Also - what would the %s command (if possible) I can give in vi (or
>>>Vim) that would achieve this.
>>
>>>Thanks.
>>
>
> Yes - I could use awk - but it seems it is possible using sed.

> But the solution only replaces the leading value, I am trying to
> explore if I can modify this to help in my case..

That's a bit like saying "Thanks for suggesting a pneumatic drill, but it seems
it's posisble to break up this concrete with a toothpick. I'm going to try to
make that work..". Sed is a fine tool, but just not the right one for this job.

Ed.

Re: Search & replace, but increment with every replace

Radoulov, Dimitre wrote:
> Anoop wrote:
> [...]
>
>> insert into data (1,'test');
>> insert into data (1,'hj');
>> insert into data (1,'tejkdsst');
>> insert into data (1,'hjsd');
>> insert into data (1,'tdfgest');
>> insert into data (1,'dsffshj');
>>
>> Now I want to replace all the 1's with a sequential number so that the
>> final output would be like this:
>>
>> insert into data (1,'test');
>> insert into data (2,'hj');
>> insert into data (3,'tejkdsst');
>> insert into data (4,'hjsd');
>> insert into data (5,'tdfgest');
>> insert into data (6,'dsffshj');
>
> [...]
>
> If you can use awk:
>
>
> awk 'sub(/1/,++c)' filename
>
>
> Use nawk or /usr/xpg4/bin/awk on Solaris.
>
>
> P.S. You may need to adjust the pattern
> if you have more than one 1 in the record.

Why do you think so? sub() will replace just the first one, so your
solution is fine in that respect. The bigger problem with your code
is that it will print any lines (and only those) that match a /1/,
and another problem that it will increment the counter whether there
is a match or not.

Janis

>
>
> Dimitre

Re: Search & replace, but increment with every replace

Anoop wrote:
>
> I have a file that contains 100's of such lines:
>
> insert into data (1,'test');
> insert into data (1,'hj');
> insert into data (1,'tejkdsst');
> insert into data (1,'hjsd');
> insert into data (1,'tdfgest');
> insert into data (1,'dsffshj');
>
> Now I want to replace all the 1's with a sequential number so that the
> final output would be like this:
>
> insert into data (1,'test');
> insert into data (2,'hj');
> insert into data (3,'tejkdsst');
> insert into data (4,'hjsd');
> insert into data (5,'tdfgest');
> insert into data (6,'dsffshj');
>
> Can this be achieved using sed in a single search replace operation -
> I know I can write a shell script that goes line by line and changes &
> increments the number.

If you don't mind using Perl:

$ echo "insert into data (1,'test');
insert into data (1,'hj');
insert into data (1,'tejkdsst');
insert into data (1,'hjsd');
insert into data (1,'tdfgest');
insert into data (1,'dsffshj');" | perl -pe's/(?<=insert into data
\()(\d+)/++$count/e'
insert into data (1,'test');
insert into data (2,'hj');
insert into data (3,'tejkdsst');
insert into data (4,'hjsd');
insert into data (5,'tdfgest');
insert into data (6,'dsffshj');



John
--
use Perl;
program
fulfillment

Re: Search & replace, but increment with every replace

Janis Papanagnou wrote:
> Radoulov, Dimitre wrote:
>> Anoop wrote:
>> [...]
>>
>>> insert into data (1,'test');
>>> insert into data (1,'hj');
>>> insert into data (1,'tejkdsst');
>>> insert into data (1,'hjsd');
>>> insert into data (1,'tdfgest');
>>> insert into data (1,'dsffshj');
>>>
>>> Now I want to replace all the 1's with a sequential number so that the
>>> final output would be like this:
>>>
>>> insert into data (1,'test');
>>> insert into data (2,'hj');
>>> insert into data (3,'tejkdsst');
>>> insert into data (4,'hjsd');
>>> insert into data (5,'tdfgest');
>>> insert into data (6,'dsffshj');
>>
>> [...]
>>
>> If you can use awk:
>>
>>
>> awk 'sub(/1/,++c)' filename
>>
>>
>> Use nawk or /usr/xpg4/bin/awk on Solaris.
>>
>>
>> P.S. You may need to adjust the pattern
>> if you have more than one 1 in the record.
>
> Why do you think so? sub() will replace just the first one, so your
> solution is fine in that respect.
[...]

% echo "insert into data1 (1,'test');
insert into data1 (1,'hj');"|awk 'sub(/1/,++c)'
insert into data1 (1,'test');
insert into data2 (1,'hj');



> The bigger problem with your code
> is that it will print any lines (and only those) that match a /1/,
> and another problem that it will increment the counter whether there
> is a match or not.
[...]


Sure,
my response was based on the OP sample.



Dimitre

Re: Search & replace, but increment with every replace

Radoulov, Dimitre wrote:
> Janis Papanagnou wrote:
>
>> Radoulov, Dimitre wrote:
>>
>>> Anoop wrote:
>>> [...]
>>>
>>>> insert into data (1,'test');
>>>> insert into data (1,'hj');
>>>> insert into data (1,'tejkdsst');
>>>> insert into data (1,'hjsd');
>>>> insert into data (1,'tdfgest');
>>>> insert into data (1,'dsffshj');
>>>>
>>>> Now I want to replace all the 1's with a sequential number so that the
>>>> final output would be like this:
>>>>
>>>> insert into data (1,'test');
>>>> insert into data (2,'hj');
>>>> insert into data (3,'tejkdsst');
>>>> insert into data (4,'hjsd');
>>>> insert into data (5,'tdfgest');
>>>> insert into data (6,'dsffshj');
>>>
>>>
>>> [...]
>>>
>>> If you can use awk:
>>>
>>>
>>> awk 'sub(/1/,++c)' filename
>>>
>>>
>>> Use nawk or /usr/xpg4/bin/awk on Solaris.
>>>
>>>
>>> P.S. You may need to adjust the pattern
>>> if you have more than one 1 in the record.
>>
>>
>> Why do you think so? sub() will replace just the first one, so your
>> solution is fine in that respect.
>
> [...]
>
> % echo "insert into data1 (1,'test');
> insert into data1 (1,'hj');"|awk 'sub(/1/,++c)'
> insert into data1 (1,'test');
> insert into data2 (1,'hj');

Below you say you assumed the OP's sample. In this case it's possible
that the actual _data_ may contain numbers like data (1,'test1');
which is no problem as I mentioned. The contents of the actual strings
in the data is what varies dynamically, while the table and attribute
names are mostly well known and fixed.

>
>
>
>> The bigger problem with your code
>> is that it will print any lines (and only those) that match a /1/,
>> and another problem that it will increment the counter whether there
>> is a match or not.
>
> [...]
>
>
> Sure,
> my response was based on the OP sample.

It's very dangerous to assume here that the complete file is _exactly
structured_ as in the few lines example. Such files have often headers
(e.g. for version control), empty lines, or even other SQL commands. So
this is indeed the bigger problem with your suggestion.

But the OP already got other solutions, anyway.

Janis

>
>
>
> Dimitre

Re: Search & replace, but increment with every replace

Janis Papanagnou wrote:
> Radoulov, Dimitre wrote:
>> Janis Papanagnou wrote:
>>
>>> Radoulov, Dimitre wrote:
>>>
>>>> Anoop wrote:
>>>> [...]
>>>>
>>>>> insert into data (1,'test');
>>>>> insert into data (1,'hj');
>>>>> insert into data (1,'tejkdsst');
>>>>> insert into data (1,'hjsd');
>>>>> insert into data (1,'tdfgest');
>>>>> insert into data (1,'dsffshj');
>>>>>
>>>>> Now I want to replace all the 1's with a sequential number so that the
>>>>> final output would be like this:
>>>>>
>>>>> insert into data (1,'test');
>>>>> insert into data (2,'hj');
>>>>> insert into data (3,'tejkdsst');
>>>>> insert into data (4,'hjsd');
>>>>> insert into data (5,'tdfgest');
>>>>> insert into data (6,'dsffshj');
>>>>
>>>>
>>>> [...]
>>>>
>>>> If you can use awk:
>>>>
>>>>
>>>> awk 'sub(/1/,++c)' filename
>>>>
>>>>
>>>> Use nawk or /usr/xpg4/bin/awk on Solaris.
>>>>
>>>>
>>>> P.S. You may need to adjust the pattern
>>>> if you have more than one 1 in the record.
>>>
>>>
>>> Why do you think so? sub() will replace just the first one, so your
>>> solution is fine in that respect.
>>
>> [...]
>>
>> % echo "insert into data1 (1,'test');
>> insert into data1 (1,'hj');"|awk 'sub(/1/,++c)'
>> insert into data1 (1,'test');
>> insert into data2 (1,'hj');
>
> Below you say you assumed the OP's sample. In this case it's possible
> that the actual _data_ may contain numbers like data (1,'test1');
> which is no problem as I mentioned. The contents of the actual strings
> in the data is what varies dynamically, while the table and attribute
> names are mostly well known and fixed.
[...]
>>> The bigger problem with your code
>>> is that it will print any lines (and only those) that match a /1/,
>>> and another problem that it will increment the counter whether there
>>> is a match or not.
>>
>> [...]
>>
>>
>> Sure,
>> my response was based on the OP sample.
>
> It's very dangerous to assume here that the complete file is _exactly
> structured_ as in the few lines example. Such files have often headers
> (e.g. for version control), empty lines, or even other SQL commands. So
> this is indeed the bigger problem with your suggestion.
[...]

I agree.
Thank you for pointing out the problems
and clarifying the right assumptions!



Regards
Dimitre

Re: Search & replace, but increment with every replace

Anoop,

I see that you are adding a numeric value to your SQL script, and
incrementing
it for each row/record inserted. This IS a valid approach, and using
awk to preset
the numeric value does seem pretty effective... BUT as a more flexible
approach,
you may want to consider usign a "Sequence Number" within the database
itself
and having the database generate these incremental numbers.

I'm not sure what flavor of database you are using (Oracle / DB2 /
Sybase / Postgresss / SQL-Server )
but several of these do support having a unique incrementing number be
generated easily in the DB.

This would mean that your insert statements don't have to contain a
hard-coded numeric value at all,
but would just wait until insert time to get their assigned number.
You could also implement this as
a database trigger or procedure, depending on your preference.

In Oracle for instance, you could say:
SQL> Create Sequence ID_NUM start with 1 increment by 1 maxvalue
8000000;

Now, your file that contains 100's of such lines, would read more
like:
>
> insert into data (ID_NUM.NextVal,'test');
> insert into data (ID_NUM.NextVal,'hj');
> insert into data (ID_NUM.NextVal,'tejkdsst');
> insert into data (ID_NUM.NextVal,'hjsd');
> insert into data (ID_NUM.NextVal,'tdfgest');
> insert into data (ID_NUM.NextVal,'dsffshj');
>
> So, now you don't NEED to replace all the 1's with a hard-coded sequential number
> before you run the SQL command script.
>
> Can this be achieved using sed in a single search replace operation -
> I know I can write a shell script that goes line by line and changes &
> increments the number.
>
> As for using "vi" directly, to create these incremental numbers...
> I don't think "vi" has the ability to do math on a number, to even increment
> from 1 to 2,3,4 etc... so you would have to use another mechanism to make
> the numbers themselves, and then use 'vi' to set those numbers in place.
>
> So here's an approach: Let's say that you have your example text in "vi"

insert into data (1,'test');
insert into data (1,'hj');
insert into data (1,'tejkdsst');
insert into data (1,'hjsd');
insert into data (1,'tdfgest');
insert into data (1,'dsffshj');

#-- Go to the top of your file, and pipe through "cat -n" to generate
line numbers:

1G !G cat -n

#-- which gives you:

1 insert into data (1,'test');
2 insert into data (1,'hj');
3 insert into data (1,'tejkdsst');
4 insert into data (1,'hjsd');
6 insert into data (1,'tdfgest');
7 insert into data (1,'dsffshj');

#-- Now, let's put those numbers in place, using a substitute command:
> what would the %s command (if possible) I can give in vi (or Vim) that would achieve this??

:%s/^[ ]*\([0-9]*\)[ ]*\(ins.*(\)1/\2 \1/

#-- That would do it... It says:
skip the spaces, remember the numbers as \1
skip more spaces, remember everything from "ins" to the "("
drop the "1" that you want to replace,
stick in the \2 pattern of "insert...(" substitute in the \1 number .

Re: Search & replace, but increment with every replace

On Oct 27, 1:57 pm, "Radoulov, Dimitre" wrote:
> Anoop wrote:
>
> [...]
>
>
>
> > insert into data (1,'test');
> > insert into data (1,'hj');
> > insert into data (1,'tejkdsst');
> > insert into data (1,'hjsd');
> > insert into data (1,'tdfgest');
> > insert into data (1,'dsffshj');
>
> > Now I want to replace all the 1's with a sequential number so that the
> > final output would be like this:
>
> > insert into data (1,'test');
> > insert into data (2,'hj');
> > insert into data (3,'tejkdsst');
> > insert into data (4,'hjsd');
> > insert into data (5,'tdfgest');
> > insert into data (6,'dsffshj');
>
> [...]
>
> If you can use awk:
>
> awk 'sub(/1/,++c)' filename
>
> Use nawk or /usr/xpg4/bin/awk on Solaris.
>
> P.S. You may need to adjust the pattern
> if you have more than one 1 in the record.
>
> Dimitre- Hide quoted text -
>
> - Show quoted text -

Sweet! Thanks I was able to use that string to fit my needs for
another issue.
I had to replace field one of a file with a new set of incrementing
numbers starting after 1 million.
awk 'sub($1,1000000++C)' > file.new

Great stuff thanks!

Re: Search & replace, but increment with every replace

On Nov 20, 12:35 pm, Tonagon wrote:
> On Oct 27, 1:57 pm, "Radoulov, Dimitre" wrote:
>
>
>
>
>
> > Anoop wrote:
>
> > [...]
>
> > > insert into data (1,'test');
> > > insert into data (1,'hj');
> > > insert into data (1,'tejkdsst');
> > > insert into data (1,'hjsd');
> > > insert into data (1,'tdfgest');
> > > insert into data (1,'dsffshj');
>
> > > Now I want to replace all the 1's with a sequential number so that the
> > > final output would be like this:
>
> > > insert into data (1,'test');
> > > insert into data (2,'hj');
> > > insert into data (3,'tejkdsst');
> > > insert into data (4,'hjsd');
> > > insert into data (5,'tdfgest');
> > > insert into data (6,'dsffshj');
>
> > [...]
>
> > If you can useawk:
>
> >awk'sub(/1/,++c)' filename
>
> > Use nawk or /usr/xpg4/bin/awkon Solaris.
>
> > P.S. You may need to adjust the pattern
> > if you have more than one 1 in the record.
>
> > Dimitre- Hide quoted text -
>
> > - Show quoted text -
>
> Sweet! Thanks I was able to use that string to fit my needs for
> another issue.
> I had to replace field one of a file with a new set of incrementing
> numbers starting after 1 million.awk'sub($1,1000000++C)' > file.new
>
> Great stuff thanks!- Hide quoted text -
>
> - Show quoted text -

Crud, I spoke to soon. It replaces the numbers in field 1, and works
fine for up to 9 digits, but that is all.
It seems to leave the original digits alone and when it reached 10 is
adds another digit to the end of the number.
Say the file has 2000 entries.
We can start it at 100 (for example) and it is fine up to 109, but
then instead of 110 I get 1010. Then it is fine up to 1099 but after
that I get 10100.
Does anyone have a way for me to get around this littel snafu?
I need the file to just count up from 100 to 2,100 (two thousand lines
in the example file).
field is is already strictly numeric, and already in order, but it
needs to be changed from one range to another.
Originally from 800,000-802,000 to 1,000,000-1,002,000.
Thanks for any suggestions!

My profile is old, I need to update my email address for my new work
email, so currently I can only see responses through this posting.

Re: Search & replace, but increment with every replace

On 2007-11-27, Tonagon wrote:
>
> On Nov 20, 12:35 pm, Tonagon wrote:
>> Sweet! Thanks I was able to use that string to fit my needs for
>> another issue.
>> I had to replace field one of a file with a new set of incrementing
>> numbers starting after 1 million.awk'sub($1,1000000++C)' > file.new
>>
>> Great stuff thanks!- Hide quoted text -
>>
>> - Show quoted text -
>
>
> Crud, I spoke to soon. It replaces the numbers in field 1, and works
> fine for up to 9 digits, but that is all.
> It seems to leave the original digits alone and when it reached 10 is
> adds another digit to the end of the number.
> Say the file has 2000 entries.
> We can start it at 100 (for example) and it is fine up to 109, but
> then instead of 110 I get 1010. Then it is fine up to 1099 but after
> that I get 10100.
> Does anyone have a way for me to get around this littel snafu?
> I need the file to just count up from 100 to 2,100 (two thousand lines
> in the example file).
> field is is already strictly numeric, and already in order, but it
> needs to be changed from one range to another.
> Originally from 800,000-802,000 to 1,000,000-1,002,000.
> Thanks for any suggestions!
>
> My profile is old, I need to update my email address for my new work
> email, so currently I can only see responses through this posting.

awk 'BEGIN{c=1000000}{sub($1,++c)}'

Re: Search & replace, but increment with every replace

On Nov 27, 5:35 pm, Bill Marcum wrote:
> On 2007-11-27, Tonagon wrote:

> awk 'BEGIN{c=1000000}{sub($1,++c)}'- Hide quoted text -
>
> - Show quoted text -

Thanks Bill, you rock!
I gave it a whirl though and have not had any luck yet.
I simplified my script to contain just this line now:

awk 'BEGIN{c=1000000}{sub($1,++c)}' $1 > $1.new

So it is run with an arument (the file name to be parsed by awk) and
then redirects the output to the same filename with a .new extension.
Sadly it just creates a zero byte file.
However, something is missing, as just running the awk command against
a file
awk 'BEGIN{c=1000000}{sub($1,++c)}' master.out
Gives me no results.
I tried throwing a print in there, but no luck.
I am sure this is something simple that I am missing, but after five
years I have gotten extremely rusty with these things!
What is missing?

Thanks again,
Tonagon

Re: Search & replace, but increment with every replace

On 11/28/2007 10:47 AM, Tonagon wrote:
> On Nov 27, 5:35 pm, Bill Marcum wrote:
>
>>On 2007-11-27, Tonagon wrote:
>
>
>>awk 'BEGIN{c=1000000}{sub($1,++c)}'- Hide quoted text -
>>
>>- Show quoted text -
>
>
> Thanks Bill, you rock!
> I gave it a whirl though and have not had any luck yet.
> I simplified my script to contain just this line now:
>
> awk 'BEGIN{c=1000000}{sub($1,++c)}' $1 > $1.new
>
> So it is run with an arument (the file name to be parsed by awk) and
> then redirects the output to the same filename with a .new extension.
> Sadly it just creates a zero byte file.

Right, you didn't tell it to output anything.

> However, something is missing, as just running the awk command against
> a file
> awk 'BEGIN{c=1000000}{sub($1,++c)}' master.out
> Gives me no results.
> I tried throwing a print in there, but no luck.

Really? That's surprising.

> I am sure this is something simple that I am missing, but after five
> years I have gotten extremely rusty with these things!
> What is missing?

Try this:

awk 'BEGIN{c=1000000}{sub($1,++c); print}' "$1" > "$1".new

Regards,

Ed.

Re: Search & replace, but increment with every replace

On 11/28/2007 11:39 AM, Ed Morton wrote:
>
> On 11/28/2007 10:47 AM, Tonagon wrote:
>
>>On Nov 27, 5:35 pm, Bill Marcum wrote:
>>
>>
>>>On 2007-11-27, Tonagon wrote:
>>
>>
>>>awk 'BEGIN{c=1000000}{sub($1,++c)}'- Hide quoted text -
>>>
>>>- Show quoted text -
>>
>>
>>Thanks Bill, you rock!
>>I gave it a whirl though and have not had any luck yet.
>>I simplified my script to contain just this line now:
>>
>>awk 'BEGIN{c=1000000}{sub($1,++c)}' $1 > $1.new
>>
>>So it is run with an arument (the file name to be parsed by awk) and
>>then redirects the output to the same filename with a .new extension.
>>Sadly it just creates a zero byte file.
>
>
> Right, you didn't tell it to output anything.
>
>
>>However, something is missing, as just running the awk command against
>>a file
>>awk 'BEGIN{c=1000000}{sub($1,++c)}' master.out
>>Gives me no results.
>>I tried throwing a print in there, but no luck.
>
>
> Really? That's surprising.
>
>
>>I am sure this is something simple that I am missing, but after five
>>years I have gotten extremely rusty with these things!
>>What is missing?
>
>
> Try this:
>
> awk 'BEGIN{c=1000000}{sub($1,++c); print}' "$1" > "$1".new
>

Note, however, that that's dangerous if $1 contains an RE wildcard, e.g.:

$ echo "a b c" | awk 'sub($1,9)'
9 b c

$ echo ".* b c" | awk 'sub($1,9)'
9

If you don't care about preserving the spacing between fields you could do this
instead:

awk 'BEGIN{c=1000000}{$1=++c; print}' "$1" > "$1".new

If you do care about the field separators and an RE wildcard in $1 is a
possibility, you need a more complex solution...

Ed.

Re: Search & replace, but increment with every replace

Ed Morton wrote:
>
> On 11/28/2007 11:39 AM, Ed Morton wrote:
>> On 11/28/2007 10:47 AM, Tonagon wrote:
>>
>>> On Nov 27, 5:35 pm, Bill Marcum wrote:
>>>
>>>
>>>> On 2007-11-27, Tonagon wrote:
>>>
>>>> awk 'BEGIN{c=1000000}{sub($1,++c)}'- Hide quoted text -
>>>>
>>>> - Show quoted text -
>>>
>>> Thanks Bill, you rock!
>>> I gave it a whirl though and have not had any luck yet.
>>> I simplified my script to contain just this line now:
>>>
>>> awk 'BEGIN{c=1000000}{sub($1,++c)}' $1 > $1.new
>>>
>>> So it is run with an arument (the file name to be parsed by awk) and
>>> then redirects the output to the same filename with a .new extension.
>>> Sadly it just creates a zero byte file.
>>
>> Right, you didn't tell it to output anything.
>>
>>
>>> However, something is missing, as just running the awk command against
>>> a file
>>> awk 'BEGIN{c=1000000}{sub($1,++c)}' master.out
>>> Gives me no results.
>>> I tried throwing a print in there, but no luck.
>>
>> Really? That's surprising.
>>
>>
>>> I am sure this is something simple that I am missing, but after five
>>> years I have gotten extremely rusty with these things!
>>> What is missing?
>>
>> Try this:
>>
>> awk 'BEGIN{c=1000000}{sub($1,++c); print}' "$1" > "$1".new
>>
>
> Note, however, that that's dangerous if $1 contains an RE wildcard, e.g.:
>
> $ echo "a b c" | awk 'sub($1,9)'
> 9 b c
>
> $ echo ".* b c" | awk 'sub($1,9)'
> 9
>
> If you don't care about preserving the spacing between fields you could do this
> instead:
>
> awk 'BEGIN{c=1000000}{$1=++c; print}' "$1" > "$1".new
>
> If you do care about the field separators and an RE wildcard in $1 is a
> possibility, you need a more complex solution...
>
> Ed.
>

"complex" like this:

awk 'BEGIN{c=1000000} $1~/^[0-9]*$/ {sub($1, ++c); print}'

Or use a sub-string expression; this works on old awk's, too:

awk 'BEGIN{c=1000000} {print ++c substr($0, length($1)+1)}'

--
Michael Tosch @ hp : com

Re: Search & replace, but increment with every replace

On 11/28/2007 1:13 PM, Michael Tosch wrote:
> Ed Morton wrote:
>
>>On 11/28/2007 11:39 AM, Ed Morton wrote:
>>
>>>On 11/28/2007 10:47 AM, Tonagon wrote:
>>>
>>>
>>>>On Nov 27, 5:35 pm, Bill Marcum wrote:
>>>>
>>>>
>>>>
>>>>>On 2007-11-27, Tonagon wrote:
>>>>
>>>>>awk 'BEGIN{c=1000000}{sub($1,++c)}'- Hide quoted text -
>>>>>
>>>>>- Show quoted text -
>>>>
>>>>Thanks Bill, you rock!
>>>>I gave it a whirl though and have not had any luck yet.
>>>>I simplified my script to contain just this line now:
>>>>
>>>>awk 'BEGIN{c=1000000}{sub($1,++c)}' $1 > $1.new
>>>>
>>>>So it is run with an arument (the file name to be parsed by awk) and
>>>>then redirects the output to the same filename with a .new extension.
>>>>Sadly it just creates a zero byte file.
>>>
>>>Right, you didn't tell it to output anything.
>>>
>>>
>>>
>>>>However, something is missing, as just running the awk command against
>>>>a file
>>>>awk 'BEGIN{c=1000000}{sub($1,++c)}' master.out
>>>>Gives me no results.
>>>>I tried throwing a print in there, but no luck.
>>>
>>>Really? That's surprising.
>>>
>>>
>>>
>>>>I am sure this is something simple that I am missing, but after five
>>>>years I have gotten extremely rusty with these things!
>>>>What is missing?
>>>
>>>Try this:
>>>
>>>awk 'BEGIN{c=1000000}{sub($1,++c); print}' "$1" > "$1".new
>>>
>>
>>Note, however, that that's dangerous if $1 contains an RE wildcard, e.g.:
>>
>>$ echo "a b c" | awk 'sub($1,9)'
>>9 b c
>>
>>$ echo ".* b c" | awk 'sub($1,9)'
>>9
>>
>>If you don't care about preserving the spacing between fields you could do this
>>instead:
>>
>>awk 'BEGIN{c=1000000}{$1=++c; print}' "$1" > "$1".new
>>
>>If you do care about the field separators and an RE wildcard in $1 is a
>>possibility, you need a more complex solution...
>>
>> Ed.
>>
>
>
> "complex" like this:
>
> awk 'BEGIN{c=1000000} $1~/^[0-9]*$/ {sub($1, ++c); print}'

So there can be non-numeric $1s but we're only supposed to convert $1 if it's an
integer? That requirement, if it exists, has been snipped somewhere along the
line and I don't see it in the few earlier posts in this thread that I looked
at, but then I don't see how we got to just operating on $1 either, so I'm
probably just not looking hard enough (and not inclined to look any harder!).
I'll just leave it to the OP to put us back on track if we've wandered off track...

Ed.

> Or use a sub-string expression; this works on old awk's, too:
>
> awk 'BEGIN{c=1000000} {print ++c substr($0, length($1)+1)}'
>

Re: Search & replace, but increment with every replace

On Nov 29, 8:17 am, Ed Morton wrote:
> On 11/28/2007 1:13 PM, Michael Tosch wrote:
>
>
>
>
>
> > Ed Morton wrote:
>
> >>On 11/28/2007 11:39 AM, Ed Morton wrote:
>
> >>>On 11/28/2007 10:47 AM, Tonagon wrote:
>
> >>>>On Nov 27, 5:35 pm, Bill Marcum wrote:
>
> >>>>>On 2007-11-27, Tonagon wrote:
>
> >>>>>awk 'BEGIN{c=1000000}{sub($1,++c)}'- Hide quoted text -
>
> >>>>>- Show quoted text -
>
> >>>>Thanks Bill, you rock!
> >>>>I gave it a whirl though and have not had any luck yet.
> >>>>I simplified my script to contain just this line now:
>
> >>>>awk 'BEGIN{c=1000000}{sub($1,++c)}' $1 > $1.new
>
> >>>>So it is run with an arument (the file name to be parsed by awk) and
> >>>>then redirects the output to the same filename with a .new extension.
> >>>>Sadly it just creates a zero byte file.
>
> >>>Right, you didn't tell it to output anything.
>
> >>>>However, something is missing, as just running the awk command against
> >>>>a file
> >>>>awk 'BEGIN{c=1000000}{sub($1,++c)}' master.out
> >>>>Gives me no results.
> >>>>I tried throwing a print in there, but no luck.
>
> >>>Really? That's surprising.
>
> >>>>I am sure this is something simple that I am missing, but after five
> >>>>years I have gotten extremely rusty with these things!
> >>>>What is missing?
>
> >>>Try this:
>
> >>>awk 'BEGIN{c=1000000}{sub($1,++c); print}' "$1" > "$1".new
>
> >>Note, however, that that's dangerous if $1 contains an RE wildcard, e.g.:
>
> >>$ echo "a b c" | awk 'sub($1,9)'
> >>9 b c
>
> >>$ echo ".* b c" | awk 'sub($1,9)'
> >>9
>
> >>If you don't care about preserving the spacing between fields you could do this
> >>instead:
>
> >>awk 'BEGIN{c=1000000}{$1=++c; print}' "$1" > "$1".new
>
> >>If you do care about the field separators and an RE wildcard in $1 is a
> >>possibility, you need a more complex solution...
>
> >> Ed.
>
> > "complex" like this:
>
> > awk 'BEGIN{c=1000000} $1~/^[0-9]*$/ {sub($1, ++c); print}'
>
> So there can be non-numeric $1s but we're only supposed to convert $1 if it's an
> integer? That requirement, if it exists, has been snipped somewhere along the
> line and I don't see it in the few earlier posts in this thread that I looked
> at, but then I don't see how we got to just operating on $1 either, so I'm
> probably just not looking hard enough (and not inclined to look any harder!).
> I'll just leave it to the OP to put us back on track if we've wandered off track...
>
> Ed.
>
>
>
> > Or use a sub-string expression; this works on old awk's, too:
>
> > awk 'BEGIN{c=1000000} {print ++c substr($0, length($1)+1)}'- Hide quoted text -
>
> - Show quoted text -- Hide quoted text -
>
> - Show quoted text -

Thanks Ed and Michael.
It turns out that when I tried to add the print I did not put in a
semicolon, so it did not work.
Either you know the syntax or not, and obviously I do not.

It turns out that my file is comma delimited, which I think is a
problem for the standard awk statements I have used in the past.
Field one of the file is strictly integers however, and they are
already in numeric order - with some number having been skipped.
The file starts with record 4000 and ends at 8500, but there are only
60 lines in the actual file.
I can pull it out again using tabs instead of commas and that may
help.
I will then play with your suggestions again and see how it goes.

Re: Search & replace, but increment with every replace

On Nov 29, 2:51 pm, Tonagon wrote:
> On Nov 29, 8:17 am, Ed Morton wrote:
>
>
>
>
>
> > On 11/28/2007 1:13 PM, Michael Tosch wrote:
>
> > > Ed Morton wrote:
>
> > >>On 11/28/2007 11:39 AM, Ed Morton wrote:
>
> > >>>On 11/28/2007 10:47 AM, Tonagon wrote:
>
> > >>>>On Nov 27, 5:35 pm, Bill Marcum wrote:
>
> > >>>>>On 2007-11-27, Tonagon wrote:
>
> > >>>>>awk 'BEGIN{c=1000000}{sub($1,++c)}'- Hide quoted text -
>
> > >>>>>- Show quoted text -
>
> > >>>>Thanks Bill, you rock!
> > >>>>I gave it a whirl though and have not had any luck yet.
> > >>>>I simplified my script to contain just this line now:
>
> > >>>>awk 'BEGIN{c=1000000}{sub($1,++c)}' $1 > $1.new
>
> > >>>>So it is run with an arument (the file name to be parsed by awk) and
> > >>>>then redirects the output to the same filename with a .new extension.
> > >>>>Sadly it just creates a zero byte file.
>
> > >>>Right, you didn't tell it to output anything.
>
> > >>>>However, something is missing, as just running the awk command against
> > >>>>a file
> > >>>>awk 'BEGIN{c=1000000}{sub($1,++c)}' master.out
> > >>>>Gives me no results.
> > >>>>I tried throwing a print in there, but no luck.
>
> > >>>Really? That's surprising.
>
> > >>>>I am sure this is something simple that I am missing, but after five
> > >>>>years I have gotten extremely rusty with these things!
> > >>>>What is missing?
>
> > >>>Try this:
>
> > >>>awk 'BEGIN{c=1000000}{sub($1,++c); print}' "$1" > "$1".new
>
> > >>Note, however, that that's dangerous if $1 contains an RE wildcard, e.g.:
>
> > >>$ echo "a b c" | awk 'sub($1,9)'
> > >>9 b c
>
> > >>$ echo ".* b c" | awk 'sub($1,9)'
> > >>9
>
> > >>If you don't care about preserving the spacing between fields you could do this
> > >>instead:
>
> > >>awk 'BEGIN{c=1000000}{$1=++c; print}' "$1" > "$1".new
>
> > >>If you do care about the field separators and an RE wildcard in $1 is a
> > >>possibility, you need a more complex solution...
>
> > >> Ed.
>
> > > "complex" like this:
>
> > > awk 'BEGIN{c=1000000} $1~/^[0-9]*$/ {sub($1, ++c); print}'
>
> > So there can be non-numeric $1s but we're only supposed to convert $1 if it's an
> > integer? That requirement, if it exists, has been snipped somewhere along the
> > line and I don't see it in the few earlier posts in this thread that I looked
> > at, but then I don't see how we got to just operating on $1 either, so I'm
> > probably just not looking hard enough (and not inclined to look any harder!).
> > I'll just leave it to the OP to put us back on track if we've wandered off track...
>
> > Ed.
>
> > > Or use a sub-string expression; this works on old awk's, too:
>
> > > awk 'BEGIN{c=1000000} {print ++c substr($0, length($1)+1)}'- Hide quoted text -
>
> > - Show quoted text -- Hide quoted text -
>
> > - Show quoted text -
>
> Thanks Ed and Michael.
> It turns out that when I tried to add the print I did not put in a
> semicolon, so it did not work.
> Either you know the syntax or not, and obviously I do not.
>
> It turns out that my file is comma delimited, which I think is a
> problem for the standard awk statements I have used in the past.
> Field one of the file is strictly integers however, and they are
> already in numeric order - with some number having been skipped.
> The file starts with record 4000 and ends at 8500, but there are only
> 60 lines in the actual file.
> I can pull it out again using tabs instead of commas and that may
> help.
> I will then play with your suggestions again and see how it goes.- Hide quoted text -
>
> - Show quoted text -

Got it!
This works:
awk 'BEGIN{c=8001}{sub($1,++c);print}' "$1" > "$1".new

I just had to get the commas out of there. I did and now it looks
like we are golden. I will have to import the file back into the
database to be sure, but it looks good to me.
Now it is just a quick change to my output scripts so they stop
getting the data out in a comma delimited format, then maybe let it
take an argument for the beginning number range, and I can write up
instructions and distribute it to the folks who need it.
This will save some folks many hours of work and they will be
thrilled. Previously they have been manually retyping the numbers to
be the new ranges that they want.

Ed Morton is a cool arse dude. I don't care what his ex says about
him.

Re: Search & replace, but increment with every replace

On 11/29/2007 1:51 PM, Tonagon wrote:
> On Nov 29, 8:17 am, Ed Morton wrote:

>>>>>awk 'BEGIN{c=1000000}{sub($1,++c); print}' "$1" > "$1".new

> It turns out that my file is comma delimited, which I think is a
> problem for the standard awk statements I have used in the past.

Just tell awk that's what you're using as the field separator:

awk -F, 'BEGIN{c=1000000}{sub($1,++c); print}' "$1" > "$1".new

Regards,

Ed.