Re: uniq without sort <-------------- GURU NEEDED

Re: uniq without sort <-------------- GURU NEEDED

am 25.01.2008 08:50:01 von Thomas Troeger

> This is a tough problem, and needs a guru.

This problem is not tough.

> So it is TRIVIAL with sort.
>
> I want uniq without sorting the initial order.
>
> The algorithm is this. For every line, look above if there is another
> line like it. If so, then ignore it. If not, then output it. I am
> sure, I can spend some time to write this in C. But what is the
> solution using shell ? This way I can get an output that preserves the
> order of first occurrence. It is needed in many problems.
>
> Thanks to the star who can help
> gnuist

I'm not a star, but this will do the job:

cat somefile | awk '{ if (!h[$0]) { print $0; h[$0]=1 } }' > unique_lines