Use the following syntax:
sort {file-name} | uniq -u
sort file.log | uniq -u
Here is a sample test file called garbage.txt:
this is a test
food that are killing you
wings of fire
we hope that the labor spent in creating this software
this is a test
unix ips as well as enjoy our blog
Type the following command to get rid of all duplicate lines:
$ sort garbage.txt | uniq -u
Sample output:
food that are killing you
unix ips as well as enjoy our blog
we hope that the labor spent in creating this software
wings of fire
Where,
- -u : check for strict ordering, remove all duplicate lines.
