LINUX: AWK Command to Print All Lines In the File Without Duplicate entries/lines

Below AWK Command can be used to Print All Lines In the File Without Duplicate entries/lines.

awk '!A[substr($0, index($0, " "))]++' list.update

For e.g. if you have testfile with below lines, as you can see test-a and test-d lines are repeated with 2 occurrences.

test-a

test-b

test-c

test-a

test-d

test-d

Running above awk command on testfile, will print all lines avoiding printing duplicate line which is already printed. Output will be as follows

test-a

test-b

test-c

test-d