Delete lines containing repeated text

Question

I have a file containing two paths on each line. I want to remove the lines containing the same path twice.

I work on Linux and Solaris. I would like a one-liner in sed or awk or perl.

Example input file:

 /usr/lib/libgmp.so.3.3.3 /usr/lib/libgmp.so.3.3.3
 /usr/lib/libxslt.so.1.1.17 /usr/lib/libxslt.so.1.1.17
 /usr/lib/sse2/libgmp.so.3.3.3 /usr/lib/sse2/libgmp.so.3.3.3
 /usr/local/swp-tomcat-6.0/lib/commons-logging-1.1.1.jar /usr/local/swp-tomcat-6.0/lib/commons-logging-1.1.1.jar
 /usr/share/doc/libXrandr-1.1.1 /usr/share/doc/libXrandr-1.1.1
 /usr/share/doc/libxslt-1.1.17 /usr/share/doc/libxslt-1.1.17
 /etc/3.3.3.255 /etc/172.17.211.255
 /etc/1.1.1.255 /etc/172.17.213.255

Expected output:

 /etc/3.3.3.255 /etc/172.17.211.255
 /etc/1.1.1.255 /etc/172.17.213.255

• manatwork: grep isn't good? That is the dedicated tool to output lines from a file matching certain condition: grep -vx '\s*\(\S\+\)\s\+1円\s*' file.

Answer 1

awk '{ if (1ドル != 2ドル ) print 1ドル" "2ドル; }' file

Just replace file for the appropriate file.

Or as @manatwork mentioned in the comments and simpler

awk '1ドル!=2ドル' file

• manatwork: Why not just awk '1ドル!=2ドル' file?
• BitsOfNix: I was not aware it could be that simple. :/

Answer 2

You can express repeated text in grep's regexps (this is an extension to the mathematical notion of regular expression).

grep -v '^ *\([^ ][^ ]*\) *1円 *$'

[^ ][^ ]* matches one or more non-space character. The backslash-parentheses make this a group, and 1円 means "the same text as the first group".

• Bernhard: Except from being an alternative, can you give an argument why one should use grep over awk here?
• Gilles 'SO- stop being evil': @Bernhard Both are good methods. More people know grep basics than awk basics, but on the other hand the awk code is clearer, so I think there's no clear winner.

Answer 3

This might work for you (GNU sed):

sed -r '/(\S+)\s1円/d' file

• syntaxerror: Though untested, your (elegant!) solution should also work in non-GNU sed when written like this: sed '/\(\S\+\)\s1円/d' file. Just a little more escaping needed, and you'll be 100% compatible everywhere.