Text Processing
sed applies editing commands to each line. The substitute command s/old/new/ replaces the first match per line; g replaces all matches, I ignores case. -i edits the file in place instead of printing the result.$ sed 's/old/new/g' [file] copy
$ sed -i 's/old/new/g' [file] copy
$ sed -i 's/old/new/gI' [file] copy
sd does the same job with simpler, regex-by-default syntax.$ sd "old" "new" [file] copy
Any character can delimit the s command: s|/usr/bin|/usr/local/bin| avoids escaping slashes in paths.
d deletes matching lines; -n with p prints only selected lines. Addresses can be patterns, line numbers, or ranges.$ sed '/pattern/d' [file] copy
$ sed -i '/^$/d' [file] copy
$ sed -n '5,10p' [file] copy
$ sed -n '/pattern/p' [file] copy
awk splits every line into fields: 1ドル is the first field, 0ドル the whole line, NR the line number. -F changes the field separator from whitespace to anything else.$ awk '{print 1ドル}' [file] copy
$ awk -F: '{print 1,ドル 3ドル}' /etc/passwd copy
$ awk '{print NR, 0ドル}' [file] copy
cut is the lightweight alternative for simple column extraction, by delimiter (-d , -f ) or character position (-c ).$ cut -d',' -f1,3 [file] copy
An awk program is condition { action } : lines matching the condition run the action (default: print the line). $ awk '3ドル > 100' [file] copy
$ awk '/pattern/ {print 2ドル}' [file] copy
$ awk 'NR>=5 && NR<=10' [file] copy
Aggregate across lines with variables and an END block. $ awk '{sum += 1ドル} END {print sum}' [file] copy
$ awk '{sum += 1ドル} END {print sum/NR}' [file] copy
Alphabetical by default; -n sorts numerically, -r reverses, -u drops duplicates, -t and -k sort by a specific field. -h understands human-readable sizes like 2K and 1G. $ sort -t: -k3 -n [file] copy
uniq only compares neighboring lines, so sort first. -c counts occurrences, -d shows only duplicated lines, -u only unique ones.The sort | uniq -c | sort -rn pipeline is the classic frequency counter: it ranks every distinct line by how often it occurs.
tr maps characters from one set to another, -d deletes them, -s squeezes repeats into one.$ tr 'a-z' 'A-Z' < [file] copy
$ tr -d '[:digit:]' < [file] copy
$ tr '\n' ' ' < [file] copy
diff -u is the standard patch-style comparison; -y shows files side by side; cmp compares bytes and is ideal for binary files.$ diff -u [file1] [file2] copy
$ diff -y [file1] [file2] copy
$ cmp [file1] [file2] copy
comm shows lines unique to each sorted file and lines they share, in three columns. Suppress columns by number.$ comm [file1] [file2] copy
$ comm -12 [file1] [file2] copy
Both comm and join require their input files to be sorted.
paste glues files together line by line; -s joins all lines of one file into a single line. join matches lines from two files on a common field, like a database join.$ paste [file1] [file2] copy
$ paste -d',' [file1] [file2] copy
$ join [file1] [file2] copy
$ join -t: -1 1 -2 3 [file1] [file2] copy
Align fields into a table, or wrap long lines. fmt rewraps paragraphs intelligently, fold cuts hard at the width (-s breaks at spaces). $ fold -s -w 80 [file] copy
nl numbers only non-empty lines by default; -ba numbers all lines, like cat -n .
Lines, words, and bytes; -m counts characters in multi-byte encodings.
tac prints a file last line first, rev reverses each line's characters, shuf randomizes line order.
Copied to clipboard