awk Cheat Sheet
awk one-liners: fields, patterns, actions, FS/OFS, and common text-processing recipes.
awk reads input records, splits them into fields, and runs actions for records whose patterns match. It is particularly effective for column-oriented text, summaries, and small transformations without writing a full script.
Program structure and fields
An awk program has the shape pattern { action }. A missing pattern matches every record; a missing action prints the current record. Fields are separated according to FS.
awk '{ print $1, $3 }' data.txt
awk 'NR > 1 { print $0 }' data.csv
| Variable | Meaning |
|---|---|
$0 | Entire current record. |
$1 … $NF | Individual fields. |
NF | Number of fields in the current record. |
NR | Current record number across all input. |
FNR | Record number within the current file. |
FS | Input field separator. |
OFS | Output field separator. |
ORS | Output record separator. |
Separators and input
Use -F for a one-off separator or set FS in a BEGIN block. The default whitespace separator collapses runs of spaces and tabs.
awk -F, '{ print $1, $2 }' users.csv
awk 'BEGIN { FS = ":"; OFS = " | " } { print $1, $3 }' /etc/passwd
awk -v FS=, -v OFS='\t' '{ print $2, $4 }' data.csv
BEGIN, END, and conditions
BEGIN runs before input, END runs after the final record, and ordinary rules run once per record. Conditions can inspect fields, record numbers, or regular expressions.
awk 'BEGIN { print "name\tcount" } /error/ { print $1, $2 } END { print "done" }' app.log
awk '$3 > 100 { print $1, $3 }' metrics.txt
awk '$1 ~ /^prod/ { print $0 }' hosts.txt
awk '$2 !~ /disabled/' accounts.txt
| Pattern | Meaning |
|---|---|
/regex/ | Record matches a regular expression. |
$3 > 100 | Numeric comparison on field 3. |
$1 == "prod" | Exact string or numeric comparison. |
NR == 1 | First input record. |
NF | Records with at least one field. |
condition && other | Both conditions must match. |
Common one-liners
These commands read from standard input or a named file. Add -v OFS=... when output needs a stable delimiter.
awk '{ print $2 }' access.log
awk 'END { print NR }' input.txt
awk '{ sum += $3 } END { print sum }' measurements.txt
awk '{ count[$1]++ } END { for (value in count) print value, count[value] }' data.txt
awk '!seen[$1]++ { print $1 }' values.txt
awk 'length($0) > 80 { print FNR ":" $0 }' source.txt
Formatting and substitution
printf gives control over alignment and numeric formatting. sub changes the first match; gsub changes every match in the selected string.
awk '{ printf "%-20s %8.2f\n", $1, $2 }' prices.txt
awk '{ gsub(/[[:space:]]+/, " "); print }' messy.txt
awk '{ sub(/^ID:/, ""); print }' identifiers.txt
awk 'BEGIN { OFS = "," } { $2 = toupper($2); print }' names.tsv
Changing a field rebuilds $0 using OFS; it does not modify the original file unless output is redirected to a replacement file.
References
- GNU awk User’s Guide
man awk