awk Command Examples

Print Columns

Fields split on whitespace by default. $1 is the first field, $NF the last, $0 the whole line.

Column Ranges

Printing everything except one column is the awkward case, because awk has no slice syntax.

Assigning to a field rebuilds $0 with OFS, so $1="" flattens any column padding the file had. The substr/index form avoids that, at the cost of matching the wrong spot if that text also appears earlier in the line. For an arbitrary start column, loop:

awk '{for (i = 3; i <= NF; i++) printf "%s%s", $i, (i < NF ? OFS : ORS)}' file

Field Separators

-F sets the input separator; OFS is what goes between fields on output.

NR and NF

NR is the current line number, NF the number of fields on the line. Both double as filters.

Patterns and Conditions

The shape is pattern {action}: the action runs on lines where the pattern holds. Either half can be omitted.

if/else lives inside the action:

awk '{if ($3 > 100) print $1 " high"; else print $1 " ok"}' data.txt
awk '{print $1, ($3 > 100 ? "high" : "ok")}' data.txt

Loops and Control Flow

next skips the remaining rules for that line; exit jumps straight to END, so an END block still runs after it.

BEGIN and END

BEGIN runs before the first line, END after the last: headers, totals, averages.

printf

printf formats output with C-style specifiers and adds no newline unless you write \n.

print on a number over about 16 digits falls back to OFMT and prints 1e+18; %.0f or %d keeps the digits.

Quoting and Output

> inside the program is awk's redirect, not the shell's, so it needs no quoting beyond the single quotes already around the program. awk keeps every file it writes open, so splitting into thousands of files needs close($1".txt").

Arrays and Counting

awk arrays are string-keyed maps, which makes counting and grouping one-liners.

The count-and-sort pipeline for "top talkers" in a log (see the grep cheat sheet for the pure-grep half):

awk '{print $1}' access.log | sort | uniq -c | sort -rn | head

String Functions

sub and gsub modify in place and return the replacement count. For pure stream find-and-replace, sed is usually the shorter tool.

Variables

Shell Commands from awk

awk can shell out mid-program, which covers the jobs a pipeline alone cannot.

A command stays open after the first read, so running it again in the same program returns nothing until you close("date +%F").

Scripts and Recipes

Past one line, put the program in a file and run it with -f, or give it a shebang.

awk -f report.awk access.log

#!/usr/bin/awk -f
BEGIN { FS = ","; print "user,total" }
NR > 1 { total[$1] += $3 }
END { for (u in total) print u "," total[u] }

Everyday pipelines:

Gotchas

• Single-quote the program. In double quotes the shell expands $1 before awk runs, which silently breaks the script.
• awk: command not found means no awk is installed, which happens on slim containers: apt install gawk or apk add gawk. A syntax error at source line 1 almost always means the shell ate part of the program, so check the quoting before the logic.
• Not all awks are equal: Linux usually has gawk or mawk, macOS ships an older BSD awk. gensub(), length(array), and \d-style classes are gawk extras; everything on this sheet sticks to the portable core.
• Comparisons are type-sensitive: $1 > 5 compares numbers, but if the field has stray characters awk may compare strings. Force a number with $1+0.
• Assigning to any field (even $1=$1) rebuilds $0 with OFS, which rewrites the separators on output. Sometimes that is exactly what you want; sometimes it is the bug.
• Uninitialized variables are "" and 0 at the same time, so sum += $1 needs no setup, and a missing field prints as empty rather than erroring.
• grep selects lines and sed rewrites them; see the grep cheat sheet and sed cheat sheet for the other two thirds of the toolkit.