awk: Fields, Patterns, and Aggregation

Sun Aug 30 2026

Per GNU Awk’s gawk(1) man page, awk is a “pattern scanning and processing language” — it reads input a record (line) at a time, automatically splits each record into fields, and runs pattern/action pairs (pattern { action }) against it: any record matching pattern triggers action. This post runs five awk snippets against a synthetic access log.

The data

All five awk snippets below run against the same access.log, 30 lines, one simulated HTTP request per line in a Combined-Log-Format-style layout: client IP, two placeholder dashes, [timestamp], "METHOD PATH HTTP/1.1", status code, response size in bytes — fields separated by single spaces (the quoted request itself contains spaces too, so awk’s default whitespace splitting still breaks it into three of the ten total fields).

203.0.113.5 - - [24/Aug/2026:10:00:01 +0000] "GET /index.html HTTP/1.1" 200 532
198.51.100.23 - - [24/Aug/2026:10:00:02 +0000] "GET /index.html HTTP/1.1" 200 532
203.0.113.5 - - [24/Aug/2026:10:00:04 +0000] "POST /login HTTP/1.1" 401 128
198.51.100.23 - - [24/Aug/2026:10:00:05 +0000] "GET /static/app.js HTTP/1.1" 200 8213
192.0.2.77 - - [24/Aug/2026:10:00:07 +0000] "GET /api/users HTTP/1.1" 200 964
203.0.113.5 - - [24/Aug/2026:10:00:09 +0000] "POST /login HTTP/1.1" 200 356
192.0.2.77 - - [24/Aug/2026:10:00:11 +0000] "GET /api/orders HTTP/1.1" 500 89
198.51.100.23 - - [24/Aug/2026:10:00:12 +0000] "GET /favicon.ico HTTP/1.1" 404 0
203.0.113.5 - - [24/Aug/2026:10:00:14 +0000] "GET /api/users HTTP/1.1" 200 964
198.51.100.9 - - [24/Aug/2026:10:00:16 +0000] "GET /index.html HTTP/1.1" 200 532
192.0.2.77 - - [24/Aug/2026:10:00:18 +0000] "GET /api/orders HTTP/1.1" 500 89
203.0.113.5 - - [24/Aug/2026:10:00:20 +0000] "DELETE /api/orders/44 HTTP/1.1" 204 0
198.51.100.23 - - [24/Aug/2026:10:00:22 +0000] "GET /old-page HTTP/1.1" 301 178
192.0.2.77 - - [24/Aug/2026:10:00:24 +0000] "GET /api/orders HTTP/1.1" 200 1180
203.0.113.5 - - [24/Aug/2026:10:00:26 +0000] "GET /static/app.js HTTP/1.1" 200 8213
198.51.100.9 - - [24/Aug/2026:10:00:28 +0000] "POST /login HTTP/1.1" 401 128
192.0.2.77 - - [24/Aug/2026:10:00:30 +0000] "GET /api/users HTTP/1.1" 200 964
203.0.113.5 - - [24/Aug/2026:10:00:32 +0000] "GET /admin HTTP/1.1" 403 96
198.51.100.23 - - [24/Aug/2026:10:00:34 +0000] "GET /index.html HTTP/1.1" 200 532
192.0.2.77 - - [24/Aug/2026:10:00:36 +0000] "PUT /api/users/9 HTTP/1.1" 200 210
203.0.113.5 - - [24/Aug/2026:10:00:38 +0000] "GET /api/orders HTTP/1.1" 500 89
198.51.100.9 - - [24/Aug/2026:10:00:40 +0000] "GET /favicon.ico HTTP/1.1" 404 0
192.0.2.77 - - [24/Aug/2026:10:00:42 +0000] "GET /static/app.js HTTP/1.1" 200 8213
203.0.113.5 - - [24/Aug/2026:10:00:44 +0000] "GET /index.html HTTP/1.1" 200 532
198.51.100.23 - - [24/Aug/2026:10:00:46 +0000] "POST /login HTTP/1.1" 200 356
192.0.2.77 - - [24/Aug/2026:10:00:48 +0000] "GET /api/users HTTP/1.1" 200 964
203.0.113.5 - - [24/Aug/2026:10:00:50 +0000] "GET /old-page HTTP/1.1" 301 178
198.51.100.9 - - [24/Aug/2026:10:00:52 +0000] "GET /api/orders HTTP/1.1" 500 89
192.0.2.77 - - [24/Aug/2026:10:00:54 +0000] "GET /admin HTTP/1.1" 403 96
203.0.113.5 - - [24/Aug/2026:10:00:56 +0000] "GET /index.html HTTP/1.1" 200 532

The code

#!/usr/bin/env bash
set -e

echo "== print IP and status: \$1 and \$9 =="
awk '{print $1, $9}' access.log | head -5

echo
echo "== pattern-action: only 500-status lines, print IP and path =="
awk '$9 == 500 {print $1, $7}' access.log

echo
echo "== count requests per status code =="
awk '{count[$9]++} END {for (s in count) print s, count[s]}' access.log

echo
echo "== NR and NF: line number and field count for first 3 lines =="
awk 'NR <= 3 {print NR, NF}' access.log

echo
echo "== -F to split on the literal brackets, extract timestamp =="
awk -F'[][]' '{print $2}' access.log | head -3

Per the man page:

Field variables — each record is split on whitespace by default; $1 is the first field, $9 the ninth, and so on.
NF — “The number of fields in the current input record.”
NR — “The total number of input records seen so far.”
Pattern/action — $9 == 500 {print $1, $7} is a pattern ($9 == 500) paired with an action; the action only runs on records where the pattern is true, so it acts as a filter before it acts as a printer.
Associative arrays — count[$9]++ uses the status code as a string key into an array, incrementing a per-key counter as awk reads each record; for (s in count) in the END block (after all input is read) then iterates every key seen.
-F fs, --field-separator fs — “Use fs for the input field separator.” -F'[][]' sets the separator to the character class [ or ], so the bracketed timestamp becomes its own field.

Running it

Real output:

== print IP and status: $1 and $9 ==
203.0.113.5 200
198.51.100.23 200
203.0.113.5 401
198.51.100.23 200
192.0.2.77 200

== pattern-action: only 500-status lines, print IP and path ==
192.0.2.77 /api/orders
192.0.2.77 /api/orders
203.0.113.5 /api/orders
198.51.100.9 /api/orders

== count requests per status code ==
200 17
204 1
301 2
401 2
403 2
404 2
500 4

== NR and NF: line number and field count for first 3 lines ==
1 10
2 10
3 10

== -F to split on the literal brackets, extract timestamp ==
24/Aug/2026:10:00:01 +0000
24/Aug/2026:10:00:02 +0000
24/Aug/2026:10:00:04 +0000

{print $1, $9} pulled the IP and status code out of every line by field position. The pattern $9 == 500 acted as a filter — only the four lines with status 500 matched, and their action printed IP and path ($7). The associative-array tally counted every status code in the file at once: 17 200s, 4 500s, 2 each of 301, 401, 403, 404, and 1 204 — that’s 30 lines total, matching the log’s line count. NR/NF confirmed every line has 10 space-separated fields. -F'[][]' treated [ and ] as field separators, so $2 — the text between the brackets — was the timestamp on its own.

Takeaway

The real run showed awk’s core model in action: fields addressed by position, a pattern acting as a filter before its action runs, and an associative array turning a full pass over the file into a single-command aggregation — all without a separate sort/uniq step.