datmt.com

Chaining cut, grep, sed, and awk Into One Pipeline

Sun Aug 30 2026

cut, grep, sed, and awk are each built to do one narrow thing to a stream of text — search it, reshape it, edit it, or aggregate it. The shell’s | pipe connects one command’s stdout to the next command’s stdin, so instead of one tool doing everything, a chain of small commands passes the output forward, each one narrowing or reshaping it a bit more. This post builds one pipeline up in stages against a synthetic access log, then runs a second pipeline that uses all four tools together.

The data

Both pipelines below run against the same access.log, 30 lines, one simulated HTTP request per line in a Combined-Log-Format-style layout: client IP, two placeholder dashes, [timestamp], "METHOD PATH HTTP/1.1", status code, response size in bytes — fields separated by single spaces.

203.0.113.5 - - [24/Aug/2026:10:00:01 +0000] "GET /index.html HTTP/1.1" 200 532
198.51.100.23 - - [24/Aug/2026:10:00:02 +0000] "GET /index.html HTTP/1.1" 200 532
203.0.113.5 - - [24/Aug/2026:10:00:04 +0000] "POST /login HTTP/1.1" 401 128
198.51.100.23 - - [24/Aug/2026:10:00:05 +0000] "GET /static/app.js HTTP/1.1" 200 8213
192.0.2.77 - - [24/Aug/2026:10:00:07 +0000] "GET /api/users HTTP/1.1" 200 964
203.0.113.5 - - [24/Aug/2026:10:00:09 +0000] "POST /login HTTP/1.1" 200 356
192.0.2.77 - - [24/Aug/2026:10:00:11 +0000] "GET /api/orders HTTP/1.1" 500 89
198.51.100.23 - - [24/Aug/2026:10:00:12 +0000] "GET /favicon.ico HTTP/1.1" 404 0
203.0.113.5 - - [24/Aug/2026:10:00:14 +0000] "GET /api/users HTTP/1.1" 200 964
198.51.100.9 - - [24/Aug/2026:10:00:16 +0000] "GET /index.html HTTP/1.1" 200 532
192.0.2.77 - - [24/Aug/2026:10:00:18 +0000] "GET /api/orders HTTP/1.1" 500 89
203.0.113.5 - - [24/Aug/2026:10:00:20 +0000] "DELETE /api/orders/44 HTTP/1.1" 204 0
198.51.100.23 - - [24/Aug/2026:10:00:22 +0000] "GET /old-page HTTP/1.1" 301 178
192.0.2.77 - - [24/Aug/2026:10:00:24 +0000] "GET /api/orders HTTP/1.1" 200 1180
203.0.113.5 - - [24/Aug/2026:10:00:26 +0000] "GET /static/app.js HTTP/1.1" 200 8213
198.51.100.9 - - [24/Aug/2026:10:00:28 +0000] "POST /login HTTP/1.1" 401 128
192.0.2.77 - - [24/Aug/2026:10:00:30 +0000] "GET /api/users HTTP/1.1" 200 964
203.0.113.5 - - [24/Aug/2026:10:00:32 +0000] "GET /admin HTTP/1.1" 403 96
198.51.100.23 - - [24/Aug/2026:10:00:34 +0000] "GET /index.html HTTP/1.1" 200 532
192.0.2.77 - - [24/Aug/2026:10:00:36 +0000] "PUT /api/users/9 HTTP/1.1" 200 210
203.0.113.5 - - [24/Aug/2026:10:00:38 +0000] "GET /api/orders HTTP/1.1" 500 89
198.51.100.9 - - [24/Aug/2026:10:00:40 +0000] "GET /favicon.ico HTTP/1.1" 404 0
192.0.2.77 - - [24/Aug/2026:10:00:42 +0000] "GET /static/app.js HTTP/1.1" 200 8213
203.0.113.5 - - [24/Aug/2026:10:00:44 +0000] "GET /index.html HTTP/1.1" 200 532
198.51.100.23 - - [24/Aug/2026:10:00:46 +0000] "POST /login HTTP/1.1" 200 356
192.0.2.77 - - [24/Aug/2026:10:00:48 +0000] "GET /api/users HTTP/1.1" 200 964
203.0.113.5 - - [24/Aug/2026:10:00:50 +0000] "GET /old-page HTTP/1.1" 301 178
198.51.100.9 - - [24/Aug/2026:10:00:52 +0000] "GET /api/orders HTTP/1.1" 500 89
192.0.2.77 - - [24/Aug/2026:10:00:54 +0000] "GET /admin HTTP/1.1" 403 96
203.0.113.5 - - [24/Aug/2026:10:00:56 +0000] "GET /index.html HTTP/1.1" 200 532

The code

#!/usr/bin/env bash
set -e

echo "== step 1: grep to isolate API errors (5xx) =="
grep -E ' 50[0-9] ' access.log

echo
echo "== step 2: pipe into cut for just the path =="
grep -E ' 50[0-9] ' access.log | cut -d' ' -f7

echo
echo "== step 3: pipe into sort | uniq -c to rank failing endpoints =="
grep -E ' 50[0-9] ' access.log | cut -d' ' -f7 | sort | uniq -c | sort -rn

echo
echo "== full pipeline: grep isolates /login rows, sed labels each as OK/FAIL, =="
echo "== awk tallies OK vs FAIL per IP =="
grep '/login' access.log \
  | sed -E 's/" 200 [0-9]+$/" OK/; s/" 401 [0-9]+$/" FAIL/' \
  | awk '{result=$NF; ip=$1; count[ip" "result]++} END {for (k in count) print k, count[k]}' \
  | sort

The first three steps build up incrementally: grep -E ' 50[0-9] ' isolates 5xx lines, piping that into cut -d' ' -f7 keeps only the request path, and adding sort | uniq -c | sort -rn turns those paths into a ranked count (sort groups identical lines together so uniq -c can count them, then the second sort -rn orders by that count, numerically, descending). The final pipeline is a different task: grep '/login' isolates login attempts, sed -E rewrites each " 200 <size>"/" 401 <size>" tail into a plain OK/FAIL label, and awk tallies OK vs. FAIL counts per IP using a composite array key (count[ip" "result]++).

Running it

Real output:

== step 1: grep to isolate API errors (5xx) ==
192.0.2.77 - - [24/Aug/2026:10:00:11 +0000] "GET /api/orders HTTP/1.1" 500 89
192.0.2.77 - - [24/Aug/2026:10:00:18 +0000] "GET /api/orders HTTP/1.1" 500 89
203.0.113.5 - - [24/Aug/2026:10:00:38 +0000] "GET /api/orders HTTP/1.1" 500 89
198.51.100.9 - - [24/Aug/2026:10:00:52 +0000] "GET /api/orders HTTP/1.1" 500 89

== step 2: pipe into cut for just the path ==
/api/orders
/api/orders
/api/orders
/api/orders

== step 3: pipe into sort | uniq -c to rank failing endpoints ==
      4 /api/orders

== full pipeline: grep isolates /login rows, sed labels each as OK/FAIL, ==
== awk tallies OK vs FAIL per IP ==
198.51.100.23 OK 1
198.51.100.9 FAIL 1
203.0.113.5 FAIL 1
203.0.113.5 OK 1

Building the pipeline step by step showed what each stage removed: grep cut 30 lines down to the 4 50x lines, cut reduced each of those to just /api/orders, and sort | uniq -c | sort -rn collapsed the four identical paths into one ranked line, 4 /api/orders. The second pipeline shows a different combination — grep picked out the 4 /login lines, sed relabeled their status codes as OK/FAIL, and awk’s associative array produced a per-IP, per-outcome count: 198.51.100.23 and 203.0.113.5 each had one OK, while 198.51.100.9 and 203.0.113.5 each had one FAIL.

Takeaway

Across both pipelines in this run, each tool did one narrow job — grep filtered rows, cut/sed reshaped what was left, and sort/uniq/awk aggregated it — and chaining them with | meant no single command had to do more than the one thing it’s built for.