sed & awk

sed - the stream editor

sed reads text one line at a time and applies an editing command to each line as it streams past - it never loads the whole file into an interactive editor the way vim does; it's built for scripted, non-interactive, repeatable edits.

$ sed 's/old/new/g' file          # substitute all occurrences
$ sed -i 's/old/new/g' file       # edit file in place
$ sed -n '10,20p' file            # print lines 10-20
$ sed '/^#/d' file                # delete comment lines

The s/old/new/g command is sed's bread and butter: s for substitute, then old and new patterns separated by /, with a trailing g meaning "every occurrence on the line" (without g, only the first match per line gets replaced). By default sed prints its result to your screen without touching the original file - -i ("in place") is what actually overwrites the file, so it's worth testing a sed command without -i first to confirm it does what you expect.

awk - the field processor

Where sed thinks in terms of whole lines, awk thinks in terms of fields - it automatically splits each incoming line into columns ($1, $2, … $NF for the last field, where NF means "number of fields") using whitespace as the default separator, so you can operate on specific columns without manually parsing anything yourself.

$ awk '{print $1, $3}' data.txt          # columns 1 and 3
$ awk -F: '{print $1}' /etc/passwd       # : separator
$ awk '$3 > 1000 {print $1}' file        # conditional
$ df -h | awk 'NR>1 {print $5, $6}'      # skip header

The -F: flag changes the field separator from the default whitespace to a colon - necessary for /etc/passwd, which is colon-delimited (recall the field layout from the Users & Permissions module). The last example uses NR ("number of record," i.e. the current line number) to skip df -h's header line and only process actual data rows.

Tip: It's easy to reach for one tool to do a job another tool does better - keep this three-way split in mind and the trio stops feeling redundant: grep finds whole lines matching a pattern, sed transforms text (substitute, delete, extract line ranges) as it streams past, and awk extracts or computes on individual fields within structured, column-like data. A real troubleshooting one-liner often chains all three: grep to narrow down relevant lines, awk to pull out a specific column, sort | uniq -c to summarize it.