Processes, Jobs, and Signals

A process is a running program. Every process has a number (its PID), a parent, and a state. This lesson shows how to list processes, how to stop them with signals (small numbered messages the kernel delivers to a process), how to run commands in the background, and how a script limits and cleans up its own work.

Every example was run in an interactive bash inside tmux. PIDs, start times, memory sizes, and load numbers differ on every machine and every run; the text names the columns that vary. Examples that need a long-running process use sleep or a short python3 one-liner as a harmless target.

Example 1: ps aux

ps prints a snapshot of processes. ps aux shows every process from every user, with CPU and memory use. The first command starts a target in the background; the grep pattern [s]leep 300 matches the target but not the grep command itself, because the brackets make the pattern text differ from the text it matches.

sleep 300 &
ps aux | head -1
ps aux | grep '[s]leep 300'

[1] 2978
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      2978  0.0  0.0   2704  1728 pts/0    S    15:17   0:00 sleep 300

The [1] 2978 line comes from the shell (job number 1, PID 2978). The columns of ps aux:

aux is BSD-style syntax: no leading dash. ps aux works the same way on macOS. The number of lines (ps aux | wc -l) depends on the machine.

Example 2: ps -ef

ps -ef lists the same processes in the System V format. The difference that matters is the PPID column, the PID of the parent process, which aux does not show.

ps -ef | head -1
ps -ef | grep '[s]leep 300'

UID        PID  PPID  C STIME TTY          TIME CMD
root      2978  2959  0 15:17 pts/0    00:00:00 sleep 300

-e selects every process and -f selects the full format. C is recent CPU use. The PPID of the sleep is 2959, the PID of the shell that started it. A parent PID lets a reader trace which program started which. ps -ef also works on macOS.

Example 3: ps -o

ps -o chooses the columns, so the output holds exactly what a script or a reader needs. Two background Python processes hold 50 MB and 120 MB so the memory columns differ.

python3 -c 'import time; x = bytearray(50_000_000); time.sleep(300)' &
python3 -c 'import time; x = bytearray(120_000_000); time.sleep(300)' &
ps -o pid,ppid,stat,etime,comm -p $$,$!
ps -o pid=,comm= -p $!
ps -o pid,rss,%mem,comm -C python3 --sort=-rss

  PID  PPID STAT     ELAPSED COMMAND
 2959  2958 Ss         00:09 bash
 3020  2959 S          00:01 python3
 3020 python3
  PID   RSS %MEM COMMAND
 3020 124932  0.7 python3
 3016 56572  0.3 python3

The 120 MB process is first because of --sort=-rss. Its resident size is a little above 120,000,000 bytes (124,932 KiB) because the interpreter itself also uses memory. On macOS, ps -o accepts the same column names, but --sort does not exist; BSD ps uses -r (sort by CPU) and -m (sort by memory).

Example 4: pgrep

pgrep prints the PIDs of processes that match a pattern, so a script does not have to parse ps | grep.

pgrep sleep
pgrep -l python3
pgrep -a python3 | cut -c1-60
pgrep -f 'bytearray\(120'
pgrep -c python3
pgrep -x -u root sleep
pgrep -o python3; pgrep -n python3
pgrep nothing-here; echo "exit $?"

2978
3050
3016 python3
3020 python3
3016 python3 -c import time; x = bytearray(50_000_000); time
3020 python3 -c import time; x = bytearray(120_000_000); tim
3020
2
2978
3071
3016
3020
exit 1

The second PID in the pgrep sleep output (3050) belongs to a short sleep that the terminal harness ran at that moment, which shows that a plain name match catches every process with that name. pgrep exits with status 0 when something matched and 1 when nothing did, so if pgrep -x nginx > /dev/null; then ... works. On macOS, pgrep has -l and -f but no -a; use pgrep -fl.

Example 5: top

top shows a live, refreshing table of processes. Interactive use keeps the screen updating; batch mode (-b) prints plain text once, which can be saved or piped. -n 1 stops after one refresh and -p limits the list to given PIDs.

top -b -n 1 -p $!

top - 15:17:52 up 11 min,  0 user,  load average: 0.32, 0.19, 0.07
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.0 us,  2.4 sy,  0.0 ni, 97.6 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  16094.7 total,  15018.4 free,    707.3 used,    597.0 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.  15387.4 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 3020 root      20   0  131096 124932   5016 S   0.0   0.8   0:00.55 python3

The top five lines are a summary that varies on every run:

The table below the summary has one row per process. PR and NI are priority and nice value. VIRT, RES, and SHR are virtual, resident, and shared memory. S is the state letter from Example 1.

Keys in interactive top:

macOS top has no -b; the equivalent single snapshot is top -l 1, and its sort option is -o cpu or -o mem.

Example 6: htop

htop is a full-screen process viewer with colors, a CPU bar for each core, and mouse and arrow-key support. It is a separate package on most systems. -p limits the list to given PIDs; here it shows the 120 MB Python process. The screen below is a capture of the terminal with blank lines removed.

htop -p $!

    0[|||||||||||||||||||||                46.7%] Tasks: 28, 67 thr, 70 kthr; 4 running
    1[||||||||||||||||||||||               48.0%] Load average: 0.62 0.26 0.10
    2[|||||||||||||||||                    35.8%] Uptime: 00:12:05
    3[|||||||||||||||||                    38.1%]
  Mem[||||||                          763M/15.7G]
  Swp[                                     0K/0K]
  [Main] [I/O]
  PID USER       PRI  NI  VIRT   RES   SHR S  CPU%-MEM%   TIME+  Command
 7259 root        20   0  128M  122M  5012 S   0.0  0.8  0:00.06 python3 -c import time; x = bytearr
F1Help  F2Setup F3SearchF4FilterF5Tree  F6SortByF7Nice -F8Nice +F9Kill  F10Quit

The numbered bars at the top are the CPU cores, and the bars change on every run. Mem and Swp show memory and swap use. The bottom row lists the function keys:

The -p option makes htop show only the listed PIDs. In this capture the q key was sent while the filter prompt was active and opened a trace view instead of quitting, so quit with F10 when unsure. htop and its keys are the same on macOS if installed with a package manager.

Example 7: kill and signals (TERM)

kill sends a signal to a process. The default signal is TERM (terminate), which asks the process to exit and lets it clean up. Despite the name, kill sends any signal.

sleep 400 &
kill -l | head -3
kill -l 15
kill $!
jobs

[1] 10426
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL       5) SIGTRAP
 6) SIGABRT      7) SIGBUS       8) SIGFPE       9) SIGKILL     10) SIGUSR1
11) SIGSEGV     12) SIGUSR2     13) SIGPIPE     14) SIGALRM     15) SIGTERM
TERM
[1]+  Terminated              sleep 400

kill -l lists signal names with their numbers (the first three lines are shown). kill -l 15 converts the number 15 to the name TERM. kill $! sent TERM to the background sleep, and jobs then reported Terminated. The signals used most often:

"Handle" means the program can catch the signal, run its own code, and decide what to do. Sending a signal to another user's process needs sudo (for example sudo kill 1234). The container used here runs as root, so no sudo was needed; a normal user can signal only processes that user owns.

Example 8: kill -9 (KILL)

KILL cannot be caught or ignored, so it stops a process that ignores TERM. The script ignore-term.sh sets trap '' TERM (ignore the signal; see Example 20) and loops forever.

cat ignore-term.sh
./ignore-term.sh &
kill $!
ps -o pid=,stat=,comm= -p $!
kill -9 $!
jobs

#!/bin/bash
trap '' TERM
while true; do sleep 1; done
[1] 10593
10593 S    ignore-term.sh
[1]+  Killed                  ./ignore-term.sh

After kill $! the process still exists (ps printed its row, state S), because it ignored TERM. After kill -9 $!, jobs reports Killed. -9 and -KILL and -SIGKILL are the same. Because the process gets no chance to run code, KILL leaves temporary files, lock files, and child processes behind. The usual order is TERM, a short wait, then KILL.

Example 9: kill -INT and Ctrl+C

INT is the signal the terminal sends to the foreground job when Ctrl+C is pressed. Programs usually exit on it, and many (a server, a REPL) catch it to shut down cleanly.

sleep 100
# press Ctrl+C
echo "status $?"
sleep 100 &
kill -INT $!
jobs

^C
status 130
[1]+  Interrupt               sleep 100

The ^C is the terminal echoing the keypress. The exit status after a signal is 128 plus the signal number: 130 is 128 + 2 (INT), and a KILL ends with 137 (128 + 9). kill -INT from another shell does the same as Ctrl+C for a process that is not in the foreground, and jobs names the signal (Interrupt).

Example 10: kill -HUP

HUP (hang up) was first sent when a terminal line was closed. Programs that run for a long time, such as web servers, commonly use it to mean "reload your configuration". reload.sh shows the pattern: it handles HUP by printing a message and keeps running.

cat reload.sh
./reload.sh > reload.log &
kill -HUP $!
kill -HUP $!
kill -TERM $!
cat reload.log

#!/bin/bash
trap 'echo "reloaded config"' HUP
trap 'echo "stopping"; exit 0' TERM
echo "started"
while true; do sleep 0.2; done
[1] 3470
started
reloaded config
reloaded config
stopping

Each kill -HUP printed reloaded config and the script kept running. The TERM signal ran the second handler, and the script exited. The action for HUP is up to the program: with no handler the default action is to terminate, which is why closing a terminal ends the commands started in it (Example 17 shows how to prevent that).

Example 11: pkill

pkill is pgrep plus kill: it sends a signal to every process that matches, without looking up PIDs first. Its matching options are the same as those of pgrep (-f, -x, -u).

sleep 501 &
sleep 502 &
sleep 502 &
pkill -f 'sleep 501'
pgrep -c -f 'sleep 502'
pkill -x sleep

[3] 3514
[4] 3518
[5] 3536
[3]   Terminated              sleep 501
2
[4]-  Terminated              sleep 502
[5]+  Terminated              sleep 502

pkill -f 'sleep 501' matched the full command line and ended only the first process. pgrep -c confirmed that two sleep 502 processes remained. pkill -x sleep ended every process named exactly sleep. A signal other than TERM is given as an option: pkill -9 -f pattern or pkill -HUP nginx. Test a pattern with pgrep -a first, because pkill acts on every match, including processes the reader did not start.

Example 12: background jobs with &

A command followed by & runs in the background: the shell starts it and immediately returns the prompt.

echo "hi" ; sleep 600 &
echo "last background pid: $!"
jobs -l

hi
[3] 3693
last background pid: 3693
[3]+  3693 Running                 sleep 600 &

The shell prints [3] 3693: job number 3 and PID 3693. $! holds the PID of the most recent background command, which scripts use with wait (Example 18) and kill. The & applies to the one command before it, so echo "hi" ran in the foreground. The background job still writes to the same terminal, and output from it can appear in the middle of other work. Redirect its output (cmd > out.log 2>&1 &, see Pipes, Redirection, and Exit Codes) to avoid that.

Example 13: jobs

jobs lists the background and stopped jobs of the current shell. Job numbers (%1, %2) are per shell and are different from PIDs.

sleep 601 &
sleep 602 &
jobs
jobs -l
jobs -p
kill %1 %2
jobs

[1] 9283
[2] 9296
[1]-  Running                 sleep 601 &
[2]+  Running                 sleep 602 &
[1]-  9283 Running                 sleep 601 &
[2]+  9296 Running                 sleep 602 &
9283
9296
[1]-  Terminated              sleep 601
[2]+  Terminated              sleep 602

kill %1 %2 signals jobs by job number. After a job ends, the next jobs call reports it once and then forgets it.

Example 14: fg

fg (foreground) brings a background or stopped job back to the terminal, so it receives keyboard input again and Ctrl+C reaches it. This example stops a job first with Ctrl+Z (Example 16) to have something to bring back.

sleep 101
# press Ctrl+Z
fg
# press Ctrl+C

^Z
[1]+  Stopped                 sleep 101
sleep 101
^C

fg with no argument picks the current job (the one marked +). fg %2 picks job 2. The shell prints the command line again (sleep 101) and waits for it. Ctrl+C then ended the job.

Example 15: bg

bg resumes a stopped job in the background. The job keeps running, and the shell prompt returns.

sleep 100
# press Ctrl+Z
jobs
bg
jobs
kill %1
jobs

^Z
[1]+  Stopped                 sleep 100
[1]+  Stopped                 sleep 100
[1]+ sleep 100 &
[1]+  Running                 sleep 100 &
[1]+  Terminated              sleep 100

bg printed [1]+ sleep 100 &, which is the same line the shell prints when a command is started with &. jobs then showed Running. kill %1 ended it. Use bg after a command was started without & and turned out to take long.

Example 16: Ctrl+Z

Ctrl+Z asks the terminal to send the TSTP (terminal stop) signal to the foreground job. The job pauses, keeps its memory, and stays in the job list as Stopped. It consumes no CPU until resumed.

sleep 100
# press Ctrl+Z
jobs

^Z
[1]+  Stopped                 sleep 100
[1]+  Stopped                 sleep 100

The first [1]+ Stopped line is printed at once by the shell, and the second is the output of jobs. The state letter in ps for a stopped process is T. Resume it with fg (Example 14) or bg (Example 15), or end it with kill %1. A stopped job that the shell session never resumes is ended when the shell exits. The same job-control commands work in zsh on macOS.

Example 17: nohup

A process started from a terminal receives HUP when that terminal closes (Example 10). nohup starts a command that ignores HUP, so it survives logout, for example over SSH. Its output goes to nohup.out unless redirected.

rm -f nohup.out
nohup sh -c 'echo hello from nohup; sleep 1; echo finished' &
sleep 2
cat nohup.out

[1] 9643
nohup: ignoring input and appending output to 'nohup.out'
hello from nohup
finished

The message nohup: ignoring input and appending output to 'nohup.out' is printed to the terminal. nohup also redirects standard input from /dev/null. To choose the file, redirect explicitly: nohup ./job.sh > job.log 2>&1 &; then nohup prints nothing about nohup.out. A process that started without nohup can be protected afterwards with disown in bash. nohup exists on macOS with the same behavior.

Example 18: wait

wait pauses the shell until background jobs finish. With a PID it waits for that process and returns that process's exit status.

sleep 1 &
sleep 2 &
wait
echo "all done"
(sleep 1; exit 3) &
wait $!
echo "exit status $?"

[1] 9660
[2] 9664
[1]-  Done                    sleep 1
[2]+  Done                    sleep 2
all done
[1] 9675
[1]+  Exit 3                  ( sleep 1; exit 3 )
exit status 3

Plain wait returned after both jobs ended (about 2 seconds). wait $! waited for the subshell (sleep 1; exit 3) and $? held its status, 3. In scripts, wait is how several parallel jobs are joined before the next step. bash also has wait -n, which returns when any one job finishes.

Example 19: timeout

timeout DURATION COMMAND runs a command and sends it TERM when the duration expires. It is the standard way to stop a command that may hang.

timeout 1 sleep 5; echo "exit $?"
timeout 2 sleep 0.1; echo "exit $?"
timeout -s KILL 1 sleep 5; echo "exit $?"
timeout -k 1 1 ./ignore-term.sh; echo "exit $?"

exit 124
exit 0
Killed
exit 137
Killed
exit 137

The fourth command ran ignore-term.sh, which ignores TERM. After 1 second timeout sent TERM (ignored), and 1 second later -k sent KILL. Durations take suffixes: s (default), m, h, d. GNU timeout comes from coreutils. macOS does not ship it; with Homebrew coreutils it is available as gtimeout.

Example 20: trap

trap 'commands' SIGNAL... registers code that runs when the shell receives a signal. The pseudo-signal EXIT runs when the script ends for any reason, which makes it the usual place for cleanup. cleanup.sh creates a temporary file, removes it on exit, and reports INT separately.

cat cleanup.sh
./cleanup.sh
# press Ctrl+C
./cleanup.sh &
kill -TERM $!

#!/bin/bash
tmp=$(mktemp)
cleanup() { rm -f "$tmp"; echo "cleaned up $(basename "$tmp" | cut -c1-3)*"; }
trap cleanup EXIT
trap 'echo "got INT"; exit 130' INT
echo "working with a temp file"
while true; do sleep 0.2; done
working with a temp file
^Cgot INT
cleaned up tmp*
working with a temp file
cleaned up tmp*

Ctrl+C ran the INT handler (got INT), which called exit 130, and exit triggered the EXIT trap, so the temporary file was removed. The second run received TERM with no handler for it: the shell still ran the EXIT trap before it ended, so the cleanup line appeared. KILL cannot be trapped, so the EXIT trap does not run on kill -9. A trap is set once per script near the top, before the resource it protects is created. See Archives, tmux, and Shell Scripts for set -euo pipefail together with trap.

Interview points

• ps aux (BSD style) and ps -ef (System V style) list the same processes; -ef shows PPID, and ps -o selects columns for scripts.
• pgrep -f matches the full command line; without it only the process name is matched. pgrep exits with 1 when nothing matches.
• kill sends TERM by default. TERM, INT, and HUP can be handled; KILL and STOP cannot.
• Escalate TERM, wait, then KILL. KILL skips cleanup, so temporary files, locks, and child processes can remain.
• An exit status above 128 means death by signal: status minus 128 is the signal number (130 is INT, 137 is KILL, 143 is TERM).
• $! is the PID of the last background command, $$ is the PID of the current shell, and $? is the last exit status.
• Ctrl+C sends INT, Ctrl+Z sends TSTP (stop). fg and bg resume a stopped job in the foreground or background.
• nohup makes a command ignore HUP so it survives terminal close; disown does the same for a job already started.
• timeout exits with 124 when the limit is hit; -k adds a KILL after a grace period.
• trap ... EXIT runs cleanup on normal exit, errors, and most signals, but not on KILL.

Summary

• A process has a PID, a parent PID, a state (S, R, T, Z, D), and an owner.
• ps aux shows CPU and memory per process; ps -ef shows the parent; ps -o prints chosen columns.
• pgrep prints matching PIDs and pkill signals them; both take -f, -x, and -u.
• top and htop are live views; top -b -n 1 prints one plain-text snapshot.
• Signals are numbered messages: HUP 1, INT 2, KILL 9, TERM 15.
• kill PID sends TERM; kill -9 PID sends KILL, which a process cannot catch.
• cmd & runs a command in the background; jobs, fg, and bg manage the jobs of one shell.
• Ctrl+Z stops the foreground job; bg resumes it in the background and fg returns it to the terminal.
• nohup cmd & keeps a command alive after the terminal closes and writes to nohup.out.
• wait blocks until background jobs end and wait PID returns that job's status.
• timeout limits run time and exits with 124 on expiry.
• trap registers signal handlers; trap cleanup EXIT is the standard cleanup hook.