Unix Internals: Processes, File Descriptors, and Syscalls

Every container, every Kubernetes Pod, every shell pipeline is built on the same handful of Unix primitives that predate all of them by decades: processes, file descriptors, signals, and the syscall boundary between a program and the kernel. Container Internals covers namespaces and cgroups — this post covers what's underneath those: the Unix model they're built on top of.

---

Everything Is a File

Unix's defining design decision: files, directories, devices, pipes, and sockets are all accessed through the same small set of operations — open, read, write, close — regardless of what's actually on the other end.

flowchart TD

    API["open() / read() / write() / close()"]

    API-->Regular["Regular file <br/> /home/user/data.txt"]
    API-->Device["Device file <br/> /dev/sda, /dev/null"]
    API-->Pipe["Pipe <br/> shell | between commands"]
    API-->Socket["Socket <br/> network connection"]
    API-->TTY["Terminal <br/> /dev/tty"]

A program that writes to a file, writes to a pipe, and writes to a network socket calls the exact same write() syscall in all three cases — it's the kernel's Virtual File System (VFS) layer that dispatches the call to whatever actually backs that file descriptor. This uniformity is why cat /dev/urandom, echo hello > /dev/null, and piping one process's output into another all just work with the same handful of tools, without each needing special-cased code.

---

File Descriptors

A file descriptor (fd) is a small integer — an index into a per-process table the kernel maintains, pointing at an open file description (which in turn points at the actual file/pipe/socket).

flowchart LR

    Process["Process"]-->FDTable["Per-process FD table"]

    FDTable-->FD0["fd 0 → stdin"]
    FDTable-->FD1["fd 1 → stdout"]
    FDTable-->FD2["fd 2 → stderr"]
    FDTable-->FD3["fd 3 → /var/log/app.log"]

fd	Name	Default target
0	stdin	Keyboard / whatever is piped in
1	stdout	Terminal / whatever output is redirected to
2	stderr	Terminal, separately from stdout

# Redirect stdout to a file, leave stderr on the terminal
myprogram > output.log

# Redirect stderr into stdout's destination — order matters
myprogram > output.log 2>&1

# Inspect a running process's open file descriptors
ls -l /proc/<pid>/fd

ls -l /proc/<pid>/fd is one of the most useful debugging commands in the Unix toolbox — it shows exactly what files, sockets, and pipes a process currently has open, which is often the fastest way to diagnose a leaked file handle or figure out what a mystery process is actually connected to.

File Descriptors Across fork()

A child process inherits copies of its parent's file descriptor table — both point at the same underlying open file description, so a file position advanced by one is visible to the other, but closing a descriptor in one process doesn't close it in the other. This is exactly the mechanism a shell pipeline relies on: cmd1 | cmd2 works because the shell creates a pipe, then fork()s twice, and each child inherits one end of that same pipe before execing into the actual command.

---

Processes: fork, exec, and wait

Unix creates new processes with a two-step dance that looks unusual until you've seen it: fork duplicates the calling process, and exec replaces the current process image with a different program entirely.

sequenceDiagram
    participant Shell
    participant Child

    Shell->>Shell: fork()
    Note over Shell: Returns twice — once in parent, once in child
    Shell->>Child: (child) exec("/bin/ls")
    Note over Child: Child's memory is replaced with /bin/ls
    Child-->>Shell: Child exits
    Shell->>Shell: wait() — reap child, get exit status

Call	What it does
fork()	Creates a near-identical copy of the calling process — same memory, same open file descriptors, different PID. Returns twice: 0 in the child, the child's PID in the parent
exec() (family: execve, execvp, ...)	Replaces the calling process's memory image with a new program. Same PID, same fd table — but entirely different code running
wait() / waitpid()	Parent blocks until a child exits, retrieving its exit status

Every time a shell runs a command, it fork()s itself, and the child immediately exec()s into the requested program — which is exactly why a shell script that runs cd /tmp inside a subshell doesn't change the parent shell's directory: the subshell is a separate process with its own copy of the working directory, and exec never sends anything back to the parent.

Process States

flowchart LR
    New["New"]-->Ready["Ready <br/> (runnable, waiting for CPU)"]
    Ready-->Running["Running"]
    Running-->Ready
    Running-->Blocked["Blocked <br/> (waiting on I/O, a lock, a signal)"]
    Blocked-->Ready
    Running-->Zombie["Zombie <br/> (exited, not yet reaped)"]
    Zombie-->Terminated["Terminated <br/> (reaped by parent's wait())"]

A zombie process has already exited — its resources are freed — but its exit status is kept around in the process table until the parent calls wait() to retrieve it. A parent that never calls wait() (or exits without reaping) leaves zombies behind; if the parent itself dies first, the kernel re-parents the child to init (PID 1), which exists partly to reap these orphaned processes automatically. This is precisely why a container's PID 1 matters — a container process that never reaps its own children (because it was never written to be an init system) can quietly accumulate zombies for the lifetime of the container.

---

Signals

A signal is an asynchronous notification delivered to a process — the kernel's way of interrupting a running process to tell it something happened, without the process having to poll for it.

Signal	Default action	Typical use
SIGTERM (15)	Terminate	The polite way to ask a process to shut down — catchable, so a process can clean up first
SIGKILL (9)	Terminate	Uncatchable, unblockable — the kernel simply removes the process, no cleanup possible
SIGINT (2)	Terminate	Sent by Ctrl-C from the controlling terminal
SIGHUP (1)	Terminate	Originally "terminal hung up"; commonly repurposed to mean "reload config"
SIGCHLD (17/20)	Ignore	Sent to a parent when a child exits — a well-behaved parent handles this by calling wait()
SIGSTOP / SIGCONT	Stop / Continue	Suspend and resume a process, uncatchable like SIGKILL

# Send SIGTERM (the default) — gives the process a chance to clean up
kill <pid>

# Send SIGKILL — no cleanup, no exceptions
kill -9 <pid>

# See what a process does with each signal, and which it blocks
cat /proc/<pid>/status | grep -i sig

A process can register a handler for most signals — running its own code instead of the default action — which is exactly how a well-behaved server catches SIGTERM, finishes in-flight requests, and exits cleanly, versus getting SIGKILLed mid-request with zero warning. This is the same distinction behind Kubernetes' Pod termination: kubectl delete sends SIGTERM first and only escalates to SIGKILL after terminationGracePeriodSeconds expires.

---

The Syscall Boundary

A system call is the only door between a normal program running in user space and the kernel running in a separate, privileged mode — a program can't just read a disk block or send a network packet directly; it asks the kernel to do it via a syscall.

flowchart LR

    UserSpace["User Space <br/> your program"]
    UserSpace-->|"syscall (trap into kernel mode)"|KernelSpace["Kernel Space <br/> privileged"]
    KernelSpace-->|"return"|UserSpace

# Trace every syscall a program makes — the single most useful
# low-level debugging tool for "what is this program actually doing"
strace -f -e trace=open,read,write ./myprogram

# Count syscalls by type and time spent in each
strace -c ./myprogram

The library functions most programs actually call (fopen, malloc, printf) aren't syscalls themselves — they're user-space library code (libc) that eventually makes the relevant syscalls (open, brk/mmap, write) on the program's behalf, often batching or buffering to avoid a syscall on every single operation, since crossing into kernel mode has real, measurable overhead.

---

Pipes and IPC

An anonymous pipe (| in the shell) is a unidirectional, in-kernel buffer connecting one process's stdout to another's stdin — no file on disk involved:

# Shell pipeline: ps writes to a pipe, grep reads from the same pipe
ps aux | grep nginx

flowchart LR
    PS["ps aux <br/> writes to fd 1"]-->Pipe["Kernel pipe buffer"]
    Pipe-->Grep["grep nginx <br/> reads from fd 0"]

A named pipe (FIFO) does the same thing but with a filesystem path any two unrelated processes can open, instead of requiring a common ancestor to set it up via fork(). Beyond pipes, Unix's other inter-process communication mechanisms — shared memory, message queues, Unix domain sockets — all exist for the same underlying reason: processes have separate, isolated memory by default, and IPC is how they deliberately punch a hole through that isolation to exchange data.

---

Process Memory Layout

Every process sees its own virtual address space, laid out in the same general shape regardless of what the program actually does:

flowchart TD
    Stack["Stack <br/> (grows down — local variables, call frames)"]
    Gap["  ↕  unused  "]
    Heap["Heap <br/> (grows up — malloc'd memory)"]
    BSS["BSS <br/> (uninitialized global/static variables)"]
    Data["Data <br/> (initialized global/static variables)"]
    Text["Text <br/> (the compiled program code, read-only)"]

    Stack --- Gap --- Heap --- BSS --- Data --- Text

Segment	Contains
Text	The compiled machine code — read-only, often shared between multiple instances of the same program
Data	Global/static variables that have an explicit initial value
BSS	Global/static variables with no initial value (zero-initialized)
Heap	Dynamically allocated memory (malloc), grows upward as the program requests more
Stack	Function call frames and local variables, grows downward, one per thread

This virtual layout is what the kernel's memory manager maps onto physical RAM via page tables — the same page-table mechanism Container Internals and Kubelet Internals rely on when a cgroup enforces a memory limit: the limit is enforced against the physical pages actually backing this virtual layout, not the virtual address space size itself (which is often far larger than the limit).

---

Permissions

Every file has an owning user (UID) and group (GID), and three permission bits (read/write/execute) for each of owner, group, and everyone else:

-rwxr-xr--  1 alice  developers  4096 Jul 30 10:00 deploy.sh
# owner (alice): rwx     group (developers): r-x     other: r--

The setuid bit is the interesting exception — a setuid executable runs with the file owner's privileges rather than the invoking user's, which is how passwd (owned by root) lets an ordinary user change their own password despite that requiring writing to a root-owned file. It's also one of the more common privilege-escalation vectors when misapplied, which is why container security contexts (allowPrivilegeEscalation: false) explicitly exist to prevent a containerized process from ever gaining privileges this way.

---

Key Takeaways

Concept	Summary
Everything is a file	Files, devices, pipes, and sockets all go through the same open/read/write/close API, dispatched by the VFS layer
File descriptor	A small integer indexing a per-process table; inherited (as copies) across fork()
fork / exec / wait	Create a process copy, replace its memory image with a new program, and let the parent retrieve its exit status
Zombie / orphan	A zombie is exited-but-unreaped; an orphan is re-parented to PID 1 when its original parent dies first
Signal	An asynchronous kernel-to-process notification; SIGTERM is catchable and polite, SIGKILL is neither
Syscall	The only boundary between user-space code and the privileged kernel; strace shows every one a program makes
Pipe	An in-kernel, unidirectional buffer connecting one process's stdout to another's stdin
Process memory layout	Text/data/BSS/heap/stack — the same virtual layout cgroup memory limits enforce against physical pages
setuid	Runs a program as its file owner rather than the invoker — powerful and a classic privilege-escalation risk

Containers didn't invent isolation or resource control — namespaces and cgroups are relatively recent additions on top of a process/file-descriptor/signal model that's been stable since the 1970s. Understanding that older layer is what turns "the container won't die" or "why does this pod have a thousand zombie processes" from a mystery into a two-minute diagnosis.

---

Related Posts

• Container Internals: What a Container Really Is — namespaces, cgroups, and OverlayFS, built directly on top of the process and syscall model covered here
• Kubernetes Pod Internals: What a Pod Really Is — the pause container's PID 1 role, and why zombie reaping matters inside a Pod
• Kubelet Internals: The Node Agent That Runs Everything — how cgroup memory limits enforce against the physical pages backing a process's virtual memory layout
• Ansible: Agentless Configuration Management — SSH plus these same Unix process/fd primitives is the entire mechanism Ansible's "agentless" model runs on