Books / How Computers Work
Chapter 12

Virtual Memory: The Illusion of Memory

Virtual Memory: The Illusion of Memory

Your laptop has 16 GB of RAM, yet it comfortably runs dozens of programs at once—Chrome consuming 3 GB, VS Code taking 1 GB, plus Steam, Slack, and a handful of background processes. The total far exceeds 16 GB. Why don't they crash into each other? How does it work?

The answer is virtual memory. The operating system gives every process a grand promise: you have an entire, private memory space all to yourself (128 TB on a 64-bit system). It's an illusion. Behind the scenes, the OS and hardware conspire to maintain that illusion, quietly tucking the pages a process actually needs into physical RAM—and when RAM runs short, spilling the rest onto disk.

Core Concepts

Why Virtual Memory Exists

Two fundamental problems it solves:

1. Process isolation: if every process shared the same physical memory, a bug in process A writing to the wrong address would corrupt process B's data and crash the system immediately. Virtual memory gives each process its own independent address space. They live in separate worlds.

2. Overcommitting memory: physical RAM is finite. Virtual memory allows programs to "use" more address space than physical RAM exists. Pages that haven't been touched yet don't need to be in RAM at all; pages that haven't been touched recently can be temporarily moved to disk.

Virtual Address vs Physical Address

Every address your program works with—including the one printed by printf("%p", &x) in C—is a virtual address. The actual location in physical RAM is the physical address.

The mapping between virtual and physical addresses is managed by the operating system and recorded in the page table. Every memory access requires translating the virtual address into a physical address before the CPU can fetch the data.

Pages and Page Tables: Managing Memory in Chunks

The OS divides memory into fixed-size chunks called pages. The default page size on most systems is 4 KB (larger "huge pages" of 2 MB or 1 GB exist for special use cases).

A virtual address is split into two parts:

64-bit virtual address structure (x86-64, 4 KB pages):

 63       48  47                  12  11       0
 ┌──────────┬──────────────────────┬──────────┐
 │ unused   │ Virtual Page Number  │  Offset  │
 │ (16 bit) │      (VPN, 36 bit)   │ (12 bit) │
 └──────────┴──────────────────────┴──────────┘
                    ↓ look up page table       ↓ pass through unchanged
             Physical Frame Number (PFN)    Low 12 bits of physical address

The page table is essentially a lookup table: virtual page number → physical frame number. The OS maintains it; the CPU's MMU (Memory Management Unit) hardware performs the lookup and translation.

Full translation flow:

Program accesses virtual address 0x7fff_5000_1234
         ↓
  MMU extracts: VPN = 0x7fff_5000_1,  offset = 0x234
         ↓
  Look up page table: VPN → physical frame 0x4A2
         ↓
  Physical address = 0x4A2_000 + 0x234 = 0x4A2_234
         ↓
  CPU accesses physical RAM at 0x4A2_234

TLB: A Hardware Accelerator for Address Translation

If looking up the page table requires a memory access, and page tables live in memory, then every memory access costs two memory accesses: one for the page table lookup, one for the actual data. That would cut memory bandwidth in half.

The TLB (Translation Lookaside Buffer) fixes this. It's a tiny, extremely fast cache inside the CPU (typically 64–1024 entries) that stores recent VPN→PFN mappings.

Address translation with TLB:

Virtual address
      ↓
[TLB lookup]
      ├── Hit (>95% of the time) → physical address immediately (1 cycle)
      │
      └── Miss → walk the page table in memory → store result in TLB → continue

The TLB benefits from the same locality that helps caches: programs repeatedly access the same pages, so the same TLB entries get reused over and over.

Page Fault: Loading on Demand

Each page table entry has a valid bit. If a process accesses a virtual address whose physical page is not currently in RAM (valid bit = 0), the MMU raises a special interrupt: a page fault.

The OS handles the page fault:

Process accesses virtual address
         ↓
    MMU checks page table
         ↓
    Valid bit = 0 (physical page not in RAM)
         ↓
    Page fault → trap into OS kernel
         ↓
    OS locates the data (swap space on disk, or a memory-mapped file)
         ↓
    OS finds a free physical page, reads data into it
         ↓
    OS updates page table (valid bit = 1, fills in PFN)
         ↓
    Return to user process; re-execute the faulting instruction
         ↓
    MMU finds valid entry; access succeeds normally

From the program's perspective this is completely transparent. It just sees that accessing an address worked. It has no idea a disk I/O happened behind the scenes.

Memory-Mapped Files (mmap) and Swap

mmap: maps a file directly into virtual address space. Reading and writing virtual addresses becomes reading and writing the file; the OS handles bringing file pages into RAM on demand. Databases like SQLite and PostgreSQL use mmap heavily to access data files—the OS acts as a caching layer automatically.

Swap space: when physical RAM runs out, the OS picks underused pages and writes them to a swap partition on disk, freeing RAM for more pressing needs. When those pages are needed again, another page fault occurs and they're read back. The cost: disk is 100–1000× slower than RAM. Heavy swap usage drags a system to a crawl—sometimes called "swap death."

Try It Yourself

Inspect a process's virtual memory map on Linux:

# View the memory map of the current shell process
cat /proc/self/maps

# Sample output:
# Address range              Perms Offset Dev   Inode  Pathname
# 5564a7a00000-5564a7a01000  r--p  00000  08:01 1234   /bin/bash
# 7f9a3c000000-7f9a3c200000  rw-p  00000  00:00 0      [heap]
# 7ffd12345000-7ffd12366000  rw-p  00000  00:00 0      [stack]

# Check swap usage
free -h

Demonstrate mmap in Python—treating a file as a byte array:

import mmap

# Create a test file
with open('/tmp/test.bin', 'wb') as f:
    f.write(b'\x00' * 4096)  # 4 KB file

# Map the file into memory
with open('/tmp/test.bin', 'r+b') as f:
    mm = mmap.mmap(f.fileno(), 0)

    # Read and write it like a bytearray
    mm[0:5] = b'Hello'
    mm.seek(0)
    print(mm.read(5))  # b'Hello'

    mm.close()

# The file on disk has been updated
with open('/tmp/test.bin', 'rb') as f:
    print(f.read(5))  # b'Hello'

🔬 Going Deeper

Multi-level page tables: solving the page table size problem

A 64-bit address space of 128 TB (47 bits) with 4 KB pages (12 bits) would require 2^35 page table entries—at 8 bytes each, that's 256 GB just for the page table itself. This is obviously impossible. The solution is a multi-level page table: a tree of tables where only the branches actually in use need to exist. x86-64 uses a 4-level page table (PML4 → PDPT → PD → PT), consuming memory only for the portions of virtual address space a process actually uses. A typical process uses only a tiny fraction of its 128 TB, so the actual page table is small.

ASLR: randomizing the virtual address space

Address Space Layout Randomization (ASLR) is a security feature built into modern operating systems. Every time a process starts, the base addresses of the stack, heap, and mmap regions are randomized. An attacker who wants to exploit a buffer overflow must know where to redirect control flow—ASLR makes those addresses unpredictable, dramatically raising the cost of exploitation. Run cat /proc/self/maps twice in a row on Linux and you'll see different addresses each time.

Huge pages: reducing TLB pressure

Databases and large-memory applications (Redis, JVM, PostgreSQL) often use 2 MB huge pages (via Linux's HugeTLBfs or Transparent Huge Pages). With 4 KB pages, a fixed-size TLB covers 4 KB × 1024 entries = 4 MB. With 2 MB huge pages, the same TLB covers 2 MB × 1024 entries = 2 GB—512× more memory with zero additional TLB entries. This dramatically reduces TLB misses for applications that work with large contiguous memory areas. PostgreSQL's huge_pages = on setting can deliver measurable throughput gains on large memory servers.

Where to learn more

Rate this chapter
4.6  / 5  (26 ratings)

💬 Comments