Books / Go in Action
Chapter 36

Build an HTTP Server from Scratch

Build an HTTP Server from Scratch

In 2003, Daniel J. Bernstein published the source code for djbdns and publicly declared: "I will pay $500 to the first person to publish a verifiable security hole in any of my software." A decade later, he paid $1,000 to a researcher who found a bug. This story is worth remembering not for the prize money, but for Bernstein's methodology: the only way to truly understand code is to write it from scratch—or to go all the way to its bottom layer.

In the Go world, net/http is one of the most widely used packages in the standard library. Nearly every Go web application uses it. But the vast majority of developers treat it as a black box: register handlers, call ListenAndServe, and wait for requests to arrive.

The goal of this chapter is not to make you abandon net/http or any framework. The goal is to make you thoroughly understand what an HTTP server is actually doing—from the Accept loop on a TCP socket, to byte-level HTTP message parsing, to the complete TLS handshake sequence. With that understanding, you can genuinely know where to apply pressure during performance tuning, incident debugging, or security hardening.

Level 1 · What You Need to Know

Why Implement from Scratch?

"From scratch" is not a rejection of frameworks. Go's net/http is mature engineering, refined over more than a decade—in most situations you should use it directly. The value of implementing from scratch lies in:

Understanding debugging clues: When your service produces "connection reset by peer" or a mysterious 504, you need to know where those errors originate. At which stage did the connection break? Did request header parsing fail? Did a goroutine exit before the response was fully written? If you've never gone deep into TCP connection handling, these problems are an opaque black box.

Understanding performance bottlenecks: Why does net/http accumulate large numbers of TIME_WAIT connections under high concurrency? Why does Keep-Alive significantly reduce latency? Why is chunked transfer encoding more appropriate than Content-Length for streaming responses? The answers to these questions are buried in the design decisions of the HTTP protocol.

Understanding the cost of frameworks: Gin, Echo, Fiber—what do these frameworks add on top of net/http? Routing, middleware, parameter binding, validation... what is the implementation cost of these features? Only after understanding the foundation can you make an informed judgment about whether a framework is appropriate for your situation.

HTTP/1.1: Protocol-Level Fundamentals

HTTP is an application-layer protocol built on top of TCP. The complete sequence of an HTTP exchange:

  1. Client establishes a TCP connection (three-way handshake)
  2. Client sends an HTTP request message (plain text format)
  3. Server parses the request, executes business logic, generates a response
  4. Server sends an HTTP response message
  5. Connection closes (or stays open in Keep-Alive mode for the next request)

Format of an HTTP/1.1 request message:

GET /api/users?page=1 HTTP/1.1\r\n
Host: example.com\r\n
Accept: application/json\r\n
Authorization: Bearer eyJhbGci...\r\n
\r\n

Response message format:

HTTP/1.1 200 OK\r\n
Content-Type: application/json\r\n
Content-Length: 42\r\n
\r\n
{"users":[{"id":1,"name":"Alice"}]}

Key conventions: line endings are \r\n (CRLF); headers and body are separated by a blank line (\r\n\r\n). These details look simple, but correctly implementing a robust HTTP parser—one that handles malicious input, oversized headers, incomplete requests—carries significant engineering complexity.

net/http Architecture Overview

The core structures in the net/http package:

The key insight for understanding net/http: every request is handled in its own goroutine. The core of Server.Serve is an Accept loop that spawns a goroutine for each new connection:

// Simplified net/http internals
for {
    rw, err := l.Accept()
    if err != nil { ... }
    c := srv.newConn(rw)
    go c.serve(connCtx) // one goroutine per connection
}

Inside c.serve is another loop: handling multiple HTTP requests over the same TCP connection (Keep-Alive).

Level 2 · Principles and Mechanics

TCP Listen → Accept Loop → Goroutine Model

At the OS level, a TCP server works like this:

  1. bind: Associate the socket with a local IP address and port
  2. listen: Mark the socket as passive; the kernel begins maintaining a connection queue
  3. accept: Dequeue a fully connected (three-way handshake complete) connection from the kernel's queue

Go's net.Listen wraps bind + listen; l.Accept() wraps the accept system call.

The kernel maintains two queues:

If accept can't keep up with the arrival rate, the Accept queue fills up, causing the kernel to reject new connections or drop SYN packets. This is one source of the "thundering herd problem" in high-concurrency servers—part of why Nginx uses multiple worker processes to accept in parallel.

Go's goroutine model offers an elegant solution: one goroutine dedicates itself to Accept (or with SO_REUSEPORT, multiple goroutines accept in parallel), and for each accepted connection a lightweight goroutine is spawned to handle it. Goroutines' low cost (initial stack ~2–8KB) makes this model viable with tens of thousands of concurrent connections.

Byte-Level HTTP/1.1 Request Parsing

Correctly parsing an HTTP request is more complex than it appears. Cases to handle:

Request line parsing:

GET /path?query=value HTTP/1.1\r\n

Method, path (including query string), protocol version, separated by spaces. The path may contain encoded characters (%20 = space), requiring URL decoding.

Header parsing:

Host: example.com\r\n
Content-Length: 100\r\n
Transfer-Encoding: chunked\r\n

One header per line, Key: Value\r\n. Header names are case-insensitive (content-type and Content-Type are the same). A header can have multiple values (comma-separated or on multiple lines).

Body reading has two modes:

Keep-Alive connections: HTTP/1.1 enables Keep-Alive by default. After handling one request, the connection is not closed; the parser reads the next request. This requires the parser to know the exact boundary of the request body (via Content-Length or chunked encoding)—otherwise the start of the next request gets consumed as the tail of the previous request's body.

Security considerations:

Keep-Alive and Connection Reuse

HTTP Keep-Alive is the default behavior in HTTP/1.1. Its significance:

Without Keep-Alive, every HTTP request requires a full TCP three-way handshake (~1.5 RTT) plus TLS handshake (~2 RTT). For clients that communicate frequently (browsers, API clients), this is enormous overhead.

With Keep-Alive, a TCP connection is reused across multiple HTTP requests—paying the handshake cost only once.

In Go's net/http implementation, the connection-handling goroutine (c.serve) does not exit after writing a response. It resets its state and begins reading the next request. Connection timeout is controlled by IdleTimeout (idle connection timeout).

HTTP/1.1 head-of-line blocking: While Keep-Alive reuses connections, HTTP/1.1 does not support sending multiple concurrent requests on the same connection—requests must be serialized: the next request can only be sent after the previous response is received. This is the core problem that HTTP/2 multiplexing solves.

Chunked Transfer Encoding

When the server doesn't know the total body length at the start of a response (e.g., a streaming report being generated on the fly), it cannot set Content-Length. HTTP/1.1 solves this with chunked transfer encoding:

HTTP/1.1 200 OK\r\n
Transfer-Encoding: chunked\r\n
\r\n
1a\r\n                    ← 26 bytes (hexadecimal)
abcdefghijklmnopqrstuvwxyz\r\n
5\r\n                     ← 5 bytes
hello\r\n
0\r\n                     ← terminating chunk (length 0)
\r\n

To parse a chunked response, the client reads a length line (hex number + CRLF), then reads that many bytes of chunk data, and repeats until the terminating chunk with length 0.

Level 3 · Code Practice

From TCP to HTTP: Writing a Complete HTTP Server

Let's build layer by layer:

Step 1: TCP Listening and the Accept Loop

package main

import (
    "bufio"
    "fmt"
    "net"
    "os"
)

func main() {
    ln, err := net.Listen("tcp", ":8080")
    if err != nil {
        fmt.Fprintf(os.Stderr, "listen error: %v\n", err)
        os.Exit(1)
    }
    defer ln.Close()
    fmt.Println("listening on :8080")

    for {
        conn, err := ln.Accept()
        if err != nil {
            fmt.Fprintf(os.Stderr, "accept error: %v\n", err)
            continue
        }
        go handleConn(conn)
    }
}

Step 2: HTTP Request Parsing

type Request struct {
    Method  string
    Path    string
    Query   string
    Proto   string
    Headers map[string]string
    Body    []byte
}

func parseRequest(reader *bufio.Reader) (*Request, error) {
    // Read request line
    line, err := reader.ReadString('\n')
    if err != nil {
        return nil, fmt.Errorf("read request line: %w", err)
    }
    line = strings.TrimRight(line, "\r\n")

    parts := strings.SplitN(line, " ", 3)
    if len(parts) != 3 {
        return nil, fmt.Errorf("malformed request line: %q", line)
    }

    method, rawPath, proto := parts[0], parts[1], parts[2]

    path, query := rawPath, ""
    if idx := strings.IndexByte(rawPath, '?'); idx >= 0 {
        path, query = rawPath[:idx], rawPath[idx+1:]
    }

    req := &Request{
        Method:  method,
        Path:    path,
        Query:   query,
        Proto:   proto,
        Headers: make(map[string]string),
    }

    // Read headers
    for {
        line, err := reader.ReadString('\n')
        if err != nil {
            return nil, fmt.Errorf("read header: %w", err)
        }
        line = strings.TrimRight(line, "\r\n")
        if line == "" {
            break // blank line: end of headers
        }

        idx := strings.IndexByte(line, ':')
        if idx < 0 {
            continue // skip malformed header lines
        }
        key := strings.ToLower(strings.TrimSpace(line[:idx]))
        val := strings.TrimSpace(line[idx+1:])
        req.Headers[key] = val
    }

    // Read body (based on Content-Length)
    if cl, ok := req.Headers["content-length"]; ok {
        var n int
        fmt.Sscanf(cl, "%d", &n)
        if n > 0 {
            req.Body = make([]byte, n)
            if _, err := io.ReadFull(reader, req.Body); err != nil {
                return nil, fmt.Errorf("read body: %w", err)
            }
        }
    }

    return req, nil
}

Step 3: Trie-Based Router

type TrieNode struct {
    children map[string]*TrieNode
    param    *TrieNode  // wildcard child (e.g., :id)
    paramKey string     // parameter name (e.g., "id")
    handler  HandlerFunc
}

type HandlerFunc func(w *ResponseWriter, r *Request, params map[string]string)

type Router struct {
    roots map[string]*TrieNode // root node per HTTP method
}

func NewRouter() *Router {
    return &Router{roots: make(map[string]*TrieNode)}
}

func (r *Router) Handle(method, pattern string, handler HandlerFunc) {
    if r.roots[method] == nil {
        r.roots[method] = &TrieNode{children: make(map[string]*TrieNode)}
    }

    parts := strings.Split(strings.Trim(pattern, "/"), "/")
    node := r.roots[method]

    for _, part := range parts {
        if part == "" {
            continue
        }
        if strings.HasPrefix(part, ":") {
            if node.param == nil {
                node.param = &TrieNode{
                    children: make(map[string]*TrieNode),
                    paramKey: part[1:],
                }
            }
            node = node.param
        } else {
            if node.children[part] == nil {
                node.children[part] = &TrieNode{children: make(map[string]*TrieNode)}
            }
            node = node.children[part]
        }
    }
    node.handler = handler
}

func (r *Router) Match(method, path string) (HandlerFunc, map[string]string) {
    root, ok := r.roots[method]
    if !ok {
        return nil, nil
    }

    parts := strings.Split(strings.Trim(path, "/"), "/")
    params := make(map[string]string)

    var match func(node *TrieNode, parts []string) HandlerFunc
    match = func(node *TrieNode, parts []string) HandlerFunc {
        if len(parts) == 0 {
            return node.handler
        }
        part := parts[0]
        rest := parts[1:]

        // Exact match takes priority
        if child, ok := node.children[part]; ok {
            if h := match(child, rest); h != nil {
                return h
            }
        }

        // Parameter match
        if node.param != nil {
            params[node.param.paramKey] = part
            if h := match(node.param, rest); h != nil {
                return h
            }
            delete(params, node.param.paramKey)
        }

        return nil
    }

    handler := match(root, parts)
    return handler, params
}

Step 4: Middleware Chain

type Middleware func(HandlerFunc) HandlerFunc

// Chain composes multiple middlewares; left to right execution order
func Chain(handler HandlerFunc, middlewares ...Middleware) HandlerFunc {
    for i := len(middlewares) - 1; i >= 0; i-- {
        handler = middlewares[i](handler)
    }
    return handler
}

func LoggingMiddleware(next HandlerFunc) HandlerFunc {
    return func(w *ResponseWriter, r *Request, params map[string]string) {
        start := time.Now()
        next(w, r, params)
        fmt.Printf("[%s] %s %s %d %v\n",
            time.Now().Format(time.RFC3339),
            r.Method, r.Path,
            w.statusCode,
            time.Since(start),
        )
    }
}

func RecoveryMiddleware(next HandlerFunc) HandlerFunc {
    return func(w *ResponseWriter, r *Request, params map[string]string) {
        defer func() {
            if rec := recover(); rec != nil {
                w.WriteHeader(500)
                w.Write([]byte("Internal Server Error"))
            }
        }()
        next(w, r, params)
    }
}

Step 5: ResponseWriter and Full Connection Handling

type ResponseWriter struct {
    conn       net.Conn
    buf        *bufio.Writer
    statusCode int
    headers    http.Header
    headerSent bool
}

func newResponseWriter(conn net.Conn) *ResponseWriter {
    return &ResponseWriter{
        conn:    conn,
        buf:     bufio.NewWriter(conn),
        headers: make(http.Header),
    }
}

func (w *ResponseWriter) WriteHeader(statusCode int) {
    if w.headerSent {
        return
    }
    w.headerSent = true
    w.statusCode = statusCode

    statusText := http.StatusText(statusCode)
    fmt.Fprintf(w.buf, "HTTP/1.1 %d %s\r\n", statusCode, statusText)

    for key, vals := range w.headers {
        for _, val := range vals {
            fmt.Fprintf(w.buf, "%s: %s\r\n", key, val)
        }
    }
    fmt.Fprintf(w.buf, "\r\n")
}

func (w *ResponseWriter) Write(data []byte) (int, error) {
    if !w.headerSent {
        w.WriteHeader(200)
    }
    n, err := w.buf.Write(data)
    w.buf.Flush()
    return n, err
}

// Complete connection handler
func handleConn(conn net.Conn, router *Router) {
    defer conn.Close()
    reader := bufio.NewReader(conn)

    for {
        // Set read deadline to prevent slow connections from tying up resources
        conn.SetReadDeadline(time.Now().Add(30 * time.Second))

        req, err := parseRequest(reader)
        if err != nil {
            return // connection closed or timed out, normal exit
        }

        w := newResponseWriter(conn)
        handler, params := router.Match(req.Method, req.Path)

        if handler == nil {
            w.WriteHeader(404)
            w.Write([]byte("Not Found"))
        } else {
            handler = Chain(handler, LoggingMiddleware, RecoveryMiddleware)
            handler(w, req, params)
        }

        // Check whether to close the connection
        if strings.ToLower(req.Headers["connection"]) == "close" {
            return
        }
        // HTTP/1.1 defaults to Keep-Alive: loop back to read the next request
    }
}

Adding TLS

To add TLS, replace the net.Listener with a TLS listener:

import "crypto/tls"

func listenTLS(addr, certFile, keyFile string) (net.Listener, error) {
    cert, err := tls.LoadX509KeyPair(certFile, keyFile)
    if err != nil {
        return nil, fmt.Errorf("load cert: %w", err)
    }

    tlsConfig := &tls.Config{
        Certificates: []tls.Certificate{cert},
        MinVersion:   tls.VersionTLS13,
        // Enable HTTP/2 via ALPN
        NextProtos: []string{"h2", "http/1.1"},
    }

    return tls.Listen("tcp", addr, tlsConfig)
}

tls.Listener wraps the underlying TCP listener and performs the TLS handshake automatically inside Accept(). The handshake happens inside the goroutine (tls.Conn.Handshake()); the main Accept loop is not blocked.

Level 4 · Advanced Topics and Edge Cases

Connection Pool for Upstream Requests

An HTTP server is rarely a terminal endpoint—it needs to make requests to backend databases, caches, and microservices. Creating a new TCP connection for every upstream call is catastrophically expensive.

Go's http.Client has a built-in connection pool (via http.Transport). If you're building a proxy or need fine-grained connection control, understanding http.Transport's key parameters is essential:

transport := &http.Transport{
    MaxIdleConns:        100,              // total idle connections in the pool
    MaxIdleConnsPerHost: 20,               // idle connections per host
    MaxConnsPerHost:     50,               // max connections per host (idle + active)
    IdleConnTimeout:     90 * time.Second, // how long idle connections are kept
    TLSHandshakeTimeout: 10 * time.Second,
    DialContext: (&net.Dialer{
        Timeout:   30 * time.Second,
        KeepAlive: 30 * time.Second,
    }).DialContext,
}

client := &http.Client{
    Transport: transport,
    Timeout:   30 * time.Second,
}

MaxIdleConnsPerHost defaults to 2. This value is severely insufficient for high-concurrency scenarios—it is usually the first thing to check when diagnosing Go HTTP client performance bottlenecks.

Zero-Copy Responses: The sendfile System Call

When a server needs to send a static file (image, video, binary) to a client, the traditional path is:

Disk → kernel buffer → user-space buffer (read) → kernel network buffer (write) → NIC

This involves 4 memory copies, 2 system calls (read + write), and 2 user-space/kernel-space context switches.

The sendfile system call transmits data directly from a file descriptor to a socket, bypassing user space:

Disk → kernel buffer → NIC (via DMA)

Go's net/http already uses sendfile when serving static files (via http.ServeFile). If you write your own file server, using io.Copy with *os.File and net.Conn, Go's runtime will automatically attempt to use sendfile:

func serveFile(conn net.Conn, path string) error {
    f, err := os.Open(path)
    if err != nil {
        return err
    }
    defer f.Close()

    info, err := f.Stat()
    if err != nil {
        return err
    }

    fmt.Fprintf(conn, "HTTP/1.1 200 OK\r\n")
    fmt.Fprintf(conn, "Content-Length: %d\r\n", info.Size())
    fmt.Fprintf(conn, "\r\n")

    // io.Copy will attempt to use sendfile under the hood
    _, err = io.Copy(conn, f)
    return err
}

net/http Source Code Walkthrough: ServeMux and ResponseWriter

Understanding the standard library's implementation helps you pinpoint strange behavior quickly.

ServeMux routing logic: Before Go 1.22, ServeMux used simple longest-prefix matching and had no support for path parameters (:id) or wildcards (*). This was one of the primary reasons Gin, Echo, and similar frameworks exist—they provide more powerful routing.

Go 1.22 significantly upgraded ServeMux, adding path parameter support (/users/{id}) and method matching (GET /users/{id}), largely eliminating the need for third-party routing libraries.

ResponseWriter buffering: net/http's ResponseWriter implementation (http.response, an internal type) buffers response body data by default until the buffer fills (4096 bytes) or the caller calls Flush(). This means:

w.Write([]byte("hello"))  // not necessarily sent immediately
// After the handler function returns, net/http flushes the buffer

For SSE (Server-Sent Events) or streaming responses, you need to flush explicitly:

flusher, ok := w.(http.Flusher)
if ok {
    flusher.Flush()
}

http.ResponseWriter state tracking: The internal response struct tracks whether headers have been sent (the wroteHeader field). Once WriteHeader is called or Write is called for the first time, the status code is fixed. Calling WriteHeader again produces a warning log but does not change the status code already sent. This is a common source of confusion for beginners: middleware calling WriteHeader(500) after a handler returns has no effect.

Performance Comparison: Hand-Written vs net/http stdlib

A fair comparison. Test environment: loopback network, wrk tool, 10 concurrent connections, 1 million requests.

Hand-written HTTP server (simplified, no middleware, returns "hello" directly): ~120,000 RPS

net/http standard library (simple handler): ~95,000 RPS

This looks counterintuitive—why is the hand-written version faster? Because it omits overhead that net/http includes for generality: URL parsing, header normalization, request context creation, etc. But this "speed" has a price: we've skipped many safety and robustness checks.

The real conclusion: For business code, net/http's performance is entirely sufficient. The value of a hand-written implementation is understanding, not performance superiority. If you genuinely need extreme performance (millions of requests per second), the direction to explore is fasthttp (zero-allocation HTTP implementation), or pushing hot-path business logic ahead of the proxy layer.

HTTP/2 Multiplexing: The Core Principle

Go's standard library supports HTTP/2 via golang.org/x/net/http2 (also built into net/http, automatically negotiated when using HTTPS).

HTTP/2's core innovation is multiplexing: on a single TCP connection, multiple HTTP requests and responses can be in-flight simultaneously, distinguished by Stream ID, with none blocking the others.

This solves two fundamental HTTP/1.1 problems:

  1. Head-of-line blocking: HTTP/1.1 requests must be serialized on a connection; HTTP/2 allows concurrency.
  2. Connection overhead: Browsers open 6–8 HTTP/1.1 connections per domain for parallel resources; HTTP/2 needs only one.

Enabling HTTP/2 requires almost no code:

server := &http.Server{
    Addr:      ":443",
    Handler:   mux,
    TLSConfig: &tls.Config{MinVersion: tls.VersionTLS12},
}
// net/http automatically negotiates HTTP/2 during ServeTLS (via ALPN)
server.ListenAndServeTLS("cert.pem", "key.pem")

HTTP/2's complexity—frame format, HPACK header compression, flow control, priority—is all handled transparently by golang.org/x/net/http2. The application layer doesn't need to be aware of it.

Final Thoughts on Frameworks

After reading this chapter, you should be equipped to answer: Should I use a framework?

The answer depends on your specific needs:

Frameworks are not bad. They are tools standing on the shoulders of net/http. Understanding net/http means understanding the foundation of every framework. When you choose a framework, you know what you're gaining—and you know what you're giving up. That trade-off awareness is the most important thing this chapter set out to give you.

Rate this chapter
4.6  / 5  (3 ratings)

💬 Comments