Chapter 21

Reflect and Unsafe: Power and Risk

Reflect and Unsafe: Power and Risk

Every statically typed language faces the same fundamental tension: the static type system guarantees safety and performance, but it strips away the program's ability to inspect and manipulate its own structure at runtime. To bridge this gap, Go provides two paths: the reflect package and the unsafe package.

reflect is the main road โ€” constrained, type-safe metaprogramming capability, at the cost of performance. unsafe is the tunnel below โ€” it bypasses all type checking and directly manipulates memory, at the cost of safety and portability. Neither should be used lightly โ€” but in the right contexts, each is irreplaceable by any other tool.

Knowing when to use them โ€” and when to firmly refuse โ€” is one of the distinguishing marks between an experienced Go engineer and a newcomer.

Level 1: What You Need to Know

When Reflection Is Truly Necessary

Reflection lets a program inspect and modify its own type information and values at runtime. It is irreplaceable in these scenarios:

Serialization and deserialization (JSON/XML/Protobuf): encoding/json's Marshal and Unmarshal must handle arbitrary user-defined structs whose concrete types are unknown at compile time. Without reflection, there is no way to traverse struct fields, read struct tags, or dynamically set field values.

Object-relational mapping (ORM): Frameworks like GORM need to read struct definitions at runtime to infer table names, column names, generate SQL statements, and map query results back into Go structs. All of this depends on reflection.

Dependency injection (DI) frameworks: wire (compile-time) needs no reflection, but runtime DI frameworks (dig, fx) must inspect function signatures at startup to infer dependencies and inject arguments automatically.

Generic test utilities: reflect.DeepEqual, testify's assert.Equal, and similar tools need to compare values of arbitrary types, requiring recursive reflection traversal over structs, slices, and maps.

RPC frameworks: Parts of gRPC-Go's internals, and net/rpc's handler registration, rely on reflection to dynamically dispatch method calls.

Reflection Is the Last Resort

However, reflection should be the last resort, not the first choice. Four reasons:

  1. Performance: A reflective call is 10โ€“100ร— slower than a direct call, and it typically blocks inlining and escape-analysis optimizations.
  2. Loss of type safety: Reflection bypasses compiler type checking โ€” errors are only discovered at runtime, not at compile time.
  3. Readability: Code heavy with reflection is hostile to readers, and IDE support is poor (no jump-to-definition, no autocompletion).
  4. Silent breakage on interface changes: If a struct field is renamed, reflection code will not produce a compile error โ€” it will silently fail at runtime.

Before reaching for reflection, ask: can this be done with generics (Go 1.18+)? With code generation? With an interface? Only when none of these work should you consider reflection.

What unsafe Is For

The unsafe package provides the ability to bypass Go's type system, and is used primarily for:

Go's official documentation warns: the behavior of the unsafe package is implementation-defined and is not covered by Go's compatibility guarantee. Code using unsafe may break in future Go versions.

Level 2: Principles

reflect.Type vs reflect.Value

The reflect package is designed around two core types:

Entry points:

var x int = 42

// Get Type
t := reflect.TypeOf(x)   // reflect.Type
fmt.Println(t)           // int
fmt.Println(t.Kind())    // int

// Get Value
v := reflect.ValueOf(x)  // reflect.Value
fmt.Println(v.Int())     // 42
fmt.Println(v.Type())    // int (reflect.Value also carries type info)

How reflect.ValueOf works internally: when you call reflect.ValueOf(x), x is boxed into an interface{} (i.e., eface). Then reflect extracts _type and data from the eface and constructs a reflect.Value:

// Simplified reflect.Value structure
type Value struct {
    typ *rtype         // type metadata (from eface._type)
    ptr unsafe.Pointer // data pointer (from eface.data)
    flag uintptr       // flags: kind, addressability, settability, etc.
}

Kind vs Type

The difference between Kind and Type is one of the most common sources of confusion in reflection:

type Celsius float64
type Fahrenheit float64

c := Celsius(100)
f := Fahrenheit(212)

tc := reflect.TypeOf(c)
tf := reflect.TypeOf(f)

fmt.Println(tc)              // main.Celsius
fmt.Println(tf)              // main.Fahrenheit
fmt.Println(tc == tf)        // false (different Type)
fmt.Println(tc.Kind())       // float64
fmt.Println(tf.Kind())       // float64
fmt.Println(tc.Kind() == tf.Kind()) // true (same Kind)

In practice: use Kind to check the underlying category ("is this a struct?"), and Type for precise matching ("is this a time.Time?").

Settability Rules

This is the most trap-laden area of reflection. A reflect.Value cannot always be modified โ€” only values obtained through certain paths are "settable":

Rule: a Value is settable if and only if it was obtained by indirecting through a pointer.

// Not settable: x is a copy
x := 42
v := reflect.ValueOf(x)
fmt.Println(v.CanSet()) // false
// v.SetInt(100) // panic: reflect.Value.SetInt using unaddressable value

// Settable: via pointer
v2 := reflect.ValueOf(&x).Elem()
fmt.Println(v2.CanSet()) // true
v2.SetInt(100)
fmt.Println(x) // 100

The deep reason: reflect.ValueOf(x) passes a copy of x (copied during boxing). Modifying the copy has no effect on the original variable, so reflect refuses the operation. reflect.ValueOf(&x) passes the address of x; .Elem() dereferences it to a Value that points at the original memory, so modification is allowed.

For structs, there is a second constraint: unexported fields are never settable:

type MyStruct struct {
    Exported   int
    unexported int
}

s := MyStruct{Exported: 1, unexported: 2}
v := reflect.ValueOf(&s).Elem()

vExported := v.FieldByName("Exported")
fmt.Println(vExported.CanSet()) // true
vExported.SetInt(99)

vUnexported := v.FieldByName("unexported")
fmt.Println(vUnexported.CanSet()) // false

The Three Laws of Reflection

The Go blog summarizes reflection in three laws:

  1. Reflection goes from interface value to reflection object. (reflect.ValueOf, reflect.TypeOf)
  2. Reflection goes from reflection object to interface value. (Value.Interface())
  3. To modify a reflection object, the value must be settable. (obtained through a pointer)

The 6 Valid Patterns for unsafe.Pointer

The Go specification (Section 17) explicitly lists the valid use patterns for unsafe.Pointer. Any use outside these patterns is undefined behavior:

Pattern 1: Convert any pointer to unsafe.Pointer

p := unsafe.Pointer(&x) // *int โ†’ unsafe.Pointer

Pattern 2: Convert unsafe.Pointer to any pointer type

var p unsafe.Pointer = ...
q := (*int)(p) // unsafe.Pointer โ†’ *int

Pattern 3: Convert unsafe.Pointer to uintptr for arithmetic (never store the uintptr)

// Only use within a single expression; never store a uintptr and convert back later
offset := unsafe.Offsetof(T{}.Field)
ptr := (*int)(unsafe.Pointer(uintptr(unsafe.Pointer(&s)) + offset))

Pattern 4: When passing uintptr arguments to syscall.Syscall

Pattern 5: Converting the result of reflect.Value.Pointer() or reflect.Value.UnsafeAddr()

Pattern 6: Converting between reflect.SliceHeader/StringHeader and slice/string (largely superseded by the newer unsafe.Slice/unsafe.String functions in Go 1.17+/1.20+)

Critical warning: uintptr is not a pointer. The garbage collector does not track uintptr. If you convert an unsafe.Pointer to uintptr, the GC may move or collect the original object before you use the uintptr again, creating a dangling reference.

Level 3: Code Practice

Building a Generic Deep Copy

The standard library provides no general deep-copy function (reflect.DeepEqual compares but does not copy). Here is a robust deep-copy implementation using reflection:

package deepcopy

import (
    "fmt"
    "reflect"
)

// DeepCopy returns a deep copy of src.
// Supports struct, slice, map, pointer, and primitive types.
// chan and func values are returned as-is (shallow).
func DeepCopy(src interface{}) interface{} {
    if src == nil {
        return nil
    }
    original := reflect.ValueOf(src)
    copy := reflect.New(original.Type()).Elem()
    deepCopyValue(original, copy)
    return copy.Interface()
}

func deepCopyValue(src, dst reflect.Value) {
    switch src.Kind() {
    case reflect.Ptr:
        srcElem := src.Elem()
        if !srcElem.IsValid() {
            return // nil pointer
        }
        dst.Set(reflect.New(srcElem.Type()))
        deepCopyValue(srcElem, dst.Elem())

    case reflect.Struct:
        t := src.Type()
        for i := 0; i < src.NumField(); i++ {
            if t.Field(i).PkgPath != "" {
                continue // skip unexported fields
            }
            deepCopyValue(src.Field(i), dst.Field(i))
        }

    case reflect.Slice:
        if src.IsNil() {
            return
        }
        dst.Set(reflect.MakeSlice(src.Type(), src.Len(), src.Cap()))
        for i := 0; i < src.Len(); i++ {
            deepCopyValue(src.Index(i), dst.Index(i))
        }

    case reflect.Map:
        if src.IsNil() {
            return
        }
        dst.Set(reflect.MakeMap(src.Type()))
        for _, key := range src.MapKeys() {
            srcVal := src.MapIndex(key)
            dstVal := reflect.New(srcVal.Type()).Elem()
            deepCopyValue(srcVal, dstVal)
            dstKey := reflect.New(key.Type()).Elem()
            deepCopyValue(key, dstKey)
            dst.SetMapIndex(dstKey, dstVal)
        }

    case reflect.Interface:
        if src.IsNil() {
            return
        }
        srcElem := src.Elem()
        dstElem := reflect.New(srcElem.Type()).Elem()
        deepCopyValue(srcElem, dstElem)
        dst.Set(dstElem)

    default:
        // Primitive types: direct assignment (bool, int, float, string, complex, ...)
        dst.Set(src)
    }
}

// Usage
type Address struct {
    Street string
    City   string
}

type Person struct {
    Name    string
    Age     int
    Address *Address
    Tags    []string
    Meta    map[string]string
}

func Example() {
    original := Person{
        Name: "Alice",
        Age:  30,
        Address: &Address{Street: "123 Main St", City: "Anytown"},
        Tags:    []string{"go", "reflect"},
        Meta:    map[string]string{"role": "engineer"},
    }

    copied := DeepCopy(original).(Person)
    copied.Name = "Bob"
    copied.Address.City = "Othertown"
    copied.Tags[0] = "java"

    fmt.Println(original.Name)         // Alice   โ€” untouched
    fmt.Println(original.Address.City) // Anytown โ€” untouched
    fmt.Println(original.Tags[0])      // go      โ€” untouched
}

Implementing a Simple JSON Encoder

The core idea behind encoding/json can be demonstrated in roughly 100 lines of reflection code:

package jsonlite

import (
    "bytes"
    "fmt"
    "reflect"
    "strings"
    "unicode"
)

// Encode serializes any Go value to a JSON string.
func Encode(v interface{}) (string, error) {
    var buf bytes.Buffer
    if err := encodeValue(&buf, reflect.ValueOf(v)); err != nil {
        return "", err
    }
    return buf.String(), nil
}

func encodeValue(buf *bytes.Buffer, v reflect.Value) error {
    for v.Kind() == reflect.Interface || v.Kind() == reflect.Ptr {
        if v.IsNil() {
            buf.WriteString("null")
            return nil
        }
        v = v.Elem()
    }

    switch v.Kind() {
    case reflect.Bool:
        if v.Bool() {
            buf.WriteString("true")
        } else {
            buf.WriteString("false")
        }
    case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
        fmt.Fprintf(buf, "%d", v.Int())
    case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
        fmt.Fprintf(buf, "%d", v.Uint())
    case reflect.Float32, reflect.Float64:
        fmt.Fprintf(buf, "%g", v.Float())
    case reflect.String:
        encodeString(buf, v.String())
    case reflect.Slice:
        if v.IsNil() {
            buf.WriteString("null")
            return nil
        }
        buf.WriteByte('[')
        for i := 0; i < v.Len(); i++ {
            if i > 0 {
                buf.WriteByte(',')
            }
            if err := encodeValue(buf, v.Index(i)); err != nil {
                return err
            }
        }
        buf.WriteByte(']')
    case reflect.Map:
        if v.IsNil() {
            buf.WriteString("null")
            return nil
        }
        buf.WriteByte('{')
        for i, key := range v.MapKeys() {
            if i > 0 {
                buf.WriteByte(',')
            }
            encodeString(buf, fmt.Sprintf("%v", key.Interface()))
            buf.WriteByte(':')
            if err := encodeValue(buf, v.MapIndex(key)); err != nil {
                return err
            }
        }
        buf.WriteByte('}')
    case reflect.Struct:
        return encodeStruct(buf, v)
    default:
        buf.WriteString("null")
    }
    return nil
}

func encodeStruct(buf *bytes.Buffer, v reflect.Value) error {
    t := v.Type()
    buf.WriteByte('{')
    first := true

    for i := 0; i < t.NumField(); i++ {
        field := t.Field(i)
        if field.PkgPath != "" { // unexported
            continue
        }
        name := field.Name
        if tag, ok := field.Tag.Lookup("json"); ok {
            parts := strings.Split(tag, ",")
            if parts[0] == "-" {
                continue
            }
            if parts[0] != "" {
                name = parts[0]
            }
        }
        if name == field.Name {
            runes := []rune(name)
            runes[0] = unicode.ToLower(runes[0])
            name = string(runes)
        }

        if !first {
            buf.WriteByte(',')
        }
        first = false
        encodeString(buf, name)
        buf.WriteByte(':')
        if err := encodeValue(buf, v.Field(i)); err != nil {
            return err
        }
    }
    buf.WriteByte('}')
    return nil
}

func encodeString(buf *bytes.Buffer, s string) {
    buf.WriteByte('"')
    for _, r := range s {
        switch r {
        case '"':
            buf.WriteString(`\"`)
        case '\\':
            buf.WriteString(`\\`)
        case '\n':
            buf.WriteString(`\n`)
        case '\r':
            buf.WriteString(`\r`)
        case '\t':
            buf.WriteString(`\t`)
        default:
            buf.WriteRune(r)
        }
    }
    buf.WriteByte('"')
}

Struct Tag Parsing Patterns

Struct tags are one of Go's core metaprogramming tools. The idiomatic parsing approach:

package main

import (
    "fmt"
    "reflect"
    "strings"
)

type User struct {
    ID       int    `db:"id" json:"id" validate:"required"`
    Username string `db:"username" json:"username,omitempty" validate:"required,min=3,max=32"`
    Email    string `db:"email" json:"email" validate:"required,email"`
    password string // unexported, no tag
}

// Extract all db tags
func getDBColumns(v interface{}) map[string]string {
    result := make(map[string]string)
    t := reflect.TypeOf(v)
    if t.Kind() == reflect.Ptr {
        t = t.Elem()
    }
    if t.Kind() != reflect.Struct {
        return result
    }
    for i := 0; i < t.NumField(); i++ {
        field := t.Field(i)
        if field.PkgPath != "" {
            continue
        }
        dbTag := field.Tag.Get("db")
        if dbTag == "" || dbTag == "-" {
            continue
        }
        result[field.Name] = dbTag
    }
    return result
}

// Parse validate tag
type ValidationRule struct {
    Required bool
    Min      int
    Max      int
    Email    bool
}

func parseValidateTag(tag string) ValidationRule {
    var rule ValidationRule
    for _, part := range strings.Split(tag, ",") {
        part = strings.TrimSpace(part)
        switch {
        case part == "required":
            rule.Required = true
        case part == "email":
            rule.Email = true
        case strings.HasPrefix(part, "min="):
            fmt.Sscanf(part, "min=%d", &rule.Min)
        case strings.HasPrefix(part, "max="):
            fmt.Sscanf(part, "max=%d", &rule.Max)
        }
    }
    return rule
}

func printTags(v interface{}) {
    t := reflect.TypeOf(v)
    for i := 0; i < t.NumField(); i++ {
        field := t.Field(i)
        if field.PkgPath != "" {
            continue
        }
        fmt.Printf("Field: %-12s | json: %-25s | db: %-12s | validate: %s\n",
            field.Name,
            field.Tag.Get("json"),
            field.Tag.Get("db"),
            field.Tag.Get("validate"),
        )
    }
}

func main() {
    printTags(User{})
    cols := getDBColumns(User{})
    for goName, dbName := range cols {
        fmt.Printf("Go: %-10s โ†’ DB: %s\n", goName, dbName)
    }
}

reflect.DeepEqual vs Custom Comparison

reflect.DeepEqual is a convenient "last resort" comparator, but it has several gotchas:

package main

import (
    "fmt"
    "reflect"
    "time"
)

func main() {
    // Case 1: nil slice vs empty slice โ€” DeepEqual distinguishes them
    var s1 []int
    s2 := []int{}
    fmt.Println(reflect.DeepEqual(s1, s2)) // false!

    // Case 2: struct containing a func โ€” always unequal
    type WithFunc struct{ F func() }
    a := WithFunc{F: func() {}}
    b := WithFunc{F: func() {}}
    fmt.Println(reflect.DeepEqual(a, b)) // false (funcs are not comparable)

    // Case 3: time.Time and time zones
    t1 := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)
    t2 := time.Date(2024, 1, 1, 8, 0, 0, 0, time.FixedZone("CST", 8*3600))
    fmt.Println(t1.Equal(t2))              // true (same instant)
    fmt.Println(reflect.DeepEqual(t1, t2)) // false! (different internal representation)

    // Case 4: cyclic references โ€” DeepEqual handles correctly
    type Node struct {
        Val  int
        Next *Node
    }
    n1 := &Node{Val: 1}
    n1.Next = n1
    n2 := &Node{Val: 1}
    n2.Next = n2
    fmt.Println(reflect.DeepEqual(n1, n2)) // true (no infinite loop)
}

Recommendation: For business object comparison, implement an Equal(other T) bool method. Use reflect.DeepEqual or testify's assert.Equal in tests, but understand their semantics before relying on them.

Level 4: Advanced Topics and Edge Cases

uintptr vs unsafe.Pointer: The Critical Distinction

unsafe.Pointer:  GC-aware opaque pointer
                 โ†’ GC tracks and updates it (if heap objects move)
                 โ†’ Keeps the pointed-to object alive

uintptr:         Unsigned integer with no pointer semantics
                 โ†’ GC does not track it
                 โ†’ Storing a uintptr does not prevent the object from being GC'd

This distinction leads to a common mistake:

// Dangerous: uintptr may become a dangling reference during GC
func dangerousAccess() {
    s := make([]byte, 1024)
    p := uintptr(unsafe.Pointer(&s[0])) // valid at this moment
    // GC may run between these two lines, collecting or moving s's backing array
    runtime.GC() // force GC to demonstrate the problem
    _ = *(*byte)(unsafe.Pointer(p)) // may access freed memory!
}

// Safe: always hold unsafe.Pointer; use uintptr only inline
func safeAccess() {
    s := make([]byte, 1024)
    p := unsafe.Pointer(&s[0])
    // Use uintptr arithmetic inline, in a single expression
    elem5 := (*byte)(unsafe.Pointer(uintptr(p) + 5))
    *elem5 = 0xFF
    runtime.KeepAlive(s) // ensure s is not GC'd before this point
}

Go 1.17+ safer pointer arithmetic with unsafe.Add and unsafe.Slice:

// Old way (error-prone)
p2 := (*byte)(unsafe.Pointer(uintptr(unsafe.Pointer(p)) + offset))

// New way (Go 1.17+, clearer intent)
p2 := (*byte)(unsafe.Add(p, offset))

// Construct a slice from pointer and length (Go 1.17+)
s := unsafe.Slice((*byte)(p), n)

Zero-Copy Conversion Between string and []byte

In high-performance paths, converting between string and []byte involves a memory copy. Using unsafe, you can do this with zero allocation:

package zerocopy

import "unsafe"

// StringToBytes converts a string to []byte with zero allocation.
// WARNING: The returned slice must NEVER be modified. Strings are immutable.
func StringToBytes(s string) []byte {
    // Go 1.20+
    return unsafe.Slice(unsafe.StringData(s), len(s))
}

// BytesToString converts []byte to string with zero allocation.
// WARNING: Do not modify b's contents while the returned string is in use.
func BytesToString(b []byte) string {
    // Go 1.20+
    return unsafe.String(unsafe.SliceData(b), len(b))
}

When this is worth it: only when:

go:nocheckptr Directive

Go 1.14 introduced pointer checking (-d=checkptr), which validates unsafe pointer operations in race-detector mode. Some legitimate but complex unsafe uses trigger false positives. In such cases, //go:nocheckptr disables the check for a specific function:

//go:nocheckptr
func unsafeButLegal() {
    // Complex unsafe operations that are actually valid...
}

This should be extremely rare. Use it only when you are certain your unsafe code is correct and checkptr is producing a false positive.

Reflection Performance Optimization: Caching reflect.Type

One of the largest reflection performance costs is repeated type lookups. In framework-level code, cache the results of reflection operations over a given type:

package cached_reflect

import (
    "reflect"
    "sync"
)

type fieldInfo struct {
    index   int
    name    string
    dbTag   string
    jsonTag string
}

type typeCache struct {
    mu     sync.RWMutex
    fields map[reflect.Type][]fieldInfo
}

var globalCache = &typeCache{
    fields: make(map[reflect.Type][]fieldInfo),
}

func getFields(t reflect.Type) []fieldInfo {
    globalCache.mu.RLock()
    if fields, ok := globalCache.fields[t]; ok {
        globalCache.mu.RUnlock()
        return fields
    }
    globalCache.mu.RUnlock()

    globalCache.mu.Lock()
    defer globalCache.mu.Unlock()
    // Double-checked locking
    if fields, ok := globalCache.fields[t]; ok {
        return fields
    }

    var fields []fieldInfo
    for i := 0; i < t.NumField(); i++ {
        f := t.Field(i)
        if f.PkgPath != "" {
            continue
        }
        fields = append(fields, fieldInfo{
            index:   i,
            name:    f.Name,
            dbTag:   f.Tag.Get("db"),
            jsonTag: f.Tag.Get("json"),
        })
    }
    globalCache.fields[t] = fields
    return fields
}

// Benchmark results (illustrative):
// BenchmarkWithoutCache    50000    24312 ns/op   (reflection on every call)
// BenchmarkWithCache     2000000     642 ns/op    (38x faster with cache)

Atomic Operations on Arbitrary Types: sync/atomic.Value

For atomically updating complex structs, sync/atomic.Value is a safe, high-level wrapper over the unsafe.Pointer + atomic operations pattern:

package main

import (
    "fmt"
    "sync/atomic"
)

type Config struct {
    Host    string
    Port    int
    Timeout int
}

var currentConfig atomic.Value

func init() {
    currentConfig.Store(&Config{Host: "localhost", Port: 8080, Timeout: 30})
}

// Atomic read of current config
func GetConfig() *Config {
    return currentConfig.Load().(*Config)
}

// Atomic config update (copy-on-write)
func UpdateConfig(host string, port int) {
    old := GetConfig()
    newCfg := *old // value copy
    newCfg.Host = host
    newCfg.Port = port
    currentConfig.Store(&newCfg)
}

func main() {
    cfg := GetConfig()
    fmt.Printf("Config: %s:%d\n", cfg.Host, cfg.Port)

    UpdateConfig("production.example.com", 443)
    cfg = GetConfig()
    fmt.Printf("Updated: %s:%d\n", cfg.Host, cfg.Port)
}

Constraints of atomic.Value: the stored type must be consistent (you cannot Store a *Config then Store a string), and in Go versions before 1.17 it does not support storing interface values.

Summary

reflect and unsafe are Go metaprogramming's two scalpels:

Both are "last resorts" โ€” before choosing them, exhaust the safer alternatives: generics, code generation, interfaces, protocol conventions. When you genuinely need them, a deep understanding of how they work is the only reliable defense against writing subtly broken code.

Rate this chapter
4.7  / 5  (10 ratings)

๐Ÿ’ฌ Comments