Reflect and Unsafe: Power and Risk
Reflect and Unsafe: Power and Risk
Every statically typed language faces the same fundamental tension: the static type system guarantees safety and performance, but it strips away the program's ability to inspect and manipulate its own structure at runtime. To bridge this gap, Go provides two paths: the reflect package and the unsafe package.
reflect is the main road โ constrained, type-safe metaprogramming capability, at the cost of performance. unsafe is the tunnel below โ it bypasses all type checking and directly manipulates memory, at the cost of safety and portability. Neither should be used lightly โ but in the right contexts, each is irreplaceable by any other tool.
Knowing when to use them โ and when to firmly refuse โ is one of the distinguishing marks between an experienced Go engineer and a newcomer.
Level 1: What You Need to Know
When Reflection Is Truly Necessary
Reflection lets a program inspect and modify its own type information and values at runtime. It is irreplaceable in these scenarios:
Serialization and deserialization (JSON/XML/Protobuf): encoding/json's Marshal and Unmarshal must handle arbitrary user-defined structs whose concrete types are unknown at compile time. Without reflection, there is no way to traverse struct fields, read struct tags, or dynamically set field values.
Object-relational mapping (ORM): Frameworks like GORM need to read struct definitions at runtime to infer table names, column names, generate SQL statements, and map query results back into Go structs. All of this depends on reflection.
Dependency injection (DI) frameworks: wire (compile-time) needs no reflection, but runtime DI frameworks (dig, fx) must inspect function signatures at startup to infer dependencies and inject arguments automatically.
Generic test utilities: reflect.DeepEqual, testify's assert.Equal, and similar tools need to compare values of arbitrary types, requiring recursive reflection traversal over structs, slices, and maps.
RPC frameworks: Parts of gRPC-Go's internals, and net/rpc's handler registration, rely on reflection to dynamically dispatch method calls.
Reflection Is the Last Resort
However, reflection should be the last resort, not the first choice. Four reasons:
- Performance: A reflective call is 10โ100ร slower than a direct call, and it typically blocks inlining and escape-analysis optimizations.
- Loss of type safety: Reflection bypasses compiler type checking โ errors are only discovered at runtime, not at compile time.
- Readability: Code heavy with reflection is hostile to readers, and IDE support is poor (no jump-to-definition, no autocompletion).
- Silent breakage on interface changes: If a struct field is renamed, reflection code will not produce a compile error โ it will silently fail at runtime.
Before reaching for reflection, ask: can this be done with generics (Go 1.18+)? With code generation? With an interface? Only when none of these work should you consider reflection.
What unsafe Is For
The unsafe package provides the ability to bypass Go's type system, and is used primarily for:
- Interoperating with C code (passing pointers through CGo)
- Implementing runtime internals (the
syncandreflectpackages themselves make heavy use ofunsafe) - Extreme performance optimization (avoiding copies by operating directly on memory layouts)
- Accessing unexported fields (an irregular technique, strongly discouraged in production code)
Go's official documentation warns: the behavior of the unsafe package is implementation-defined and is not covered by Go's compatibility guarantee. Code using unsafe may break in future Go versions.
Level 2: Principles
reflect.Type vs reflect.Value
The reflect package is designed around two core types:
reflect.Type: Represents a Go type โ static information, no concrete value attached. You can query the type's Kind, method set, field information, and which interfaces it implements.reflect.Value: Represents a Go value โ dynamic information that contains both the type and the concrete data. You can read or modify the value (when settability conditions are met), call methods, index slices, and more.
Entry points:
var x int = 42
// Get Type
t := reflect.TypeOf(x) // reflect.Type
fmt.Println(t) // int
fmt.Println(t.Kind()) // int
// Get Value
v := reflect.ValueOf(x) // reflect.Value
fmt.Println(v.Int()) // 42
fmt.Println(v.Type()) // int (reflect.Value also carries type info)
How reflect.ValueOf works internally: when you call reflect.ValueOf(x), x is boxed into an interface{} (i.e., eface). Then reflect extracts _type and data from the eface and constructs a reflect.Value:
// Simplified reflect.Value structure
type Value struct {
typ *rtype // type metadata (from eface._type)
ptr unsafe.Pointer // data pointer (from eface.data)
flag uintptr // flags: kind, addressability, settability, etc.
}
Kind vs Type
The difference between Kind and Type is one of the most common sources of confusion in reflection:
reflect.Kind: The underlying kind of the type โ an enumerated value. Go has 26 Kinds:Bool,Int,String,Struct,Ptr,Slice,Map,Func,Interface,Chan, etc. A named type and its underlying type share the same Kind.reflect.Type: The type itself, distinguishingintfromtype MyInt intโ they have different Types but the same Kind.
type Celsius float64
type Fahrenheit float64
c := Celsius(100)
f := Fahrenheit(212)
tc := reflect.TypeOf(c)
tf := reflect.TypeOf(f)
fmt.Println(tc) // main.Celsius
fmt.Println(tf) // main.Fahrenheit
fmt.Println(tc == tf) // false (different Type)
fmt.Println(tc.Kind()) // float64
fmt.Println(tf.Kind()) // float64
fmt.Println(tc.Kind() == tf.Kind()) // true (same Kind)
In practice: use Kind to check the underlying category ("is this a struct?"), and Type for precise matching ("is this a time.Time?").
Settability Rules
This is the most trap-laden area of reflection. A reflect.Value cannot always be modified โ only values obtained through certain paths are "settable":
Rule: a Value is settable if and only if it was obtained by indirecting through a pointer.
// Not settable: x is a copy
x := 42
v := reflect.ValueOf(x)
fmt.Println(v.CanSet()) // false
// v.SetInt(100) // panic: reflect.Value.SetInt using unaddressable value
// Settable: via pointer
v2 := reflect.ValueOf(&x).Elem()
fmt.Println(v2.CanSet()) // true
v2.SetInt(100)
fmt.Println(x) // 100
The deep reason: reflect.ValueOf(x) passes a copy of x (copied during boxing). Modifying the copy has no effect on the original variable, so reflect refuses the operation. reflect.ValueOf(&x) passes the address of x; .Elem() dereferences it to a Value that points at the original memory, so modification is allowed.
For structs, there is a second constraint: unexported fields are never settable:
type MyStruct struct {
Exported int
unexported int
}
s := MyStruct{Exported: 1, unexported: 2}
v := reflect.ValueOf(&s).Elem()
vExported := v.FieldByName("Exported")
fmt.Println(vExported.CanSet()) // true
vExported.SetInt(99)
vUnexported := v.FieldByName("unexported")
fmt.Println(vUnexported.CanSet()) // false
The Three Laws of Reflection
The Go blog summarizes reflection in three laws:
- Reflection goes from interface value to reflection object. (
reflect.ValueOf,reflect.TypeOf) - Reflection goes from reflection object to interface value. (
Value.Interface()) - To modify a reflection object, the value must be settable. (obtained through a pointer)
The 6 Valid Patterns for unsafe.Pointer
The Go specification (Section 17) explicitly lists the valid use patterns for unsafe.Pointer. Any use outside these patterns is undefined behavior:
Pattern 1: Convert any pointer to unsafe.Pointer
p := unsafe.Pointer(&x) // *int โ unsafe.Pointer
Pattern 2: Convert unsafe.Pointer to any pointer type
var p unsafe.Pointer = ...
q := (*int)(p) // unsafe.Pointer โ *int
Pattern 3: Convert unsafe.Pointer to uintptr for arithmetic (never store the uintptr)
// Only use within a single expression; never store a uintptr and convert back later
offset := unsafe.Offsetof(T{}.Field)
ptr := (*int)(unsafe.Pointer(uintptr(unsafe.Pointer(&s)) + offset))
Pattern 4: When passing uintptr arguments to syscall.Syscall
Pattern 5: Converting the result of reflect.Value.Pointer() or reflect.Value.UnsafeAddr()
Pattern 6: Converting between reflect.SliceHeader/StringHeader and slice/string (largely superseded by the newer unsafe.Slice/unsafe.String functions in Go 1.17+/1.20+)
Critical warning: uintptr is not a pointer. The garbage collector does not track uintptr. If you convert an unsafe.Pointer to uintptr, the GC may move or collect the original object before you use the uintptr again, creating a dangling reference.
Level 3: Code Practice
Building a Generic Deep Copy
The standard library provides no general deep-copy function (reflect.DeepEqual compares but does not copy). Here is a robust deep-copy implementation using reflection:
package deepcopy
import (
"fmt"
"reflect"
)
// DeepCopy returns a deep copy of src.
// Supports struct, slice, map, pointer, and primitive types.
// chan and func values are returned as-is (shallow).
func DeepCopy(src interface{}) interface{} {
if src == nil {
return nil
}
original := reflect.ValueOf(src)
copy := reflect.New(original.Type()).Elem()
deepCopyValue(original, copy)
return copy.Interface()
}
func deepCopyValue(src, dst reflect.Value) {
switch src.Kind() {
case reflect.Ptr:
srcElem := src.Elem()
if !srcElem.IsValid() {
return // nil pointer
}
dst.Set(reflect.New(srcElem.Type()))
deepCopyValue(srcElem, dst.Elem())
case reflect.Struct:
t := src.Type()
for i := 0; i < src.NumField(); i++ {
if t.Field(i).PkgPath != "" {
continue // skip unexported fields
}
deepCopyValue(src.Field(i), dst.Field(i))
}
case reflect.Slice:
if src.IsNil() {
return
}
dst.Set(reflect.MakeSlice(src.Type(), src.Len(), src.Cap()))
for i := 0; i < src.Len(); i++ {
deepCopyValue(src.Index(i), dst.Index(i))
}
case reflect.Map:
if src.IsNil() {
return
}
dst.Set(reflect.MakeMap(src.Type()))
for _, key := range src.MapKeys() {
srcVal := src.MapIndex(key)
dstVal := reflect.New(srcVal.Type()).Elem()
deepCopyValue(srcVal, dstVal)
dstKey := reflect.New(key.Type()).Elem()
deepCopyValue(key, dstKey)
dst.SetMapIndex(dstKey, dstVal)
}
case reflect.Interface:
if src.IsNil() {
return
}
srcElem := src.Elem()
dstElem := reflect.New(srcElem.Type()).Elem()
deepCopyValue(srcElem, dstElem)
dst.Set(dstElem)
default:
// Primitive types: direct assignment (bool, int, float, string, complex, ...)
dst.Set(src)
}
}
// Usage
type Address struct {
Street string
City string
}
type Person struct {
Name string
Age int
Address *Address
Tags []string
Meta map[string]string
}
func Example() {
original := Person{
Name: "Alice",
Age: 30,
Address: &Address{Street: "123 Main St", City: "Anytown"},
Tags: []string{"go", "reflect"},
Meta: map[string]string{"role": "engineer"},
}
copied := DeepCopy(original).(Person)
copied.Name = "Bob"
copied.Address.City = "Othertown"
copied.Tags[0] = "java"
fmt.Println(original.Name) // Alice โ untouched
fmt.Println(original.Address.City) // Anytown โ untouched
fmt.Println(original.Tags[0]) // go โ untouched
}
Implementing a Simple JSON Encoder
The core idea behind encoding/json can be demonstrated in roughly 100 lines of reflection code:
package jsonlite
import (
"bytes"
"fmt"
"reflect"
"strings"
"unicode"
)
// Encode serializes any Go value to a JSON string.
func Encode(v interface{}) (string, error) {
var buf bytes.Buffer
if err := encodeValue(&buf, reflect.ValueOf(v)); err != nil {
return "", err
}
return buf.String(), nil
}
func encodeValue(buf *bytes.Buffer, v reflect.Value) error {
for v.Kind() == reflect.Interface || v.Kind() == reflect.Ptr {
if v.IsNil() {
buf.WriteString("null")
return nil
}
v = v.Elem()
}
switch v.Kind() {
case reflect.Bool:
if v.Bool() {
buf.WriteString("true")
} else {
buf.WriteString("false")
}
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
fmt.Fprintf(buf, "%d", v.Int())
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
fmt.Fprintf(buf, "%d", v.Uint())
case reflect.Float32, reflect.Float64:
fmt.Fprintf(buf, "%g", v.Float())
case reflect.String:
encodeString(buf, v.String())
case reflect.Slice:
if v.IsNil() {
buf.WriteString("null")
return nil
}
buf.WriteByte('[')
for i := 0; i < v.Len(); i++ {
if i > 0 {
buf.WriteByte(',')
}
if err := encodeValue(buf, v.Index(i)); err != nil {
return err
}
}
buf.WriteByte(']')
case reflect.Map:
if v.IsNil() {
buf.WriteString("null")
return nil
}
buf.WriteByte('{')
for i, key := range v.MapKeys() {
if i > 0 {
buf.WriteByte(',')
}
encodeString(buf, fmt.Sprintf("%v", key.Interface()))
buf.WriteByte(':')
if err := encodeValue(buf, v.MapIndex(key)); err != nil {
return err
}
}
buf.WriteByte('}')
case reflect.Struct:
return encodeStruct(buf, v)
default:
buf.WriteString("null")
}
return nil
}
func encodeStruct(buf *bytes.Buffer, v reflect.Value) error {
t := v.Type()
buf.WriteByte('{')
first := true
for i := 0; i < t.NumField(); i++ {
field := t.Field(i)
if field.PkgPath != "" { // unexported
continue
}
name := field.Name
if tag, ok := field.Tag.Lookup("json"); ok {
parts := strings.Split(tag, ",")
if parts[0] == "-" {
continue
}
if parts[0] != "" {
name = parts[0]
}
}
if name == field.Name {
runes := []rune(name)
runes[0] = unicode.ToLower(runes[0])
name = string(runes)
}
if !first {
buf.WriteByte(',')
}
first = false
encodeString(buf, name)
buf.WriteByte(':')
if err := encodeValue(buf, v.Field(i)); err != nil {
return err
}
}
buf.WriteByte('}')
return nil
}
func encodeString(buf *bytes.Buffer, s string) {
buf.WriteByte('"')
for _, r := range s {
switch r {
case '"':
buf.WriteString(`\"`)
case '\\':
buf.WriteString(`\\`)
case '\n':
buf.WriteString(`\n`)
case '\r':
buf.WriteString(`\r`)
case '\t':
buf.WriteString(`\t`)
default:
buf.WriteRune(r)
}
}
buf.WriteByte('"')
}
Struct Tag Parsing Patterns
Struct tags are one of Go's core metaprogramming tools. The idiomatic parsing approach:
package main
import (
"fmt"
"reflect"
"strings"
)
type User struct {
ID int `db:"id" json:"id" validate:"required"`
Username string `db:"username" json:"username,omitempty" validate:"required,min=3,max=32"`
Email string `db:"email" json:"email" validate:"required,email"`
password string // unexported, no tag
}
// Extract all db tags
func getDBColumns(v interface{}) map[string]string {
result := make(map[string]string)
t := reflect.TypeOf(v)
if t.Kind() == reflect.Ptr {
t = t.Elem()
}
if t.Kind() != reflect.Struct {
return result
}
for i := 0; i < t.NumField(); i++ {
field := t.Field(i)
if field.PkgPath != "" {
continue
}
dbTag := field.Tag.Get("db")
if dbTag == "" || dbTag == "-" {
continue
}
result[field.Name] = dbTag
}
return result
}
// Parse validate tag
type ValidationRule struct {
Required bool
Min int
Max int
Email bool
}
func parseValidateTag(tag string) ValidationRule {
var rule ValidationRule
for _, part := range strings.Split(tag, ",") {
part = strings.TrimSpace(part)
switch {
case part == "required":
rule.Required = true
case part == "email":
rule.Email = true
case strings.HasPrefix(part, "min="):
fmt.Sscanf(part, "min=%d", &rule.Min)
case strings.HasPrefix(part, "max="):
fmt.Sscanf(part, "max=%d", &rule.Max)
}
}
return rule
}
func printTags(v interface{}) {
t := reflect.TypeOf(v)
for i := 0; i < t.NumField(); i++ {
field := t.Field(i)
if field.PkgPath != "" {
continue
}
fmt.Printf("Field: %-12s | json: %-25s | db: %-12s | validate: %s\n",
field.Name,
field.Tag.Get("json"),
field.Tag.Get("db"),
field.Tag.Get("validate"),
)
}
}
func main() {
printTags(User{})
cols := getDBColumns(User{})
for goName, dbName := range cols {
fmt.Printf("Go: %-10s โ DB: %s\n", goName, dbName)
}
}
reflect.DeepEqual vs Custom Comparison
reflect.DeepEqual is a convenient "last resort" comparator, but it has several gotchas:
package main
import (
"fmt"
"reflect"
"time"
)
func main() {
// Case 1: nil slice vs empty slice โ DeepEqual distinguishes them
var s1 []int
s2 := []int{}
fmt.Println(reflect.DeepEqual(s1, s2)) // false!
// Case 2: struct containing a func โ always unequal
type WithFunc struct{ F func() }
a := WithFunc{F: func() {}}
b := WithFunc{F: func() {}}
fmt.Println(reflect.DeepEqual(a, b)) // false (funcs are not comparable)
// Case 3: time.Time and time zones
t1 := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)
t2 := time.Date(2024, 1, 1, 8, 0, 0, 0, time.FixedZone("CST", 8*3600))
fmt.Println(t1.Equal(t2)) // true (same instant)
fmt.Println(reflect.DeepEqual(t1, t2)) // false! (different internal representation)
// Case 4: cyclic references โ DeepEqual handles correctly
type Node struct {
Val int
Next *Node
}
n1 := &Node{Val: 1}
n1.Next = n1
n2 := &Node{Val: 1}
n2.Next = n2
fmt.Println(reflect.DeepEqual(n1, n2)) // true (no infinite loop)
}
Recommendation: For business object comparison, implement an Equal(other T) bool method. Use reflect.DeepEqual or testify's assert.Equal in tests, but understand their semantics before relying on them.
Level 4: Advanced Topics and Edge Cases
uintptr vs unsafe.Pointer: The Critical Distinction
unsafe.Pointer: GC-aware opaque pointer
โ GC tracks and updates it (if heap objects move)
โ Keeps the pointed-to object alive
uintptr: Unsigned integer with no pointer semantics
โ GC does not track it
โ Storing a uintptr does not prevent the object from being GC'd
This distinction leads to a common mistake:
// Dangerous: uintptr may become a dangling reference during GC
func dangerousAccess() {
s := make([]byte, 1024)
p := uintptr(unsafe.Pointer(&s[0])) // valid at this moment
// GC may run between these two lines, collecting or moving s's backing array
runtime.GC() // force GC to demonstrate the problem
_ = *(*byte)(unsafe.Pointer(p)) // may access freed memory!
}
// Safe: always hold unsafe.Pointer; use uintptr only inline
func safeAccess() {
s := make([]byte, 1024)
p := unsafe.Pointer(&s[0])
// Use uintptr arithmetic inline, in a single expression
elem5 := (*byte)(unsafe.Pointer(uintptr(p) + 5))
*elem5 = 0xFF
runtime.KeepAlive(s) // ensure s is not GC'd before this point
}
Go 1.17+ safer pointer arithmetic with unsafe.Add and unsafe.Slice:
// Old way (error-prone)
p2 := (*byte)(unsafe.Pointer(uintptr(unsafe.Pointer(p)) + offset))
// New way (Go 1.17+, clearer intent)
p2 := (*byte)(unsafe.Add(p, offset))
// Construct a slice from pointer and length (Go 1.17+)
s := unsafe.Slice((*byte)(p), n)
Zero-Copy Conversion Between string and []byte
In high-performance paths, converting between string and []byte involves a memory copy. Using unsafe, you can do this with zero allocation:
package zerocopy
import "unsafe"
// StringToBytes converts a string to []byte with zero allocation.
// WARNING: The returned slice must NEVER be modified. Strings are immutable.
func StringToBytes(s string) []byte {
// Go 1.20+
return unsafe.Slice(unsafe.StringData(s), len(s))
}
// BytesToString converts []byte to string with zero allocation.
// WARNING: Do not modify b's contents while the returned string is in use.
func BytesToString(b []byte) string {
// Go 1.20+
return unsafe.String(unsafe.SliceData(b), len(b))
}
When this is worth it: only when:
- Processing MB-scale string/[]byte conversions in a tight loop
- You are certain the converted value will not be modified
- A profiler has confirmed this is a hot spot
go:nocheckptr Directive
Go 1.14 introduced pointer checking (-d=checkptr), which validates unsafe pointer operations in race-detector mode. Some legitimate but complex unsafe uses trigger false positives. In such cases, //go:nocheckptr disables the check for a specific function:
//go:nocheckptr
func unsafeButLegal() {
// Complex unsafe operations that are actually valid...
}
This should be extremely rare. Use it only when you are certain your unsafe code is correct and checkptr is producing a false positive.
Reflection Performance Optimization: Caching reflect.Type
One of the largest reflection performance costs is repeated type lookups. In framework-level code, cache the results of reflection operations over a given type:
package cached_reflect
import (
"reflect"
"sync"
)
type fieldInfo struct {
index int
name string
dbTag string
jsonTag string
}
type typeCache struct {
mu sync.RWMutex
fields map[reflect.Type][]fieldInfo
}
var globalCache = &typeCache{
fields: make(map[reflect.Type][]fieldInfo),
}
func getFields(t reflect.Type) []fieldInfo {
globalCache.mu.RLock()
if fields, ok := globalCache.fields[t]; ok {
globalCache.mu.RUnlock()
return fields
}
globalCache.mu.RUnlock()
globalCache.mu.Lock()
defer globalCache.mu.Unlock()
// Double-checked locking
if fields, ok := globalCache.fields[t]; ok {
return fields
}
var fields []fieldInfo
for i := 0; i < t.NumField(); i++ {
f := t.Field(i)
if f.PkgPath != "" {
continue
}
fields = append(fields, fieldInfo{
index: i,
name: f.Name,
dbTag: f.Tag.Get("db"),
jsonTag: f.Tag.Get("json"),
})
}
globalCache.fields[t] = fields
return fields
}
// Benchmark results (illustrative):
// BenchmarkWithoutCache 50000 24312 ns/op (reflection on every call)
// BenchmarkWithCache 2000000 642 ns/op (38x faster with cache)
Atomic Operations on Arbitrary Types: sync/atomic.Value
For atomically updating complex structs, sync/atomic.Value is a safe, high-level wrapper over the unsafe.Pointer + atomic operations pattern:
package main
import (
"fmt"
"sync/atomic"
)
type Config struct {
Host string
Port int
Timeout int
}
var currentConfig atomic.Value
func init() {
currentConfig.Store(&Config{Host: "localhost", Port: 8080, Timeout: 30})
}
// Atomic read of current config
func GetConfig() *Config {
return currentConfig.Load().(*Config)
}
// Atomic config update (copy-on-write)
func UpdateConfig(host string, port int) {
old := GetConfig()
newCfg := *old // value copy
newCfg.Host = host
newCfg.Port = port
currentConfig.Store(&newCfg)
}
func main() {
cfg := GetConfig()
fmt.Printf("Config: %s:%d\n", cfg.Host, cfg.Port)
UpdateConfig("production.example.com", 443)
cfg = GetConfig()
fmt.Printf("Updated: %s:%d\n", cfg.Host, cfg.Port)
}
Constraints of atomic.Value: the stored type must be consistent (you cannot Store a *Config then Store a string), and in Go versions before 1.17 it does not support storing interface values.
Summary
reflect and unsafe are Go metaprogramming's two scalpels:
-
reflect: For framework code that must handle arbitrary types. The cost is performance and partial loss of type safety. The key discipline is to cache reflection results and avoid using reflection on hot paths.
-
unsafe: For performance-critical scenarios and C interop. The cost is safety and portability. You must strictly follow the 6 valid patterns from the spec, and never store a
uintptrfor later conversion back to a pointer.
Both are "last resorts" โ before choosing them, exhaust the safer alternatives: generics, code generation, interfaces, protocol conventions. When you genuinely need them, a deep understanding of how they work is the only reliable defense against writing subtly broken code.