Avro Schema Guide
Basic Avro Schema
{
"type": "record",
"name": "User",
"namespace": "com.example",
"doc": "A user record",
"fields": [
{"name": "id", "type": "int"},
{"name": "name", "type": "string"},
{"name": "email", "type": ["null", "string"], "default": null},
{"name": "age", "type": "int", "default": 0},
{"name": "active", "type": "boolean", "default": true},
{"name": "created_at", "type": "long", "logicalType": "timestamp-millis"}
]
}
Avro Primitive Types
| Type | Description | Size |
|---|---|---|
| null | No value | 0 |
| boolean | Boolean | 1 bit |
| int | 32-bit signed integer | variable |
| long | 64-bit signed integer | variable |
| float | 32-bit float | 4 bytes |
| double | 64-bit float | 8 bytes |
| bytes | Byte sequence | variable |
| string | Unicode string | variable |
Complex Types
// Array
{"type": "array", "items": "string"}
// Map
{"type": "map", "values": "int"}
// Union (nullable field)
["null", "string"]
// Enum
{
"type": "enum",
"name": "Status",
"symbols": ["ACTIVE", "INACTIVE", "PENDING"]
}
// Nested record
{
"type": "record",
"name": "Address",
"fields": [
{"name": "street", "type": "string"},
{"name": "city", "type": "string"}
]
}
Schema Evolution Rules
| Change | Compatible? |
|---|---|
| Add field with default | ✓ Forward + Backward |
| Remove field with default | ✓ Forward + Backward |
| Add field without default | Forward only |
| Remove required field | Backward only |
| Change field type | ✗ Breaking |
Python Example
import avro.schema
from avro.datafile import DataFileWriter
from avro.io import DatumWriter
schema = avro.schema.parse(open('user.avsc').read())
writer = DataFileWriter(open("users.avro", "wb"), DatumWriter(), schema)
writer.append({"id": 1, "name": "Alice", "email": None, "active": True})
writer.close()