Memory Four-Layer Architecture: Session Context, Daily Logs, MEMORY.md and Vector Index

Chapter 26　The Four-Layer Memory Architecture: Session Context → Daily Logs → MEMORY.md → Vector Index

"The model only 'remembers' what gets saved to disk — there is no hidden state." — OpenClaw Design Documentation

26.1　Why Layered Memory?

Large language models are fundamentally stateless. When an inference call completes, all intermediate activations and attention weights evaporate. Without an external persistence mechanism, an agent loses all memory the moment a conversation ends — it forgets what you told it last week, can't recall your name-spelling preferences, and has no way to review decisions made three days ago.

OpenClaw's Memory system resolves this fundamental tension through a four-layer architecture: externalizing "things worth remembering" into persistent files on disk, then reinjecting them into the Context Window at the appropriate moment.

The four layers, from most ephemeral to most durable, are:

Session Context
    ↓ may be promoted after compaction
Daily Logs
    ↓ periodically consolidated
MEMORY.md (Long-term Memory)
    ↓ indexed in parallel
Vector Index (SQLite + Embeddings)

Each layer has a clearly defined storage location, write timing, and read strategy. Understanding these four layers is central to mastering OpenClaw's state management.

26.2　Layer 1: Session Context

26.2.1　Storage Location and Format

~/.openclaw/agents/<agentId>/sessions/<sessionId>.jsonl

Each line is a JSON message record following the standard conversation history format:

{"role":"user","content":"Analyze this sales report","timestamp":"2026-04-26T09:00:00Z"}
{"role":"assistant","content":"Sure, please upload the file...","timestamp":"2026-04-26T09:00:02Z"}
{"role":"tool","name":"read_file","result":"...","timestamp":"2026-04-26T09:00:05Z"}

26.2.2　Loading Timing

Session Context is the most "live" layer — it resides in the Context Window throughout the current conversation. When an agent starts a new session, the corresponding .jsonl file is read in full (if it exists), and each new message turn is appended to the same file.

Key constraint: The Context Window is finite (e.g., 200K tokens). When session history grows too long, Compaction or Pruning must be triggered (see Chapter 27).

26.2.3　Information Types Best Suited for This Layer

Session Context is a high-density, short-lifetime information layer. Content is rich but fleeting; once the session ends, anything not explicitly persisted is gone forever.

26.3　Layer 2: Daily Logs

26.3.1　Storage Location and Format

~/.openclaw/workspace/<workspaceId>/memory/YYYY-MM-DD.md

For example:

~/.openclaw/workspace/myproject/memory/2026-04-26.md
~/.openclaw/workspace/myproject/memory/2026-04-25.md

The format is plain Markdown, with content structure left to the agent's discretion. Entries typically appear as timestamped append records:

## 2026-04-26

### 09:15 — Sales Report Analysis

User uploaded Q1-2026-sales.xlsx. East China region grew 23% YoY. User wants
to prepare a PPT presentation next week.

### 14:30 — Code Review

Reviewed PR #142 on auth-service. Found a potential SQL injection point;
notified the user.

26.3.2　Loading Timing

Every time a session starts, OpenClaw automatically loads the today's and yesterday's log files (if they exist). This ensures that at the start of a new session, the agent is aware of significant events from the past 48 hours without requiring the user to re-explain context.

# Loading policy in config (default)
dailyLogDays: 2   # load today + yesterday

26.3.3　Write Timing

Daily Logs are written in two situations:

1. Compaction Pre-flush: When the Context Window is about to fill, the agent writes important information to the daily log before compression (see Chapter 27 for details).
2. Session-end hook: When a session closes normally, the agent may optionally summarize key takeaways into the log.

26.3.4　Information Types Best Suited for This Layer

Daily Logs form a medium-density, periodic-lifecycle information layer — a buffer zone between Session Context and MEMORY.md.

26.4　Layer 3: Long-term Memory (MEMORY.md)

26.4.1　Storage Location

~/.openclaw/workspace/<workspaceId>/MEMORY.md

This is a single, curated Markdown file that stores core knowledge that spans time.

26.4.2　Content Example

# Memory Index

- [User Profile](user_profile.md) — Prefers concise, direct answers; dislikes excessive formalities
- [Architecture Decisions](arch_decisions.md) — Microservices architecture; gRPC for inter-service communication
- [Coding Standards](coding_standards.md) — TypeScript strict mode; no `any` type
- [Key Contacts](contacts.md) — Tech lead: Zhang San ([email protected])

MEMORY.md typically serves as an index file pointing to more detailed sub-files; it can also be a compact inline summary.

26.4.3　Loading Timing

MEMORY.md is automatically loaded only in Primary Private Sessions. This design is intentional:

• Primary Private Session: Has full access to long-term memory
• Sub-session: Not loaded by default; only receives minimal task context
• Group Session: Does not load personal MEMORY.md; only loads workspace-shared context

# Session types and MEMORY.md loading rules
session:
  primary_private:
    load_memory: true
  sub_session:
    load_memory: false
    inherit_context: minimal
  group_session:
    load_memory: false
    load_shared_context: true

26.4.4　Write Timing

Updates to MEMORY.md are deliberate and selective, occurring when:

1. Dreaming process: A background consolidation process periodically promotes important information from Daily Logs into MEMORY.md
2. Explicit agent decision: The agent determines a piece of information has long-term value and writes it proactively
3. User instruction: The user explicitly says "remember this"

26.4.5　Information Types Best Suited for This Layer

26.5　Layer 4: Vector Index

26.5.1　Storage Location

~/.openclaw/memory/<agentId>.sqlite

The SQLite database contains two key components:

-- BM25 full-text search virtual table
CREATE VIRTUAL TABLE memory_fts USING fts5(content, ...);

-- Vector storage table (sqlite-vec extension)
CREATE TABLE memory_embeddings (
    id INTEGER PRIMARY KEY,
    content TEXT,
    embedding FLOAT[768],  -- dimension depends on embedding model
    source_file TEXT,
    created_at INTEGER
);

26.5.2　Loading Timing

The Vector Index is never "loaded" into the Context Window — it is an on-demand query layer. When an agent needs to recall certain information, it issues a semantic search query; the retrieved results are injected into the context as text fragments.

User question → Agent judges retrieval needed → Vector query → Top-K chunks returned → Injected into Context → Response generated

26.5.3　Content Sources

The Vector Index indexes content from the other three layers: Daily Logs, MEMORY.md, and fragments from historical sessions that are flagged as worth retrieving. It is a derived layer — all its content originates from upper-layer files and can be rebuilt at any time through re-indexing.

# Rebuild vector index (no original information is lost)
openclaw memory rebuild-index --workspace myproject

26.5.4　Information Types Best Suited for This Layer

26.6　Files as Database: The Design Philosophy in Depth

26.6.1　Why Markdown Instead of a Database?

OpenClaw made a distinctive technical choice: Markdown files rather than a relational database or proprietary format as the primary storage. The reasoning:

1. Human Readability

# Anyone can inspect agent memory with standard tools
cat ~/.openclaw/workspace/myproject/MEMORY.md
grep "database" ~/.openclaw/workspace/myproject/memory/2026-04-26.md

By contrast, SQLite binary files and proprietary vector database formats require special tools to read.

2. Version Control Friendly

# Memory can be placed under Git version control
cd ~/.openclaw/workspace/myproject
git init
git add memory/ MEMORY.md
git commit -m "Agent memory snapshot: 2026-04-26"

Every change to MEMORY.md can be tracked, rolled back, and diffed.

3. Human Editable

Users can open Markdown files directly, manually correct wrong memories, delete outdated entries, or inject external knowledge. This transparency is impossible with closed-source memory systems.

4. No Hidden State

"The model only 'remembers' what gets saved to disk — there is no hidden state."

This design declaration means: all of an agent's "memory" is visible on the filesystem. There is no mysterious internal state — what you see is everything it knows.

26.6.2　SQLite's Role as an Acceleration Layer

SQLite (the Vector Index layer) is the only "non-human-readable" store in the entire architecture, and its role is that of an acceleration layer rather than primary storage:

Markdown files (authoritative source)
    ↓ async indexing
SQLite (retrieval acceleration layer)
    ↓ can be rebuilt at any time

The vectors and BM25 indexes in SQLite are entirely derived from Markdown files. If SQLite is corrupted or deleted:

openclaw memory rebuild-index --workspace myproject
# Re-reads all Markdown files and rebuilds the index
# No information is lost

This design ensures Markdown is the Single Source of Truth, and SQLite only makes search faster.

26.7　Memory and the Context Window

The Context Window is the total information an LLM actually "sees" during each inference call. The core mission of the Memory system is deciding which information is worth occupying precious Context space.

Context Window (200K token example)
┌──────────────────────────────────────────────┐
│ System Prompt (AGENTS.md / SOUL.md)     ~8K  │
│ MEMORY.md (if primary session)          ~4K  │
│ Daily Logs (today + yesterday)          ~8K  │
│ Retrieved Memory Chunks (vector search) ~4K  │
│ Current Session History                 rest  │
│ Available for new messages             ~176K  │
└──────────────────────────────────────────────┘

As session history grows and available space shrinks, the Compaction mechanism intervenes (see Chapter 27).

26.8　Memory Loading Differences Across Session Types

Sub-session design follows the principle of minimal context: it receives only the minimum information needed to complete the current sub-task. This avoids polluting the task context with unrelated historical information and also reduces token consumption.

26.9　Practical Guide: Which Information Belongs in Which Layer

Decision Tree

How long does this information need to be retained?
├── Only today / this conversation → Session Context (auto-managed, no action needed)
├── Need to recall within a few days → Daily Logs (auto-written during Compaction)
├── Keep permanently, needed in every session → MEMORY.md (explicit write or Dreaming promotion)
└── Need to find via semantic search → Vector Index (auto-indexed, no action needed)

Layer Selection Principles

Common Mistakes

❌ Wrong: Writing every conversation summary into MEMORY.md
   → Result: MEMORY.md balloons to 50K tokens, consuming enormous Context each time

✓ Correct: Only write "decisions that affect long-term agent behavior" into MEMORY.md
   → Result: MEMORY.md stays at 2-4K tokens, highly refined

❌ Wrong: Relying on Session Context to preserve important decisions (no persistence)
   → Result: After the session ends, critical decisions are lost forever

✓ Correct: At session end, trigger a Memory Flush to write key points to Daily Logs

26.10　Chapter Summary

OpenClaw's four-layer Memory architecture is a systematic solution to the fundamental constraint of "LLM statelessness":

1. Session Context — Working memory for the current conversation; auto-managed
2. Daily Logs — Buffer for recent memory; append-written, auto-loaded
3. MEMORY.md — Curated long-term memory; maintained carefully, loaded in primary sessions
4. Vector Index — Semantic retrieval acceleration layer; derived, rebuildable

The "files as database" philosophy runs throughout: Markdown is the authoritative source, and SQLite is merely a derived layer that accelerates retrieval. This design makes the agent's memory system fully transparent and fully controllable — no hidden state of any kind.

In the next chapter, we dive into the Compaction mechanism — how the system performs "memory compression" when the Context Window approaches its limit without losing critical information.

Next: Chapter 27 — Compaction Algorithm: Trigger Formula, Pre-flush Mechanism, and Long-session Information Preservation