Zouroboros Bench

Benchmark harness for AI memory systems. Evaluates LongMemEval, LoCoMo, and ConvoMem datasets against any memory backend via the zouroboros-memory CLI. Inclu...

What to consider before installing/running: - The tool is intended for benchmarking memory backends and mostly does what it claims, but the runtime will by default open and may write to a persistent memory DB at ~/.zouroboros/memory.db (or whatever ZOUROBOROS_MEMORY_DB points to). If you have an existing Zouroboros memory DB you care about, the benchmark may modify it (it can store 'mimir-correction-*' facts). - The code calls external services: OpenAI (api.openai.com) if you set OPENAI_API_KEY, a configurable 'Mimir' gate endpoint (MIMIR_GATE_URL, default http://localhost:7820), and an Ollama local model URL. Running with OPENAI_API_KEY will send prompts and model responses (potentially dataset or memory contents) to OpenAI's API. Ensure you are comfortable with that data leaving your machine. - Mitigations: run the benchmark in an isolated environment (container, VM, or throwaway account), set ZOUROBOROS_MEMORY_DB to a temporary path so the tool does not touch your real home DB, and avoid providing OPENAI_API_KEY if you do not want cloud uploads — use an offline Ollama instance instead. Inspect or run the package source locally before npm installing globally. Review and, if necessary, modify dist/adapters/mimir-judge.js and other adapters if you want to disable the code path that writes to the memory system. - If you need a simple read-only benchmark run: ensure ZOUROBOROS_MEMORY_CLI points to a no-op or stub binary or use ZOUROBOROS_MEMORY_DB pointing to an isolated SQLite file; do not provide OPENAI_API_KEY and set MIMIR_GATE_URL to localhost or an internal endpoint you control. - If you are not comfortable with these behaviors or cannot run in an isolated environment, treat this skill as potentially risky and avoid giving it access to your real memory DB or OpenAI credentials.

Type: OpenClaw Skill Name: zouroboros-bench Version: 1.0.0 The bundle is a benchmark harness for AI memory systems that utilizes high-risk execution patterns, specifically the use of `execSync` to construct shell commands with potentially unsanitized inputs in `mimir-judge.ts` and several adapter files (e.g., `convomem-adapter.js`). It interacts with a local SQLite database at `~/.zouroboros/memory.db` and performs network requests to `api.openai.com` and local endpoints like `http://localhost:11434` (Ollama) and `http://localhost:7820` (Mimir Gate). While these actions are plausibly necessary for benchmarking and evaluating memory backends, the implementation creates a significant attack surface for shell injection and unauthorized local data manipulation via the `zouroboros-memory` CLI.