← Back to Skills Marketplace
Memory Bench Designer
by
TatsuKo Tsukimi
· GitHub ↗
· v0.1.0
· MIT-0
88
Downloads
0
Stars
0
Active Installs
1
Versions
Install in OpenClaw
/install memory-bench-designer
Description
Designs a custom agent-memory benchmark for the user's specific use case. Activate when the user asks which memory strategy fits their agent, how to evaluate...
Usage Guidance
Before installing or enabling this skill, verify the following: (1) Where does the 'memory-bench' runner come from? The skill expects to run 'memory-bench' but the registry gives no install or binary source — ask the publisher for an official install/instruction or a trusted release URL. (2) Templates referenced in SKILL.md (templates/scenario.yaml.tmpl, templates/weights.yaml.tmpl) are not included in the package; request those or confirm how they are created. (3) Running the skill will write files into your current working directory and will run a local CLI that may download ~90 MB of model data from external hosts (Hugging Face or similar). If you are uncomfortable with filesystem writes or network/model downloads, do not enable autonomous runs; prefer manual invocation. (4) Because the skill's source and homepage are unknown, exercise extra caution: ask the owner for provenance, an install guide, and a checksum/verified release of the runner. If those clarifications are provided (runner origin, templates included, or an explicit install spec), the coherence issues would be resolved; otherwise, treat the skill as suspicious and avoid giving it autonomous execution privileges.
Capability Analysis
Type: OpenClaw Skill
Name: memory-bench-designer
Version: 0.1.0
The memory-bench-designer skill is a legitimate tool for evaluating AI memory strategies (e.g., BM25, ACT-R, Embeddings). It follows a structured four-stage process to elicit user requirements, generate YAML configuration files, and execute a local benchmarking CLI tool (`memory-bench`). The instructions in SKILL.md and the supporting documentation in the references/ and examples/ directories are consistent, transparent, and strictly aligned with the stated purpose. There is no evidence of data exfiltration, malicious prompt injection, or unauthorized system access; the mentioned model download (~90MB for sentence-transformers) is standard for the semantic retrieval tasks described.
Capability Tags
Capability Assessment
Purpose & Capability
SKILL.md clearly expects to run an external runner (memory-bench run ...) and to use template files (templates/scenario.yaml.tmpl, templates/weights.yaml.tmpl). The registry metadata declares no required binaries and includes no templates or install instructions. Either the metadata is incomplete or the skill assumes tools/files that are not provided — that is an incoherence between claimed functionality and declared requirements.
Instruction Scope
Runtime instructions tell the agent to (a) conduct multi-turn elicitation, (b) write scenario-<name>.yaml and weights-<name>.yaml into the user's current working directory, (c) invoke a CLI command that produces results/<name>/results.md and results.json, and (d) read and interpret results.md. The instructions reference template files that are not present in the file manifest. They also note that the runner will download a ~90 MB sentence-transformers model on first run. These behaviors involve filesystem writes, executing a local CLI, and network downloads — all beyond what's declared in the metadata.
Install Mechanism
There is no install spec (instruction-only), which is low-risk in itself. However, the skill assumes the 'memory-bench' CLI exists on PATH and will cause a model download (Hugging Face / sentence-transformers) when invoked with --embedding. The absence of an install step or a declared source for the runner binary means it's unclear how that binary would be obtained or whether it is safe/trusted.
Credentials
The skill declares no required environment variables, credentials, or config paths, and SKILL.md does not ask for secrets. That is proportionate. Note: network activity (model download) and reading/writing local files will still occur; no explicit credential access is requested.
Persistence & Privilege
always:false (good) and no install-time persistence is requested. The default autonomous-invocation setting is enabled (disable-model-invocation:false) — normal for skills — but combined with the instruction to execute a CLI and write files, this increases the impact if the agent runs the skill without clear user approval. The skill does not request to modify other skills or system-wide settings.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install memory-bench-designer - After installation, invoke the skill by name or use
/memory-bench-designer - Provide required inputs per the skill's parameter spec and get structured output
Version History
v0.1.0
Initial release: 4-stage elicitation skill + companion Python runner benchmarking 5 memory strategies (Recency, BM25, ACT-R, Embedding, Composite) across 8 dimensions in 4 families (Exploration, Ranking, Adaptation, Maintenance). Three worked examples (game-AI, NPC cognition, coding agent) demonstrate different use cases produce different winners. Runner at https://github.com/TatsuKo-Tsukimi/memory-bench-designer
Metadata
Frequently Asked Questions
What is Memory Bench Designer?
Designs a custom agent-memory benchmark for the user's specific use case. Activate when the user asks which memory strategy fits their agent, how to evaluate... It is an AI Agent Skill for Claude Code / OpenClaw, with 88 downloads so far.
How do I install Memory Bench Designer?
Run "/install memory-bench-designer" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is Memory Bench Designer free?
Yes, Memory Bench Designer is completely free, licensed under MIT-0. You can download, install and use it at no cost.
Which platforms does Memory Bench Designer support?
Memory Bench Designer is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created Memory Bench Designer?
It is built and maintained by TatsuKo Tsukimi (@tatsuko-tsukimi); the current version is v0.1.0.
More Skills