Smartness Eval Open Source

OpenClaw 智能度综合评伌技能。围绕 14 个维度（含规划能力、幻觉控制）输出综合评分、证据、风险与趋势。对齐 CLEAR/T-Eval/Anthropic 行业标准。

What to check before installing or running: - Review scripts/eval.py and scripts/state_probe.py (the run-time engine) to confirm validate_command() actually blocks inline execution, absolute paths, path traversal, shell=True, and network unless --llm-judge is explicitly used. - Understand that the test commands call other workspace scripts (e.g., scripts/cognitive-kernel-v6.py, scripts/api-fallback-v5.py) which are NOT bundled here; inspect those external scripts (in your OpenClaw workspace) to ensure they don't read or send data beyond what you expect. - The skill will read many state files and a reasoning SQLite DB. Ensure those files do not contain secrets or sensitive user data you don't want evaluated tools to access. - The LLM-judge feature is opt-in and requires setting DEEPSEEK_API_KEY or OPENAI_API_KEY; do not set those env vars unless you accept sending the aggregated summary described in docs. - If you need stronger assurance, provide the full eval.py source (runtime portion that builds/validates/executes commands) for review; that would raise confidence to high.

Type: OpenClaw Skill Name: smartness-eval-open-source Version: 0.3.3 The skill bundle is a comprehensive evaluation and benchmarking framework for the OpenClaw agent. The core logic in `scripts/eval.py` collects system metrics from local workspace files (e.g., `state/error-tracker.json`, `.reasoning/reasoning-store.sqlite`) and executes functional tests defined in `config/task-suite.json`. While the skill executes subprocesses, it implements a `validate_command` function to mitigate risks by enforcing a whitelist (only `python3`), blocking inline code execution (`-c`, `exec(`), and preventing path traversal. External network communication is limited to an optional 'LLM Judge' feature that requires explicit user opt-in and API keys. The code and extensive documentation (README.md, SKILL.md) are consistent with the stated purpose of measuring agent performance across 14 dimensions.