Description

Agent observability — monitors tool invocations, LLM calls, token usage, costs, and anomalies with pluggable alerts and a real-time dashboard.

README (SKILL.md)

Claw Reliability — Agent Observability Skill

Name: Claw Reliability
Author: fiddyrod

You are an AI agent with observability capabilities. Use this skill to monitor, analyze, and report on agent behavior.

When to use this skill

When the user asks to monitor agent activity, check agent health, or review agent metrics
When the user asks about tool usage, failure rates, costs, or token consumption
When the user asks to set up alerts or check for anomalies
When the user asks for a reliability report or dashboard

Available commands

Start monitoring

Run the monitoring daemon to begin collecting metrics:

cd {baseDir} && python3 scripts/monitor.py start --config {baseDir}/config.yaml

Show metrics summary

Display current metrics for the active session or all sessions:

cd {baseDir} && python3 scripts/monitor.py summary

Show tool report

Display tool invocation success/failure rates:

cd {baseDir} && python3 scripts/monitor.py tools

Show cost report

Display token usage and cost projections:

cd {baseDir} && python3 scripts/monitor.py costs

Check for anomalies

Run anomaly detection on recent activity:

cd {baseDir} && python3 scripts/monitor.py anomalies

List alerts

Show recent alerts and their severity:

cd {baseDir} && python3 scripts/monitor.py alerts

Configure alert destination

Set up where alerts are sent (Discord, Slack, log file, etc.):

cd {baseDir} && python3 scripts/monitor.py configure-alerts --destination discord --webhook-url \x3CURL>

Launch dashboard

Start the FastAPI + React dashboard for visual monitoring:

cd {baseDir} && python3 dashboard/backend/main.py

Then open http://localhost:8777 in a browser.

How metrics are collected

This skill reads OpenClaw gateway events and session transcripts to extract:

Tool invocations: tool name, success/fail, duration, arguments
LLM calls: model, tokens in/out, latency, estimated cost
Session lifecycle: start/end times, message counts
Anomalies: repeated failures, cost spikes, loop detection

All data is stored in a local SQLite database at {baseDir}/data/metrics.db.

Alert thresholds (defaults, configurable)

Tool failure: 3+ consecutive errors on the same tool
Cost spike: Token spend exceeds 2x the rolling 1-hour average
Loop detection: Same tool called 10+ times in a single agent turn
Unusual activity: Tool called that has never been used before in this agent's history

Notes

This skill does NOT send data externally unless you configure an alert destination
All metrics stay local in SQLite
The dashboard runs on localhost only by default

Usage Guidance

What to check before installing: - Understand data access: this skill reads OpenClaw session transcripts (default ~/.openclaw) which may contain sensitive tool arguments, tokens, or file paths. That behavior is expected for an observability tool, but be sure you want that data read and stored locally. - Config vs. instructions mismatch: SKILL.md references a {baseDir} placeholder and the registry metadata mentions agents.defaults.workspace, but the code reads monitoring.openclaw_state_dir (config.yaml) and uses the skill's package path as base. Before running, open config.yaml and confirm monitoring.openclaw_state_dir and monitoring.db_path point to the locations you expect. - External endpoints: alerts are only sent when you explicitly configure a webhook (e.g., Discord). If you configure a webhook, the skill will send sanitized alert text — sanitization is heuristic (regex-based) and can miss secrets or custom token formats. Only send alerts to trusted endpoints and consider reviewing or extending scripts/alerts/sanitizer.py to match your secret formats. - Dashboard CDN risk: the frontend loads React/Babel from public CDNs. If you operate in an air-gapped or high-security environment, serve frontend assets locally or avoid launching the dashboard. The README already recommends this. - Run in a controlled environment first: consider running the monitor in a VM or isolated environment, inspect data written to data/metrics.db and data/alerts.log, and run monitor.py summary/tools/costs to validate behavior before enabling any external alert destinations. - Small code inconsistencies: there are duplicate modules (scripts/ and scripts/scripts/) and a minor mismatch between declared registry config and code. These look like packaging quirks rather than malicious behavior, but you may want to inspect the source files in the bundle if you are cautious. If you want higher assurance, ask the skill author for clarity on the intended mapping of {baseDir} / agents.defaults.workspace and for a way to run the parser in read-only mode (no writes) or to opt out of front-end CDN usage.

Capability Assessment

⚠ Purpose & Capability

The skill's code matches the described purpose: it parses OpenClaw session transcripts, stores metrics in a local SQLite DB, runs anomaly checks, and can send alerts to webhooks and run a local dashboard. However the registry metadata / SKILL.md declare a required config path (agents.defaults.workspace) and use a {baseDir} placeholder, while the code actually reads monitoring.openclaw_state_dir (default ~/.openclaw) and uses its own internal base path. That mismatch between declared config requirements and what the code uses is an incoherence worth noting.

⚠ Instruction Scope

The SKILL.md instructions are narrowly scoped to starting the monitor, viewing metrics, configuring alerts, and launching the dashboard. The instructions explicitly state the skill will read gateway events and session transcripts (sensitive data) and store metrics locally, which the parser code does. Two things to watch: (1) the SKILL.md uses a {baseDir} placeholder which is not referenced in the code paths (the monitor uses its package base path and config.yaml), and (2) the dashboard frontend loads React/Babel from public CDNs — a runtime supply-chain / privacy risk if you expose the dashboard or run in a high-security environment. Alerts to external webhooks only occur if you configure them, but sanitizer is heuristic and may not catch all secrets.

✓ Install Mechanism

No network install script or remote downloads in an install spec. Files are bundled with the skill and the runtime requires python3; dependencies are listed in requirements.txt. There is no automatic fetching of arbitrary code during install. The dashboard's frontend, however, references public CDNs at runtime (client-side), which is a separate supply-chain consideration described in the README.

ℹ Credentials

The skill requests no environment variables and does not demand unrelated cloud credentials. It does read user OpenClaw state (default ~/.openclaw) and will write a local SQLite DB and logs under the skill directory (data/metrics.db, data/alerts.log). Those accesses are proportional to monitoring, but they involve sensitive transcripts and tool arguments — the README and sanitizer claim redaction, but redaction is pattern-based and not guaranteed to catch every secret.

✓ Persistence & Privilege

The skill does not set always:true and does not request elevated platform privileges. It writes its own artifacts (DB, logs, config.yaml) in its directory and reads from the user's OpenClaw state directory. It does not modify other skills or system-wide agent settings. Writing webhook URLs into config.yaml (configure-alerts) is expected behavior for a notifier.

Version History

v1.0.6

- Improved Discord alert configuration: fixed webhook URL processing in discord_alerter.py - README and documentation updates for clarity and accuracy

v1.0.5

- Initial introduction of a modular alerting and monitoring system. - Added scripts for core monitoring, analysis, parsing, and storage. - Introduced pluggable alert support with Discord and log file alert backends. - Provided comprehensive tests and code restructuring into organized modules.

v1.0.4

claw-reliability 1.0.4 - Updated internal logic in scripts/alerts/base.py and scripts/analyzer.py. - No user-facing command or configuration changes.

v1.0.3

- Documentation updated in README.md; no functional or code changes. - Clarified usage instructions, available commands, and alert configuration. - No changes to monitored metrics, commands, or skill behavior.

v1.0.2

- Added sanitizer module for alert content in scripts/alerts/sanitizer.py. - Improved alerting to better handle and sanitize Discord alert messages. - Updated metadata to include new configuration requirement: agents.defaults.workspace. - General code and documentation maintenance across multiple files.

v1.0.0

Initial release of Claw Reliability — Agent Observability Skill. - Provides agent monitoring for tool use, LLM calls, token usage, and anomalies - Supports real-time pluggable alerts and a FastAPI + React dashboard - Tracks metrics via local SQLite storage with user-configurable alert thresholds - Offers commands for viewing summaries, costs, anomalies, and alert configuration - By default, all metrics and alerts remain local unless an alert destination is set

Metadata

Slug claw-reliability

Version 1.0.6

License MIT-0

All-time Installs 1

Active Installs 1

Total Versions 6

Frequently Asked Questions

What is Claw Reliability?

Agent observability — monitors tool invocations, LLM calls, token usage, costs, and anomalies with pluggable alerts and a real-time dashboard. It is an AI Agent Skill for Claude Code / OpenClaw, with 169 downloads so far.

How do I install Claw Reliability?

Run "/install claw-reliability" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.

Is Claw Reliability free?

Yes, Claw Reliability is completely free, licensed under MIT-0. You can download, install and use it at no cost.

Which platforms does Claw Reliability support?

Claw Reliability is cross-platform and runs anywhere OpenClaw / Claude Code is available (linux, darwin).

Who created Claw Reliability?

It is built and maintained by fiddyrod (@fiddyrod); the current version is v1.0.6.

More Skills

Claw Reliability