Description

Chat Vitals - Monitor chat conversation health with real-time insights. Tracks conversation quality metrics: first-try success rate, promise fulfillment, tok...

README (SKILL.md)

📊 Chat Vitals v1.1.0

Name: Chat Vitals
Author: carolava

Monitor your AI conversation health like a doctor checks vital signs. Auto-collection + Real-time dashboard + Actionable insights.

✨ What's New in v1.1.0

🚀 Auto-Collection (Zero Friction)

Before: Manual tracking for every turn
Now: One command, automatic tracking

vitals start claude-sonnet-4.6
# All conversations tracked automatically!

🖥️ Real-Time Dashboard

Live terminal UI with health monitoring:

vitals dashboard

Shows:

⚡ Live health score (0-100) with color coding
📊 Real-time metrics refresh
🚨 Instant alerts for problems
📈 Token usage tracking

🚀 Quick Start

# 1. Start monitoring (one command!)
vitals start claude-sonnet-4.6

# 2. View real-time dashboard (in another terminal)
vitals dashboard

# 3. Check status anytime
vitals status

# 4. Generate report when done
vitals report

📋 All Commands

Command	Description
`vitals start [model]`	Start auto-monitoring session
`vitals dashboard`	Launch real-time dashboard
`vitals status`	Show current session status
`vitals summary`	Quick session summary
`vitals complete`	Mark session as complete
`vitals report`	Generate detailed report

🎯 Core Metrics

Metric	Description	Healthy	Warning	Danger
First-Try Success	% tasks without rework	≥70%	50-70%	\x3C50%
Rework Count	Corrections per task	0	1-2	>2
Promise Fulfillment	% promises delivered	≥80%	60-80%	\x3C60%
Plan Inflation	Actual / Promised turns	≤1.3x	1.3-2.0x	>2.0x
Token Efficiency	Value per token	≥0.15	0.08-0.15	\x3C0.08

🏥 Health Status

Score	Status	Emoji	Color
85-100	Excellent	🟢	Green
70-84	Good	🟡	Yellow
50-69	Warning	🟠	Orange
\x3C50	Critical	🔴	Red

📊 Sample Output

📊 Chat Vitals Dashboard
──────────────────────────────────────────────────

Session: a1b2c3d4
Model:   claude-sonnet-4.6
Started: 2026-04-08T11:34:27

Health Score:
🟢 85/100 - Excellent
[██████████████████████████████] 85%

Key Metrics:
──────────────────────────────────────────────────
  ✅ First-Try Success: 85%
  ✅ Rework Count: 0
  ⚠️  Promise Fulfillment: 75%
  ✅ Plan Inflation: 1.1x
  ✅ Token Efficiency: 0.22

Session Stats:
  💬 Total Turns: 3
  🔢 Total Tokens: 1,030
  📊 Avg Tokens/Turn: 343

⚙️ Configuration

Edit ~/.openclaw/skills/chat-vitals/config.json:

{
  "monitor": {
    "token_thresholds": {
      "report_daily": 50000
    },
    "health_thresholds": {
      "first_try_success_rate": {
        "excellent": 85,
        "good": 70,
        "warning": 50
      }
    }
  },
  "patterns": {
    "correction_keywords": ["不对", "错了", "重新"],
    "promise_patterns": ["接下来我会", "首先让我"]
  }
}

📁 Project Structure

chat-vitals/
├── vitals                  # ⭐ Simple CLI entry point
├── SKILL.md
├── README.md
├── config.json
├── scripts/
│   ├── collector.py        # Core data collection
│   ├── analyzer.py         # Metric analysis
│   ├── reporter.py         # Report generation
│   ├── auto_collector.py   # ⭐ Auto-collection
│   └── dashboard.py        # ⭐ Real-time dashboard
└── tests/
    └── test_vitals.py      # Test suite

🧪 Testing

cd ~/.openclaw/skills/chat-vitals
./tests/run_tests.sh

# Test new features
vitals start test-model
vitals status
vitals dashboard  # In a real terminal

🔌 Advanced Usage

Legacy Manual Mode

python3 scripts/collector.py create ...
python3 scripts/collector.py record ...
python3 scripts/collector.py complete ...

Programmatic Access

from llmchat_vitals.scripts import auto_collector

# Start monitoring
auto_collector.auto_start_session("gpt-4")

# Record turn
auto_collector.auto_record_turn(user_input, model_output)

# Get live summary
summary = auto_collector.get_session_summary()
print(f"Health: {summary['health_score']}/100")

🛣️ Roadmap

Auto-collection (v1.1.0)
Real-time dashboard (v1.1.0)
Simplified CLI (v1.1.0)
Webhook integrations (Feishu/Slack)
Intent drift detection
Web dashboard
Multi-model comparison
Prompt optimization suggestions

🤝 Contributing

Issues and PRs welcome!

📄 License

MIT License

Usage Guidance

This package appears to do what it says: local collection, per-session JSON files in ~/.openclaw/skills/chat-vitals/data, analysis, dashboard, and report generation. Before installing: 1) Review the mismatched CLI/module names (the README/SKILL.md refer to 'vitals', 'monitor', and different Python package names) — the skill may require you to run the scripts directly (python3 scripts/*.py) rather than a 'vitals' command. 2) Confirm OpenClaw will call the provided hook functions (on_user_message, on_assistant_message, on_session_end) if you expect automatic collection; otherwise auto-collection may not work. 3) Note that all collected data is stored unencrypted under ~/.openclaw/skills/chat-vitals — if your conversations are sensitive, restrict file permissions or disable auto-collection. 4) The code uses subprocess to invoke analyzer.py in reporter.py (local invocation only); if you add webhook/networking later, re-review network code. If you want extra assurance, run the test suite and inspect the scripts in a sandbox before enabling auto-collection.

Capability Analysis

Type: OpenClaw Skill Name: chat-vitals Version: 1.1.0 The skill is a conversation monitoring utility designed to track LLM performance metrics like token efficiency and 'rework' rates. It functions entirely locally, saving session data to the user's home directory (~/.openclaw). Review of the Python scripts (scripts/collector.py, scripts/analyzer.py, scripts/auto_collector.py) confirms the logic is consistent with the stated purpose, with no evidence of data exfiltration, persistence mechanisms, or malicious prompt injection. The use of subprocess.run in reporter.py and os.system in dashboard.py is restricted to internal operations and UI management.

Capability Assessment

ℹ Purpose & Capability

The skill claims auto-collection and a simple CLI ('vitals start ...') and provides hook functions (on_user_message, on_assistant_message, on_session_end) for OpenClaw to call — the implementation (auto_collector.py / collector.py / dashboard.py / reporter.py) matches the stated monitoring/dashboard purpose. Minor incoherences: documentation and examples use inconsistent CLI/module names (vitals, monitor, llmchat_vitals, llm_monitor) and there's no obvious top-level executable named 'vitals' in the repo. These are likely sloppy packaging/documentation issues rather than malicious behavior.

✓ Instruction Scope

SKILL.md and the scripts operate only on local data under ~/.openclaw/skills/chat-vitals (creating session JSON files, reading config.json). Runtime instructions reference local CLI commands and local config paths declared in the README/SKILL.md, and the code's behavior (collecting turns, analyzing sessions, rendering dashboard) stays within that scope. There are no instructions to read unrelated system files, environment variables, or to transmit data to external endpoints.

✓ Install Mechanism

No install spec is provided (instruction-only), and the code is pure Python using the standard library. There are no remote downloads or obscure install URLs. The repo includes scripts that will be written to disk as part of the skill installation — standard for OpenClaw skills — but no package manager or external fetch is used.

✓ Credentials

The skill declares only python3 as a required binary and does not request environment variables, credentials, or unrelated configuration paths. The code reads and writes only to its own skill directory (~/.openclaw/skills/chat-vitals). Optional webhook dependencies are only noted in comments and no network libraries are used in the shipped code.

✓ Persistence & Privilege

The skill stores session and report files under its own skill directory and does not request always:true or modify other skills' configs. It uses regular file I/O within its directory and does not attempt to persist system-wide configuration or escalate privileges.

Version History

v1.1.0

**Major update with auto-collection and real-time dashboard features:** - Added zero-friction auto-collection for conversation monitoring—start with one command. - Introduced a live real-time dashboard for session health, scores, and instant alerts. - Unified health scoring with a 4-tier color system and expanded actionable metric reports. - Simplified command-line interface with new and updated commands for quick status, summaries, and detailed reports. - Core metrics and health threshold configurations are now easier to customize. - Legacy manual tracking is still available, with enhanced programmatic access for advanced users.

Metadata

Slug chat-vitals

Version 1.1.0

License MIT-0

All-time Installs 0

Active Installs 0

Total Versions 1

Frequently Asked Questions

What is Chat Vitals?

Chat Vitals - Monitor chat conversation health with real-time insights. Tracks conversation quality metrics: first-try success rate, promise fulfillment, tok... It is an AI Agent Skill for Claude Code / OpenClaw, with 93 downloads so far.

How do I install Chat Vitals?

Run "/install chat-vitals" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.

Is Chat Vitals free?

Yes, Chat Vitals is completely free, licensed under MIT-0. You can download, install and use it at no cost.

Which platforms does Chat Vitals support?

Chat Vitals is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).

Who created Chat Vitals?

It is built and maintained by 珈乐不困 (@carolava); the current version is v1.1.0.

More Skills

Chat Vitals