← 返回 Skills 市场
ralphesber

LangSmith CLI

作者 Ralph Esber · GitHub ↗ · v1.1.0 · MIT-0
cross-platform ✓ 安全检测通过
213
总下载
1
收藏
0
当前安装
5
版本数
在 OpenClaw 中安装
/install langsmith-cli
功能描述
Query and analyze LangSmith traces with natural language or structured commands. Use when the user asks about LangSmith runs, trace failures, latency, cost,...
使用说明 (SKILL.md)

LangSmith CLI Skill

CLI: scripts/langsmith.py. Requires LANGSMITH_API_KEY in env (or ~/.zshrc).

No second API key needed — the ask command fetches and formats traces as structured context for your agent to analyze. No trace data is sent to any third-party LLM.

Commands

Tier 0 — Ask (agent Q&A over traces)

python3 scripts/langsmith.py ask "\x3Cquestion>" --project \x3Cname> [--since 24h] [--limit 50]

Fetches recent runs and prints them as structured JSON context. Your agent reads the output and answers the question — no external LLM calls, no data leaving your machine beyond the LangSmith API.

Examples:

  • ask "why is my chain slow this week" --project my-project
  • ask "what do failing runs have in common" --project my-project --since 7d
  • ask "did the system prompt change on Friday affect output quality" --project my-project

Tier 1 — Situational Awareness

python3 scripts/langsmith.py runs \x3Cproject> [--since 2h] [--status error|success] [--limit 20]
python3 scripts/langsmith.py cost \x3Cproject> [--since 7d]       # token spend by chain/node
python3 scripts/langsmith.py latency \x3Cproject> [--since 24h]   # p50/p95/p99 per run name

Tier 2 — Before/After Comparisons

python3 scripts/langsmith.py diff \x3Cproject> --before \x3CISO_date> --after \x3CISO_date>
python3 scripts/langsmith.py prompt-diff \x3Crun_id_a> \x3Crun_id_b>

diff compares avg latency, error rate, cost, output length across two time windows. prompt-diff shows side-by-side system prompts + outputs for two specific runs.

Tier 3 — Deep Analysis (stubs, expand as needed)

python3 scripts/langsmith.py cluster-failures \x3Cproject> [--since 7d]
python3 scripts/langsmith.py replay \x3Crun_id>

Auth Setup

export LANGSMITH_API_KEY=\x3Cyour-key>
# or add to ~/.zshrc

Test with: python3 scripts/langsmith.py runs \x3Cproject> --limit 3

Security & Data Flow

This skill makes outbound network requests only to api.smith.langchain.com (the LangSmith API). That's it.

  • LANGSMITH_API_KEY — sent as an HTTP header to api.smith.langchain.com only. Never logged or stored.
  • Trace data — fetched from LangSmith and printed to stdout for your agent to read. No trace data is sent to any third-party LLM or external service.
  • No second API key required — the ask command outputs structured trace context for your existing agent to analyze, rather than making its own LLM calls.
  • No telemetry — the script collects no usage data.

The script is ~300 lines of pure Python with no obfuscation. Audit it at scripts/langsmith.py.

API Reference

See references/langsmith-api.md for endpoint details and run object schema.

安全使用建议
This skill appears to do exactly what it claims: fetch LangSmith traces and print summaries. Before installing, verify you intend the agent to see raw trace inputs/outputs (these may contain secrets or PII). Confirm you trust the environment's agent configuration so that printed traces are not forwarded to external LLMs you don't control. Review scripts/langsmith.py yourself (it's included) and limit the LANGSMITH_API_KEY scope or rotate the key if you are unsure. If you want to be extra cautious, run the script locally from a terminal rather than allowing autonomous agent runs, and test it on a non-production project first.
功能分析
Type: OpenClaw Skill Name: langsmith-cli Version: 1.1.0 The langsmith-cli skill is a legitimate tool for querying and analyzing LangSmith traces. The core logic in scripts/langsmith.py interacts exclusively with the official LangSmith API (api.smith.langchain.com) using the user-provided LANGSMITH_API_KEY. The script performs standard data retrieval, aggregation, and formatting tasks (e.g., calculating latency percentiles and token costs) without any signs of obfuscation, data exfiltration to third parties, or malicious execution patterns.
能力评估
✓ Purpose & Capability
Name/description match the behavior: the script calls LangSmith endpoints and requires a single LANGSMITH_API_KEY. No extraneous services, binaries, or credentials are requested.
ℹ Instruction Scope
SKILL.md and the script restrict network I/O to the LangSmith API and print fetched trace data to stdout for local analysis. Note: printing traces to stdout is necessary for 'ask' usage but means sensitive trace contents will be visible to the agent and any destinations the agent is configured to use.
✓ Install Mechanism
No install spec; the skill is instruction-only with an included Python script. Nothing is downloaded at install time and no archives or external installers are used.
✓ Credentials
Only LANGSMITH_API_KEY is required and is the declared primary credential. The key is used solely as an HTTP header to api.smith.langchain.com; no other secrets or unrelated env vars are requested.
✓ Persistence & Privilege
The skill is not always-enabled and does not request system-wide persistence or modify other skills. It can be invoked autonomously by the agent (platform default) but the package itself does not escalate privileges.
如何使用
  1. 确保已安装 OpenClaw（本地或 Docker 部署）
  2. 在对话框中输入安装命令：/install langsmith-cli
  3. 安装完成后，直接呼叫该 Skill 的名称或使用 /langsmith-cli 触发
  4. 根据 Skill 的参数说明提供必要输入，即可获得结构化输出
版本历史
v1.1.0
Removed internal LLM calls from ask command. Traces are now returned as structured context for the agent to analyze — no third-party LLM calls, no ANTHROPIC_API_KEY needed. Only outbound connection is to api.smith.langchain.com.
v1.0.3
Fixed metadata frontmatter: added openclaw.requires.env and primaryEnv for LANGSMITH_API_KEY so the registry correctly declares required credentials.
v1.0.2
Added structured credential and data_sharing declarations to frontmatter; clarified that ask command sends truncated trace data to Anthropic API.
v1.0.1
Added Security & Data Flow section to SKILL.md documenting outbound API usage (LangSmith + Anthropic only, no telemetry).
v1.0.0
Initial release — ask, runs, cost, latency, diff, prompt-diff commands. Uses correct POST query API with is_root filter.
元数据
Slug langsmith-cli
版本 1.1.0
许可证 MIT-0
累计安装 0
当前安装数 0
历史版本数 5
常见问题

LangSmith CLI 是什么？

Query and analyze LangSmith traces with natural language or structured commands. Use when the user asks about LangSmith runs, trace failures, latency, cost,... 它是一个面向 Claude Code / OpenClaw 的 AI Agent Skill 插件，目前累计下载 213 次。

如何安装 LangSmith CLI？

在 OpenClaw 或 Claude Code 对话框中运行命令「/install langsmith-cli」即可一键安装，无需额外配置。

LangSmith CLI 是免费的吗？

是的，LangSmith CLI 完全免费，采用 MIT-0 许可证，可自由下载、安装和使用。

LangSmith CLI 支持哪些平台？

LangSmith CLI 跨平台运行，可在任意部署了 OpenClaw / Claude Code 的环境中使用（cross-platform）。

谁开发了 LangSmith CLI？

由 Ralph Esber（@ralphesber）开发并维护，当前版本 v1.1.0。

推荐 Skills
self-improving agent
Captures learnings, errors, and corrections to enable continuous improvement. Use when: (1) A command or operation fails unexpectedly, (2) User corrects Clau...
⬇ 463283 · by pskoett
Skill Vetter
Security-first skill vetting for AI agents. Use before installing any skill from ClawdHub, GitHub, or other sources. Checks for red flags, permission scope, and suspicious patterns.
⬇ 259410 · by spclaudehome
Polymarket
Query Polymarket prediction markets. Check odds, find trending markets, search events, track price movements.
⬇ 232503 · by joelchance
Self-Improving + Proactive Agent
Self-reflection + Self-criticism + Self-learning + Self-organizing memory. Agent evaluates its own work, catches mistakes, and improves permanently. Use when...
⬇ 200423 · by ivangdavila
Github
Interact with GitHub using the `gh` CLI. Use `gh issue`, `gh pr`, `gh run`, and `gh api` for issues, PRs, CI runs, and advanced queries.
⬇ 191115 · by steipete
ontology
Typed knowledge graph for structured agent memory and composable skills. Use when creating/querying entities (Person, Project, Task, Event, Document), linkin...
⬇ 190156 · by oswalpalash

💬 留言讨论