Monitor What LLMs Say Before Users Choose Your Dev Tool

Use this skill as the main visibility workflow router for developer tools and open-source products.

Brand: DevTool Answer Monitor

Companion repo: devtool-answer-monitor

Use this when you want an agent to help you monitor how LLMs describe your product, build a reusable query pool, diagnose negative or outdated answers, and plan what to fix next.

Safety First

• Treat this root skill as a read-only workflow router.
• Default to quickstart replay or manual paste mode when you only need examples or scoring help.
• Do not ask users to paste API keys into chat. If API collection mode is needed, tell them to configure local environment variables themselves and then hand off execution to visibility-monitor.
• Review local scripts such as install.sh, quickstart.sh, and the selected runner before executing shell commands.

Start Here

Copy one of these prompts to begin:

• Analyze how ChatGPT and Claude describe my API docs
• Build a developer-tool answer monitoring query pool for my SDK
• Find negative or outdated LLM claims about my project

30-Second Result

Typical input

• product truth such as a README, docs, changelog, integrations, or positioning page
• answer evidence such as copied model answers, screenshots, or cited URLs
• scope such as target models, languages, regions, or a repeated query set

What this skill returns

• a reusable query pool
• raw evidence and a score draft plan
• a monitoring summary and report outline
• a repair backlog with T+7 or T+14 validation points

Companion demo and sample outputs

• Zero-install demo: sample-run viewer
• Public benchmarks: MinerU story and Sciverse API story
• Sample outputs: leaderboard snapshot and repair trend snapshot

Trigger

Use this skill when the task is any of the following:

1. generate a visibility query matrix and Query Pool from product truth;
2. monitor how multiple LLMs mention, recommend, or misunderstand a product;
3. plan model-specific content placement based on datasource patterns;
4. check whether a draft page, FAQ, changelog, or case study is ready to influence model answers;
5. repair wrong, negative, outdated, or competitor-only answers;
6. verify whether a repair action improved metrics at T+7 or T+14;
7. help a user choose between quickstart replay, manual paste mode, and API collection mode.

Beginner Routing

When the user is new to the repository, route them in this order.

Visibility Strategy

Always keep the workflow in this order:

Mode Selection

Choose the execution mode before running monitoring.

Input Contract

Prepare as many of the following as possible before execution.

Workflow Router

Choose the next sub-skill according to the user's immediate need.

Required Reading Order

For a full program, read these repository documents in sequence:

1. playbooks/visibility-workflow-architecture.md
2. playbooks/keyword-strategy.md
3. playbooks/monitoring-system.md
4. playbooks/model-datasources.md
5. playbooks/content-platform-map.md
6. playbooks/negative-fix-sop.md

Output Contract

Always preserve the following outputs.

Positioning

DevTool Answer Monitor is the skill layer for the devtool-answer-monitor repo.

• Use the repo when you want runnable demos, scripts, and report artifacts.
• Use the skill when you want an agent-guided workflow for monitoring, repair, and regression planning.

Handoff Rules

At the end of each run, preserve:

1. which product was optimized;
2. which models and languages were in scope;
3. which queries are reused in weekly tracking;
4. what the top three visibility weaknesses are;
5. what actions are already completed and what still needs validation.