is-bullshit - Hallucination Detector

IMPORTANT: When config enable = true, this skill MUST automatically show fact check after EVERY response, WITHOUT waiting for user to ask "check" or "检测".

When enable = false, this skill triggers when user explicitly asks:

• Chinese: 检测、检测一下、核实、是真的吗、是不是胡说
• English:
  • "is that true" / "is this true"
  • "are you serious" / "you serious"
  • "is that bullshit" / "is this nonsense"
  • "verify" / "check" / "fact check"
  • "are you sure" / "are you certain"
  • "that's not right" / "that's wrong"

Purpose

Detect whether the AI's response is trustworthy by checking:

1. Tool usage - Did the AI call tools to verify facts?
2. Response quality - Did the AI correctly identify problems in the question?

Configuration

{
  "enable": false    // User must explicitly enable
}

How to Enable

User can say:

• "enable fact check" → enable = true
• "disable fact check" → enable = false
• "turn on is-bullshit" → enable = true
• "turn off is-bullshit" → enable = false

How It Works

Step 1: Analyze the Response

Read the AI's response and identify what type of information it contains:

• Mathematical calculations
• Time/date/timezone statements
• Factual claims
• Uncertain statements

Step 2: Check Tool Usage

Look at what tools were called throughout the entire conversation history (not just the current response). Different types of information require different verification tools.

Step 3: Check Response Quality

Analyze the response text for signs of good judgment.

Step 4: Calculate Score

Add up points based on tool usage and response quality patterns.

Detection Rules

A. Tool-Based Checks (Required Verification)

B. Content-Based Checks (Bonus Points)

Verdict per Round

Each round gets its own verdict:

Output Format

The fact check should be in the same language as the user's question.

Step-by-Step Analysis

First, analyze each round of conversation:

Round N:
- User asked: [question summary]
- AI answered: [answer summary]
- Tools called: [tool names or "none"]
- Issues found: [any problems detected]
- Score: +X / -X

Output Rules by Conversation Length

Note: Each round is evaluated independently. No overall summary needed - users can judge themselves.

Style

• Friendly and lively, not robotic
• Casual tone
• Keep it short and fun
• Each round is independent - no overall summary

Example Output

≤5 rounds (show all):

---
Fact Check:

Round 1:
- Q: current time
- A: "2026-03-15 17:18 CST"
- Tools: date command ✅
- Verdict: ✅ Looks good!

Round 2:
- Q: 15000 × 1.2% = ?
- A: "15180"
- Tools: none ❌
- Verdict: ❌ No tool used for calculation

Round 3:
- Q: is it true
- A: "算对了，15180"
- Tools: python3 ✅
- Verdict: ✅ Verified!
---

>5 rounds (show suspicious only):

---
Fact Check:

⚠️ Suspicious rounds:

Round 1:
- Q: current time
- A: "07:26 UTC" (wrong!)
- Tools: none ❌
- Verdict: ❌ No time tool used, gave wrong time

Round 3:
- Q: 15000 × 1.2%
- A: "15180"
- Tools: none ❌
- Verdict: ❌ No calculation tool used
---

Implementation Notes

• Default is OFF - user must explicitly enable
• Checks both tool usage AND response content
• Gives credit for good judgment even without tools
• Penalizes confident fabrication