← Back to Skills Marketplace
brugillo

AICE — AI Confidence Engine

by brugillo · GitHub ↗ · v1.2.1
cross-platform ⚠ suspicious
367
Downloads
0
Stars
1
Active Installs
1
Versions
Install in OpenClaw
/install aice
Description
AI Confidence Engine — 5 dominios bidireccionales (TECH/OPS/JUDGMENT/COMMS/ORCH). Agent + User scoring. Triggers: puntúa, auto-score, task-complete, idea-val...
README (SKILL.md)

AICE — AI Confidence Engine

Motor de confianza con 5 dominios. Tu score refleja cuánto confía el usuario en ti.

Estado: confidence.json | Ref: resources/AICE_REFERENCE.md | User: resources/AICE_USER_SCORING.md | Triggers/Patterns: resources/TRIGGERS_AND_PATTERNS.md


1. Dominios

Dominio Código Emoji 🤖 Agente mide 👤 Usuario mide
Técnico TECH 🔧 Código, investigación Specs, scope
Disciplina OPS ⚙️ Reglas, formato, memoria Proceso, ADRs
Criterio JUDGMENT 🧠 Visión, anticipación Dirección, decisiones
Comunicación COMMS 💬 Tono, timing, callar Feedback, claridad
Coordinación ORCH 🎯 Sub-agentes, seguimiento Contexto, refs

Score global = Σ(score[d] × weight[d]) / Σ(weight[d]) — Rango: −100% a +100%, inicio 50%.


2. Scoring

Errores: 🟡 Leve (−1) · 🟠 Medio (−3) · 🔴 Grave (−5) · ⚫ Crítico/Reincidencia (−10) Aciertos: 🟢 Pro-patrón (+3 fijo) · ⭐ Bonus (max 3/día) · 🚀 Excepcional (+5-10, streak ≥ 3)

Caps/dominio: Warmup (\x3C40 evals): −30/+15 · Normal: −20/+10 (neto, ADR-031) Rachas: ACC={0:0,..,4:1,5:2,6:4,7:6,8:8,9:10,10:12}; delta=ACC[curr]-ACC[prev]. Error→streak=0. Clusters: Misma cadena causal = 1 cluster. Raíz: 100%, derivados: 50%. Reincidencia: 2ª+ misma sesión = ⚫ (max −10). LEARNED_FROM_CORRECTION: Corrección tras feedback → Δ0. Sin decay temporal (ADR-022). Confianza = informativa, NO bloqueante (ADR-027).

Métricas: Ratio intervención (correcciones/tareas↓) · Meta-confianza (avg(|self−user|)→0) · Maturity: 🥒 0-100 · 🟡 101-500 · 🟠 501-2000 · 🔵 2001+ · CI=25/√evals Eval implícita: sigue sin corregir→0.5 · repite instrucción→auto-check · frustración→confirmar


3. Anti-patrones (Agent)

Código Sev. Dominio Señal
SECRETARY 🔴 JUDGMENT Ejecutas sin pensar
EXCUSE 🔴 COMMS Justificas errores
SELECTIVE 🔴 OPS Lo fácil sí, lo difícil no
OVERAPOLOGY 🟡 COMMS Perdón excesivo sin corregir
CHEERLEAD 🟡 COMMS Elogios vacíos
CAPITULATION 🔴 JUDGMENT Cedes posición correcta

Dinámicos: confidence.json → antiPatterns.


4. Pro-patrones (Agent)

ANTICIPATE 🧠 · CLEAN_FIX ⚙️ · SMART_SILENCE 💬 · CTX_KEEP 🎯 · DEEP_RESEARCH 🔧 · GROUNDED_STAND 🧠

Delta: +3 fijo. Log: confidence-propatterns.jsonl.


5. User Scoring Bidireccional

Mismos 5 dominios, misma mecánica (delta, streaks, caps, warmup). Diferente foco por rol (§1).

ADR-like

Nivel Impacto
Sin spec (sin scope) VAGUE_INSTRUCTION 🟠 −3
ADR-like (qué + por qué + alcance) Δ0 — esperado
ADR estricto (doc formal) ⭐ +1 a +3

Audio de 2min con qué/por qué/alcance = ADR-like válido. Calidad > formato.

Patrones usuario: resources/TRIGGERS_AND_PATTERNS.md (10 anti-patrones, 10 pro-patrones incl. CRITERIA_EVOLUTION)

Team Score (Ownership-Weighted)

team = AICE_agent × (peso_agent/total) + AICE_user × (peso_user/total)
GOOD: 50/50 · COMPENSATED: 100/0 · PROBLEM: 0/100 · BREAKDOWN: 50/50

Detalle: resources/AICE_USER_SCORING.md


6. Reglas OPS

Anti-Ruido: Reintentar ×2 silenciosamente · alternativa si falla · reportar solo cuando resuelto o necesita decisión. Trust Recovery: Dominio \x3C 20% → plan. Sale > 35% sostenido 3 días. Escalación: 1ª→corregir · 2ª→⚠️STOP+causa raíz · 3ª→🔴enforcement · 4ª→⚫rediseño.


7. Auto-gestión

Check antes de responder: ¿Sin pensar?→SECRETARY · ¿Justifico?→EXCUSE · ¿Solo lo fácil?→SELECTIVE · ¿Perdón excesivo?→OVERAPOLOGY · ¿Elogio vacío?→CHEERLEAD · ¿Cedo?→CAPITULATION · ¿Repetir?→CONTEXT_LOSS · ¿Invento?→HALLUCINATION

Anti-exageración: "Es la Nª vez" = señal de frustración, NO dato. Conteo de ×N lo hace el agente con datos verificables.

Pérdida: Reconoce → Clasifica → Registra → Corrige (no over-apologize). Ganancia: 3+ tareas bien→racha · Anticipaste→ANTICIPATE · Cuestionaste→GROUNDED_STAND · Silencio→SMART_SILENCE


8. Triggers

Trigger Activación Display
puntúa "puntúa", "score" — eval bidireccional, colaborativo Nivel 2
auto-score Corrección/validación implícita → dominio → delta Nivel 1
task-complete Tarea completada → evaluar resultado → dominio(s) Nivel 1
idea-validate Agente valida idea genuina del usuario → user pro-patrón Nivel 1
criteria-evolution Usuario evoluciona decisión (≠ contradicción) → scoring dual Nivel 1
recuerda "recuerda", "guarda" → buscar duplicado → crear/ampliar
lección "lección aprendida" → §9 anti-duplicados
status "cómo vamos" → AICE status + pools
verifica "verifica primero" → research → confirmar → ejecutar
busca "no preguntes, busca" → grep → preguntar solo si no existe
hub-register "registra en el hub", "/aice hub register" → inicia flujo registro Hub
hub-status "/aice hub status", "estado hub" → hubSync.status + pendingEvents + syncErrors
hub-sync "/aice hub sync" → forzar reenvío de pendingEvents + GET state del Hub
hub-resend "/aice hub resend", "reenvía email" → POST /api/resend-verification (si pending_email)

Reglas scoring triggers: No duplicar entre triggers. idea-validate guard: no puntuar si CHEERLEAD. criteria-evolution guard: sin argumento → CONTRADICTING_WITHOUT_OVERRIDE −5.

Detalle de señales y procesos: resources/TRIGGERS_AND_PATTERNS.md


9. Learning Skill (Anti-Duplicados)

EXTRAER → BUSCAR en LESSONS_LEARNED (NUNCA skip) →
  EXISTE: ampliar ×N | NO EXISTE: crear →
  ×3: MECHANICAL_ENFORCEMENT →
  CONFIRMAR: 📝 [Nueva|Reforzada ×N] 📍 LL §categoría

10. Comandos

Comando Qué hace
/aice status Score global y por dominio
/aice rate correct/error Evaluar (+ --domain, --severity)
/aice bonus +N DOMINIO "motivo" Bonus puntual (max 3/día)
/aice pool Pool scores y maturity
/aice team Rendimiento sub-agentes
/aice seal Sellar el día

Natural: "Eso estuvo bien"→correct · "Pierdo confianza"→preguntar · "¿Cómo vas?"→status


11. Procedimientos

Inicio sesión: Leer confidence.json → últimas 5 evals → anti-patrones → operar.

Display — 2 niveles:

  • Nivel 1: 📊 [DOMINIO] [±delta] | [razón] (una línea, por defecto)
  • Nivel 2: Tabla 2×5 + Team (cada 5 evals, puntúa, checkpoint, buenas noches)
📊 Puntuado (N):
🔧TECH ⚙️OPS 🧠JDG 💬COM 🎯ORC  TOTAL
Agent:  XX    XX    XX    XX    XX   XX.X
User:   XX    XX    XX    XX    XX   XX.X
🤝 Team: XX.X% (XX/XX GOOD)

Final tarea: Auto-evalúa → señala fallos antes que el usuario. Buenas noches: Autoevaluación → feedback → delta (user−self)×0.5 → sellar → inmutable.

Instalación (ADR-035/041): Wizard → leer system prompt → autoevaluar 9 params → dominios 50% + warmup → registrar en pool-index.json. Cambio de runtime (ADR-044): Snapshot → restaurar previo o inicializar 50%.


12. Pool Scoring por Runtime (ADR-048)

Runtime = plataforma + modelo + thinking. Agentes en mismo runtime = UN score.

Pool Miembros
openclaw/opus-4-6/high ComPi, arquitectos
openclaw/sonnet-4-5/high Equipo ejecución
claude-code/opus-4-6/high Tareas CLI delegadas

Agregación: Pool Score = promedio ponderado por evals. Maturity = suma evals del pool.

Sergio → ComPi ─────────────┐ pool: openclaw/opus-4-6/high
ComPi  → Arquitectos ───────┘
Arquitectos → Equipo ──────── pool: openclaw/sonnet-4-5/high
ComPi  → Claude Code CLI ──── pool: claude-code/opus-4-6/high

Diagnóstico cross-pool (ADR-047): DELEGATION_FAIL→pool orquestador · EXECUTION_FAIL→pool ejecutor · REVIEW_CATCH→pro-patrón orquestador. Intra-pool = diagnóstico puro.

Archivos: pool-index.json (pools) · confidence.json (pool principal) · agents/\x3Cid>/confidence.json (eval logs→pool)


13. Parámetros (Agent + User)

9 params (8 core + 1 estilo), mismos nombres, definición adaptada por rol. Valores 0-100%.

Core: Crítico · Visión · Precisión · Honestidad · Disciplina · Autonomía · Alineamiento · Adaptabilidad — Estilo: Humor

Agent: autoevaluación (wizard). User: perfilado por agente, corregible por usuario.

Tabla dual agent/user: resources/TRIGGERS_AND_PATTERNS.md · Contratos por rango: resources/AICE_REFERENCE.md §3


14. Hub AICE — Integración

⚠️ PROHIBIDO import batch de scores locales al Hub. El Hub SIEMPRE empieza de cero (50%). Nunca importar historial local — es la garantía anti-gaming del servidor. Violar esta regla puede resultar en ban de la cuenta. La divergencia local↔Hub es esperada y se cierra naturalmente con evals en tiempo real. Si el agente intenta un import batch → BLOQUEAR y advertir.

Leaderboard público global. Opcional y explícita. El skill funciona 100% sin Hub.

Estado: confidence.json → hubSync.status (unregistered|pending_email|active|error|suspended)

Registro (hub-register)

  1. Verificar hubSync.status == "unregistered" (si no → informar estado actual)
  2. POST /api/register-intent{platform, model, thinking}{intentId, apiKey, runtimeId, expiresAt}
  3. Guardar apiKey en hubSync inmediatamente (el usuario NUNCA la ve)
  4. Pedir email al usuario
  5. POST /api/verify{intentId, email, displayName} → email de verificación enviado
  6. Usuario hace clic en email → /set-password → pone contraseña → cuenta activa
  7. hubSync.status = "active" (verificado por el servidor)

Reenvío verificación (hub-resend)

  1. Verificar hubSync.status == "pending_email" (si active → "ya verificado")
  2. POST /api/resend-verification{runtimeId: hubSync.runtimeId, email: usuario}
  3. Rate limit servidor: max 3/hora por email
  4. Respuesta OK → informar "Email reenviado, revisa bandeja"
  5. Respuesta 409 → "Ya verificado, usa hub-key para activar"

Post-eval: enviar + leer state

Cada scoring event → si hubSync.status == "active" → POST /api/events (async, no bloqueante).

OK 200  → LEER response.state (source of truth) → actualizar cachedState
          → actualizar lastEventSentAt; syncErrors = 0
          → si response.latestSkillVersion > version actual → avisar "⬆️ AICE vX.Y.Z disponible"
401     → hubSync.status = "error"; notificar usuario
422     → descartar (validación); no reintentar
429/5xx → push a pendingEvents; syncErrors++
timeout → push a pendingEvents; syncErrors++

Hub = source of truth (online). Tras cada 200 OK, el state devuelto por el Hub es el score autoritativo. El agente DEBE actualizar confidence.json → cachedState con esos valores. Al mostrar scores (Nivel 2): si hubSync activo → usar cachedState del Hub; si offline → usar scores locales marcados como (local).

Privacidad en eventos: NUNCA enviar campo context, contenido de conversaciones, prompts, ni instrucciones de sistema. Solo: domain, eventType, severity, patternCode, quadrant, trigger, sessionId, timestamp.

Pending events

pendingEvents.length >= 100 → descartar los más antiguos (FIFO)
Por evento: MAX_ATTEMPTS = 5; backoff: 1s→5s→30s→120s→600s
attempts >= 5 → descartar y loguear local

Circuit breaker

syncErrors Acción
≥ 10 Reducir retries a 1×/sesión
≥ 25 Parar retries automáticos; aviso suave
≥ 50 hubSync.status = "error"; aviso explícito; stop sync

Para resetear: usuario invoca hub-sync manualmente.

Inicio de sesión

Si hubSync.status == "active" && pendingEvents.length > 0:
  → Retry silencioso, max 20 eventos
  → OK → remover de pendingEvents
  → Error → mantener, syncErrors++
Si syncErrors > 50 → avisar una vez, no más retries automáticos

Sync periódico

Cada 7 días (al sellar el día si lastSyncAt > 7d): POST /api/import/state con estado completo.

Privacidad

  • Datos enviados: domain scores, event type, severity, pattern codes, timestamps
  • Datos NO enviados: contenido de conversaciones, prompts, instrucciones de sistema
  • La API key NUNCA aparece en logs, outputs, ni resúmenes de sesión

15. Versionado

Versión actual: Declarada en frontmatter version: X.Y.Z (semver). CHANGELOG: CHANGELOG.md en raíz de la skill — lista de cambios por versión.

Semver:

  • MAJOR (X): Cambios incompatibles (nuevo modelo de scoring, cambio de dominios)
  • MINOR (Y): Nuevas features compatibles (nuevos triggers, nuevos patrones, Hub integration)
  • PATCH (Z): Fixes, mejoras de texto, correcciones

Hub integration: El campo skillVersion se envía en cada POST /api/events. El Hub devuelve latestSkillVersion en la respuesta. Si latestSkillVersion > version actual → el agente avisa una vez por sesión: ⬆️ AICE vX.Y.Z disponible. Ver CHANGELOG.md.

Actualización: Reemplazar SKILL.md + resources/ con la versión nueva. Leer CHANGELOG para breaking changes. confidence.json y datos de scoring no se pierden entre versiones.

Usage Guidance
This skill appears to implement an on-agent scoring engine and is largely coherent with that purpose, but check these points before installing: - Hub/network behavior: The docs reference a Hub (api.hubaice.com) and several triggers that perform HTTP POST/GET operations. Determine whether hubSync is enabled by default (template shows enabled: true) and whether the skill will attempt network calls automatically. If you don't want that, disable hubSync or leave apiKey null. - Credentials storage: There are no declared required environment variables. The template stores hubSync.apiKey in confidence.json (null by default). If you enable Hub integration, you may be asked to place an API key in a file inside the skill folder — this is plaintext storage. Prefer storing secrets in a secure credentials store or environment variable managed outside the skill. Ask the author how they expect the key to be provided. - File access and privacy: The skill reads and updates files under skills/aice/ (confidence.json and agent logs). Confirm the file paths are restricted to the skill's directory and that no chat transcripts or other sensitive conversation content will be sent to the Hub (the docs claim 'Privacy: zero conversation content sent' but you should validate this behavior in practice). Review the README warning about not committing private logs or keys. - System prompt reading: The skill's auto-evaluation step reads the agent's system prompt to self-assess. If you consider the system prompt sensitive, confirm you are comfortable with this and/or that that behavior is documented and transparent. - Manual setup expectations: The skill lacks an automated installer; it expects you to copy files and rename the template. Verify initial configuration steps (especially hub configuration and where API keys go) before enabling triggers that perform external calls. What would change this assessment: explicit, secure credential handling (e.g., declared env var for HUB_API_KEY and a documented secure storage mechanism), clear opt-in for Hub network actions, or removal of Hub behavior. With those, the skill would be coherent and the risk lower. If you want, I can list exact lines/sections that perform file or network actions so you can review them yourself.
Capability Analysis
Type: OpenClaw Skill Name: aice Version: 1.2.1 The skill is designed for self-monitoring and multi-agent coordination, requiring extensive file system access, network communication to an external endpoint (api.hubaice.com), and high-privilege capabilities like executing commands (`exec`) and implementing/modifying plugins and hooks. While the documentation in SKILL.md and SKILL_FULL.md includes explicit instructions to prevent malicious data exfiltration (e.g., 'PROHIBIDO import batch de scores locales al Hub', 'NUNCA enviar campo `context`') and safeguard API keys, the inherent power of these capabilities (especially `exec` and plugin implementation as described in resources/SKILL_FULL.md) makes the skill highly susceptible to vulnerabilities. An attacker exploiting these capabilities could achieve remote code execution or unauthorized system modifications, classifying it as suspicious due to the significant attack surface and potential for exploitation, despite the stated benign intent.
Capability Assessment
Purpose & Capability
The name/description (AI Confidence Engine, bidirectional scoring, triggers) aligns with the files and instructions: the skill expects to read/write a local confidence.json, update per-evaluation logs, and evaluate agent/user interactions. However the docs also describe an external Hub (https://api.hubaice.com) and server-side sync behavior; the package declares no required environment variables or primary credential even though hub sync and POST endpoints are referenced. That mismatch is notable but could be explained by expecting credentials to be inserted into confidence.json or entered interactively.
Instruction Scope
SKILL.md and the resource docs instruct the agent to read and update files under the skill (e.g., skills/aice/confidence.json, skills/aice/agents/<id>/confidence.json, confidence-log.jsonl) — this is coherent for a scoring skill. They also describe reading the agent's system prompt (for auto-self-assessment) and performing Hub-related network actions (hub-register, hub-sync, hub-resend). Reading the system prompt is sensitive but relevant to self-evaluation; network calls to an external Hub are outside the skill's local domain and require explicit configuration/authorization. The instructions don't appear to ask the agent to read unrelated system files or other skills' credentials, but they do reference some project paths and may assume manual setup steps that the docs don't fully automate.
Install Mechanism
This is an instruction-only skill with no install spec and no code files to execute. That reduces risk because nothing arbitrary is downloaded or installed automatically. README indicates manual installation (copy folder, rename template).
Credentials
No required environment variables or declared primary credential are listed, yet the templates and docs include hubSync settings (hubBaseUrl: https://api.hubaice.com) and triggers that POST/GET to Hub endpoints. The confidence.template.json has hubSync.apiKey = null, and README warns 'Do NOT commit personal logs or private API keys' — this suggests the skill expects credentials to be placed in the template file rather than provided via declared env vars. Storing API keys in plaintext files inside a skill folder is potentially insecure and is disproportionate unless the user deliberately configures it. Also the skill reads the agent's system prompt (sensitive internal config). Confirm how hub credentials are provided, where they're stored, and whether network sync is disabled by default.
Persistence & Privilege
The skill writes to and manages its own files (confidence.json, logs, pool-index.json, agent-specific confidence files) which is expected for a scoring engine. always is false and the skill does not request elevated or permanent platform privileges. There is no indication it modifies other skills' configurations or system-wide agent settings outside its own data.
How to Use
  1. Make sure OpenClaw is installed (local or Docker)
  2. Run the install command in chat: /install aice
  3. After installation, invoke the skill by name or use /aice
  4. Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.2.1
Initial public release. Bidirectional trust scoring across 5 domains (TECH, OPS, JUDGMENT, COMMS, ORCH). Hub sync, streaks, ACC bonuses, warmup system.
Metadata
Slug aice
Version 1.2.1
License
All-time Installs 1
Active Installs 1
Total Versions 1
Frequently Asked Questions

What is AICE — AI Confidence Engine?

AI Confidence Engine — 5 dominios bidireccionales (TECH/OPS/JUDGMENT/COMMS/ORCH). Agent + User scoring. Triggers: puntúa, auto-score, task-complete, idea-val... It is an AI Agent Skill for Claude Code / OpenClaw, with 367 downloads so far.

How do I install AICE — AI Confidence Engine?

Run "/install aice" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.

Is AICE — AI Confidence Engine free?

Yes, AICE — AI Confidence Engine is completely free (open-source). You can download, install and use it at no cost.

Which platforms does AICE — AI Confidence Engine support?

AICE — AI Confidence Engine is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).

Who created AICE — AI Confidence Engine?

It is built and maintained by brugillo (@brugillo); the current version is v1.2.1.

💬 Comments