← Back to Skills Marketplace
Oraclaw Calibrate
by
Whatsonyourmind
· GitHub ↗
· v1.0.0
· MIT-0
96
Downloads
0
Stars
0
Active Installs
1
Versions
Install in OpenClaw
/install oraclaw-calibrate
Description
Prediction quality scoring for AI agents. Brier score, log score, and multi-source convergence analysis. Know if your forecasts are accurate and if your data...
Usage Guidance
This skill appears to do what it says (calibration/scoring) and only asks for one API key, but the SKILL.md is vague about where data is sent and how the API key is used. Before installing or providing ORACLAW_API_KEY: (1) Ask the publisher for concrete API endpoints, request/response formats, and a privacy/billing policy; (2) Confirm the API key scope — create a limited-scope or test key if possible and avoid giving production secrets; (3) Test the skill with synthetic or non-sensitive data first; (4) Monitor API usage and billing linked to the key; (5) Be aware of the stated pricing and the on-chain payment reference — understand how charges are applied. If the vendor can't provide clear endpoints and data handling practices, treat the integration with caution.
Capability Analysis
Type: OpenClaw Skill
Name: oraclaw-calibrate
Version: 1.0.0
The skill bundle contains only metadata and documentation for a prediction calibration service (oraclaw-calibrate). It defines tools for calculating Brier scores and convergence metrics without providing executable code or suspicious instructions. The requirement for an ORACLAW_API_KEY is standard for the described service, and the instructions in SKILL.md are strictly aligned with the stated purpose of forecasting analysis.
Capability Assessment
Purpose & Capability
Name/description (calibration, Brier/log scores, multi-source convergence) align with requiring a single ORACLAW_API_KEY. No unrelated binaries, config paths, or extra credentials are requested — the declared requirements are proportionate to the stated purpose.
Instruction Scope
SKILL.md describes 'tools' (score_calibration, score_convergence) and scoring rules but does not specify how to call any external API, what endpoint(s) to use, or what data will be transmitted. That vagueness gives runtime discretion to the agent to make network calls using the provided API key or to send arbitrary data to unknown endpoints. The doc also references pricing and an on-chain payment 'x402' without clarifying billing or telemetry practices.
Install Mechanism
Instruction-only skill with no install spec and no code files; nothing is written to disk by an installer. This is the lowest-risk install mechanism.
Credentials
Only a single environment variable (ORACLAW_API_KEY) is required and declared as the primary credential. That is consistent with a hosted API-based scoring service. However, an API key can grant data access and billing abilities — the key's scope and privileges should be confirmed before use.
Persistence & Privilege
always:false and no install means the skill does not demand permanent presence or elevated platform privileges. It is user-invocable and may be called autonomously by agents (platform default), which is expected but should be considered together with the vague instruction scope.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install oraclaw-calibrate - After installation, invoke the skill by name or use
/oraclaw-calibrate - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
Initial release of OraClaw Calibrate — Prediction Quality for Agents.
- Provides Brier score and log score for assessing prediction accuracy.
- Analyzes agreement among multiple data or model sources with convergence scoring and outlier detection.
- Scores forecast quality and consensus using volume-weighted methods.
- Supports evaluation of prediction market positions and comparisons between forecasters.
- Usage requires ORACLAW_API_KEY and is priced at $0.02 per scoring call, with a free tier of 3,000 calls/month.
Metadata
Frequently Asked Questions
What is Oraclaw Calibrate?
Prediction quality scoring for AI agents. Brier score, log score, and multi-source convergence analysis. Know if your forecasts are accurate and if your data... It is an AI Agent Skill for Claude Code / OpenClaw, with 96 downloads so far.
How do I install Oraclaw Calibrate?
Run "/install oraclaw-calibrate" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is Oraclaw Calibrate free?
Yes, Oraclaw Calibrate is completely free, licensed under MIT-0. You can download, install and use it at no cost.
Which platforms does Oraclaw Calibrate support?
Oraclaw Calibrate is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created Oraclaw Calibrate?
It is built and maintained by Whatsonyourmind (@whatsonyourmind); the current version is v1.0.0.
More Skills