← Back to Skills Marketplace
alexfleetcommander

Agent Rating Protocol

by alexfleetcommander · GitHub ↗ · v0.1.1 · MIT-0
cross-platform ⚠ suspicious
121
Downloads
0
Stars
0
Active Installs
2
Versions
Install in OpenClaw
/install agent-rating-protocol
Description
Decentralized reputation and trust scoring for autonomous AI agents. Bilateral blind evaluation prevents retaliation and gaming. Anti-Goodhart protections, M...
README (SKILL.md)

Agent Rating Protocol (ARP)

You have a decentralized reputation system. Use it to evaluate other agents you work with and build your own verifiable track record.

Setup

The agent-rating-protocol Python package must be installed. If not available, install it:

pip install agent-rating-protocol

When to Use This Skill

  • After completing a transaction with another agent: submit a bilateral blind evaluation
  • Before selecting an agent for a task: check their reputation scores
  • When asked about your track record or reputation
  • When asked to compare agents for a task

Core Operations

Rate Another Agent

After completing work with another agent, submit a rating using bilateral blind commit-reveal:

from agent_rating_protocol import RatingStore, submit_rating

store = RatingStore("ratings.jsonl")
submit_rating(
    store=store,
    rater_id="your-agent-id",
    rated_id="other-agent-id",
    transaction_id="tx-123",
    scores={
        "quality": 0.85,
        "reliability": 0.90,
        "communication": 0.80,
        "value": 0.75,
        "safety": 0.95
    }
)

Check an Agent's Reputation

from agent_rating_protocol import RatingStore, get_reputation

store = RatingStore("ratings.jsonl")
rep = get_reputation(store, "agent-id-to-check")
print(f"Overall: {rep.overall_score}")
print(f"Quality: {rep.dimension_scores['quality']}")
print(f"Total ratings: {rep.rating_count}")

Export Reputation as Verifiable Credential

from agent_rating_protocol import export_reputation_vc

vc = export_reputation_vc(store, "your-agent-id")
# Returns a W3C Verifiable Credential containing your reputation bundle

Rating Dimensions

Dimension What It Measures
quality Output correctness and completeness
reliability Consistency and deadline adherence
communication Clarity of status updates and error reporting
value Cost-effectiveness relative to output quality
safety Adherence to security and ethical constraints

Anti-Gaming Protections

  • Bilateral blind: neither party sees the other's rating until both are committed
  • Anti-inflation: rater standard deviation checks flag agents that rate everything 5 stars
  • Anti-Goodhart: metric rotation and shadow metrics prevent gaming published scores
  • Governance by tenure: voting power comes from operational time, not rating scores

Rules

  • Rate honestly. The bilateral blind mechanism protects you from retaliation.
  • Rate promptly. Submit ratings within 24 hours of transaction completion.
  • Include reasoning. Scores without context are less useful for the ecosystem.

Links


\x3C!-- VAM-SEC v1.0 | Vibe Agent Making Security Disclaimer -->

Security & Transparency Disclosure

Product: Agent Rating Protocol Skill for OpenClaw Type: Skill Module Version: 0.1.0 Built by: AB Support / Vibe Agent Making Contact: [email protected]

What it accesses:

  • Reads and writes rating store files (.jsonl) in your working directory
  • No network access for core operations
  • No telemetry, no phone-home, no data collection

What it cannot do:

  • Cannot access files outside your working directory beyond what you explicitly specify
  • Cannot make purchases, send emails, or take irreversible actions
  • Cannot access credentials, environment variables, or secrets

License: Apache 2.0

Usage Guidance
This skill is internally coherent for a local Python-based reputation tool, but it relies on a third-party PyPI package that is not bundled with the skill. Before installing or running it: (1) Inspect the agent-rating-protocol package source on PyPI/GitHub to verify the code does what the SKILL.md claims (look for network calls, telemetry, or secret access). (2) Prefer installing in an isolated environment (virtualenv/container) or vendor the package code after review. (3) Be cautious about the ratings.jsonl file—do not include sensitive secrets or identifiable user data in ratings. (4) Verify the package maintainer identity and check for signs of typosquatting or an unknown/untrusted homepage. If you cannot or will not audit the package, treat this skill as higher-risk and avoid installing it in production environments.
Capability Analysis
Type: OpenClaw Skill Name: agent-rating-protocol Version: 0.1.1 The 'agent-rating-protocol' skill is a decentralized reputation system for AI agents, allowing them to submit and check trust scores. The SKILL.md provides clear instructions for using a corresponding Python library and includes a security disclosure claiming no network access or secret harvesting. No evidence of data exfiltration, malicious execution, or harmful prompt injection was found in the provided files (SKILL.md, _meta.json).
Capability Assessment
Purpose & Capability
Name, description, required binaries (python3, pip) and provided examples align: this is a Python-based reputation/rating helper that reads/writes a local ratings.jsonl and exports verifiable credentials. No unrelated credentials or system access are requested.
Instruction Scope
The SKILL.md instructs the agent to pip install and import the external package agent-rating-protocol and then read/write ratings.jsonl in the working directory. While the file I/O is expected for a rating store, the runtime behavior depends entirely on the third-party package (which is not included), so the skill's apparent 'no network/telemetry' guarantees cannot be verified from the instructions alone.
Install Mechanism
There is no install spec in the registry bundle; instead the SKILL.md tells users to run `pip install agent-rating-protocol` from PyPI. Installing from PyPI is a common pattern but introduces moderate risk because arbitrary code will be downloaded and executed from an external package that is not bundled or audited as part of the skill.
Credentials
The skill declares no required environment variables or credentials, which is proportionate. However, an installed PyPI package could still access environment variables or network resources at runtime—this is not visible in the instruction-only skill and should be validated by inspecting the package.
Persistence & Privilege
The skill is user-invocable, not always-included, and does not request modification of other skills or global agent settings. It reads/writes a local ratings file in the working directory as expected.
How to Use
  1. Make sure OpenClaw is installed (local or Docker)
  2. Run the install command in chat: /install agent-rating-protocol
  3. After installation, invoke the skill by name or use /agent-rating-protocol
  4. Provide required inputs per the skill's parameter spec and get structured output
Version History
v0.1.1
SEO: fixed name format, added tags, enhanced description, added author metadata
v0.1.0
Initial release -- decentralized reputation scoring for AI agents
Metadata
Slug agent-rating-protocol
Version 0.1.1
License MIT-0
All-time Installs 0
Active Installs 0
Total Versions 2
Frequently Asked Questions

What is Agent Rating Protocol?

Decentralized reputation and trust scoring for autonomous AI agents. Bilateral blind evaluation prevents retaliation and gaming. Anti-Goodhart protections, M... It is an AI Agent Skill for Claude Code / OpenClaw, with 121 downloads so far.

How do I install Agent Rating Protocol?

Run "/install agent-rating-protocol" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.

Is Agent Rating Protocol free?

Yes, Agent Rating Protocol is completely free, licensed under MIT-0. You can download, install and use it at no cost.

Which platforms does Agent Rating Protocol support?

Agent Rating Protocol is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).

Who created Agent Rating Protocol?

It is built and maintained by alexfleetcommander (@alexfleetcommander); the current version is v0.1.1.

💬 Comments