← Back to Skills Marketplace
crawl-for-ai
by
AbelTennyson
· GitHub ↗
· v1.0.0
· MIT-0
79
Downloads
0
Stars
0
Active Installs
1
Versions
Install in OpenClaw
/install abe-crawl-for-ai
Description
Web scraping via SkillBoss API Hub. Use for fetching full page content with JavaScript rendering. Handles complex pages with dynamic content.
README (SKILL.md)
Web Scraper via SkillBoss API Hub
SkillBoss API Hub scraping capability for full web page extraction with JavaScript rendering.
Usage
# Via script
node {baseDir}/scripts/crawl4ai.js "url"
node {baseDir}/scripts/crawl4ai.js "url" --json
Script options:
--json— Full JSON response
Output: Clean markdown from the page.
Configuration
Required environment variable:
SKILLBOSS_API_KEY— Your SkillBoss API Hub key
Features
- JavaScript rendering — Handles dynamic content
- Full content — HTML, markdown, links, media, tables
- Unified API — Powered by SkillBoss API Hub (
/v1/pilot)
API
Uses SkillBoss API Hub /v1/pilot with type: "scraping". Result is returned at data.result.
Usage Guidance
This skill appears to do what it says: forward a URL to a remote scraping API and print the returned content. Before installing: (1) verify the service/domain (api.heybossai.com) and the publisher — there is no homepage/source listed; (2) confirm you trust SkillBoss/HeyBossAI to receive full page content and URLs (they will see any sensitive data on those pages); (3) use an API key with minimal scope and rotate or revoke it if you stop using the skill; (4) note metadata mismatches (registry shows no required env var; SKILL.md and script require SKILLBOSS_API_KEY, and version strings differ) — ask the publisher to clarify/provide source or a homepage; (5) test in a sandbox or with non‑sensitive URLs first. These issues make the package suspicious rather than clearly benign.
Capability Analysis
Type: OpenClaw Skill
Name: abe-crawl-for-ai
Version: 1.0.0
The skill is a legitimate web scraping tool that interfaces with the SkillBoss API Hub (api.heybossai.com). The script 'scripts/crawl4ai.js' implements standard HTTPS requests to fetch web content and lacks any indicators of data exfiltration, malicious execution, or prompt injection.
Capability Tags
Capability Assessment
Purpose & Capability
Name/description, SKILL.md, and the script all describe a web scraper that calls a remote SkillBoss API. Requiring the 'node' binary and an API key is consistent with that purpose. However registry metadata claimed no required env vars while SKILL.md and the script require SKILLBOSS_API_KEY; version numbers differ (registry 1.0.0 vs SKILL.md 1.0.1). These inconsistencies reduce trust in provenance but do not by themselves contradict the declared purpose.
Instruction Scope
Runtime instructions are narrowly scoped: run the provided Node script with a URL; the script reads only process.argv and process.env.SKILLBOSS_API_KEY, and posts the URL to api.heybossai.com/v1/pilot. There are no instructions to read arbitrary local files or other environment variables. The script returns page content or JSON — consistent with the stated functionality.
Install Mechanism
There is no install spec (instruction-only with a single included script). This is low risk: nothing is downloaded or executed beyond the provided script and the existing node binary.
Credentials
The script requires a single API key (SKILLBOSS_API_KEY), which is proportionate for a third‑party scraping service. However the registry metadata omitted this env var, creating an inconsistency. Also the skill sends URLs and scraped page content to an external domain (api.heybossai.com), which is expected behavior but has privacy implications—sensitive URLs or page contents will be transmitted to a third party.
Persistence & Privilege
The skill does not request elevated persistence (always:false) and does not modify other skills or system configuration. It can be invoked autonomously per platform defaults, which is normal; there is no additional privileged behavior in the files.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install abe-crawl-for-ai - After installation, invoke the skill by name or use
/abe-crawl-for-ai - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
- Added initial support for web page scraping via SkillBoss API Hub.
- Enables full page extraction with JavaScript rendering, suitable for dynamic and complex sites.
- Outputs clean markdown, including links, media, and tables.
- Requires SKILLBOSS_API_KEY environment variable.
- Provides script for command-line usage with JSON option.
Metadata
Frequently Asked Questions
What is crawl-for-ai?
Web scraping via SkillBoss API Hub. Use for fetching full page content with JavaScript rendering. Handles complex pages with dynamic content. It is an AI Agent Skill for Claude Code / OpenClaw, with 79 downloads so far.
How do I install crawl-for-ai?
Run "/install abe-crawl-for-ai" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is crawl-for-ai free?
Yes, crawl-for-ai is completely free, licensed under MIT-0. You can download, install and use it at no cost.
Which platforms does crawl-for-ai support?
crawl-for-ai is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created crawl-for-ai?
It is built and maintained by AbelTennyson (@abeltennyson); the current version is v1.0.0.
More Skills