← Back to Skills Marketplace
42
Downloads
0
Stars
0
Active Installs
1
Versions
Install in OpenClaw
/install firecrawl-crawl
Description
Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-...
README (SKILL.md)
firecrawl crawl
Bulk extract content from a website. Crawls pages following links up to a depth/limit.
When to use
- You need content from many pages on a site (e.g., all
/docs/) - You want to extract an entire site section
- Step 4 in the workflow escalation pattern: search → scrape → map → crawl → interact
Quick start
# Crawl a docs section
firecrawl crawl "\x3Curl>" --include-paths /docs --limit 50 --wait -o .firecrawl/crawl.json
# Full crawl with depth limit
firecrawl crawl "\x3Curl>" --max-depth 3 --wait --progress -o .firecrawl/crawl.json
# Check status of a running crawl
firecrawl crawl \x3Cjob-id>
Options
| Option | Description |
|---|---|
--wait |
Wait for crawl to complete before returning |
--progress |
Show progress while waiting |
--limit \x3Cn> |
Max pages to crawl |
--max-depth \x3Cn> |
Max link depth to follow |
--include-paths \x3Cpaths> |
Only crawl URLs matching these paths |
--exclude-paths \x3Cpaths> |
Skip URLs matching these paths |
--delay \x3Cms> |
Delay between requests |
--max-concurrency \x3Cn> |
Max parallel crawl workers |
--pretty |
Pretty print JSON output |
-o, --output \x3Cpath> |
Output file path |
Tips
- Always use
--waitwhen you need the results immediately. Without it, crawl returns a job ID for async polling. - Use
--include-pathsto scope the crawl — don't crawl an entire site when you only need one section. - Crawl consumes credits per page. Check
firecrawl credit-usagebefore large crawls.
See also
- firecrawl-scrape — scrape individual pages
- firecrawl-map — discover URLs before deciding to crawl
- firecrawl-download — download site to local files (uses map + scrape)
Usage Guidance
Install only if you want an agent to crawl and extract website content. When using it, specify the exact domain/path, page limits, and concurrency or budget limits to avoid unnecessary crawling or credit use.
Capability Assessment
Purpose & Capability
The described crawling and bulk extraction behavior fits a website/documentation extraction purpose.
Instruction Scope
The trigger language appears broad enough that a user or agent could invoke it for larger crawls than intended, so requests should be explicit about domain, path, and page limits.
Install Mechanism
No risky installer, obfuscated setup, or package-level abuse signal was provided; VirusTotal telemetry was clean.
Credentials
Network crawling and concurrent extraction can consume API credits and collect more public web content than expected, but that is proportionate when the user asks for bulk site extraction.
Persistence & Privilege
No evidence was provided of persistence, privilege escalation, destructive local changes, or local credential/session harvesting.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install firecrawl-crawl - After installation, invoke the skill by name or use
/firecrawl-crawl - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
- Initial release of firecrawl-crawl: bulk extract content from entire websites or specific site sections.
- Supports crawling pages up to a specified depth, with options for path filtering and concurrency.
- Includes options for limiting pages, setting delays, and exporting results to JSON.
- Provides guidance on when and how to use the skill, including integration with other firecrawl tools.
Metadata
Frequently Asked Questions
What is Firecrawl Crawl?
Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-... It is an AI Agent Skill for Claude Code / OpenClaw, with 42 downloads so far.
How do I install Firecrawl Crawl?
Run "/install firecrawl-crawl" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is Firecrawl Crawl free?
Yes, Firecrawl Crawl is completely free, licensed under MIT-0. You can download, install and use it at no cost.
Which platforms does Firecrawl Crawl support?
Firecrawl Crawl is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created Firecrawl Crawl?
It is built and maintained by eohmig (@eohmig); the current version is v1.0.0.
More Skills