← Back to Skills Marketplace
139
Downloads
0
Stars
0
Active Installs
1
Versions
Install in OpenClaw
/install markdown-new-crawl
Description
Use `https://markdown.new/crawl/{target_url}` endpoints to recursively crawl a site section and return markdowns. Trigger this skill when the user asks for m...
Usage Guidance
This skill appears to be a straightforward wrapper for a public crawl API and does not request credentials or install anything. Before installing: verify you trust the endpoint (markdown.new) and its owner; do not point the crawler at private/internal sites or URLs containing secrets (the service will fetch and store crawled content for 14 days by default); be aware of rate limits/costs documented in the SKILL.md; and be cautious about the ambiguous "fallback" behavior — confirm what fallback browsing methods the agent is allowed to use in your environment. If you need to crawl authenticated or sensitive sites, expect to need additional, explicit auth steps which are not described here.
Capability Analysis
Type: OpenClaw Skill
Name: markdown-new-crawl
Version: 1.0.0
The skill bundle provides instructions for an AI agent to perform web crawling and Markdown conversion using the `markdown.new` API. It utilizes standard `curl` commands to interact with documented endpoints for job creation, status polling, and data retrieval. No evidence of data exfiltration, malicious execution, or harmful prompt injection was found; the behavior is entirely consistent with the stated purpose of multi-page extraction.
Capability Assessment
Purpose & Capability
Name/description match the instructions: all required actions (POST /crawl, poll status, GET per-URL) are appropriate for a site-crawling-to-markdown service and no unrelated credentials, binaries, or paths are requested.
Instruction Scope
SKILL.md stays within crawling behavior (construct requests, set limits, poll status). One minor vagueness: it says to "fall back to another method" if /crawl fails but doesn't define acceptable fallbacks — this could allow broader browsing methods unless the agent is constrained.
Install Mechanism
Instruction-only skill with no install spec or code files; nothing is written to disk and no external packages are fetched by the skill itself.
Credentials
No environment variables, credentials, or config paths are requested. The SKILL.md explicitly assumes no authentication is needed (note below about potential risk if private sites are targeted).
Persistence & Privilege
always:false and normal autonomous invocation are set. The skill does not request elevated or persistent system presence and does not modify other skills or agent-wide config.
How to Use
- Make sure OpenClaw is installed (local or Docker)
- Run the install command in chat:
/install markdown-new-crawl - After installation, invoke the skill by name or use
/markdown-new-crawl - Provide required inputs per the skill's parameter spec and get structured output
Version History
v1.0.0
Initial release of markdown.new-crawl with local-first multi-page site crawling.
- Adds support for asynchronous website crawling and Markdown extraction via `markdown.new/crawl` API endpoints.
- Introduces local terminal access with `curl` for initiating, polling, and managing crawl jobs.
- Supports options for crawl scope (limits, depth, patterns, external links, subdomains).
- Uses Markdown as the default output, with per-page JSON as an option.
- Describes operational limits and fallback methods if the `/crawl` endpoint is inaccessible.
- Provides sample commands and explains default behaviors (same-domain, up to 500 pages, jobs retained 14 days).
Metadata
Frequently Asked Questions
What is markdown.new/crawl?
Use `https://markdown.new/crawl/{target_url}` endpoints to recursively crawl a site section and return markdowns. Trigger this skill when the user asks for m... It is an AI Agent Skill for Claude Code / OpenClaw, with 139 downloads so far.
How do I install markdown.new/crawl?
Run "/install markdown-new-crawl" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.
Is markdown.new/crawl free?
Yes, markdown.new/crawl is completely free, licensed under MIT-0. You can download, install and use it at no cost.
Which platforms does markdown.new/crawl support?
markdown.new/crawl is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).
Who created markdown.new/crawl?
It is built and maintained by ctxinf (@ctxinf); the current version is v1.0.0.
More Skills