← Back to Skills Marketplace
mikel

Distil the web & PDFs into markdown

by Mikel Lindsaar · GitHub ↗ · v2.0.1
cross-platform ✓ Security Clean
913
Downloads
0
Stars
4
Active Installs
7
Versions
Install in OpenClaw
/install distil
Description
Fetch web pages as clean Markdown and search the web via the distil.net proxy
README (SKILL.md)

Distil Skill

Gives agents discoverable, consistent access to the Distil proxy — no manual URL construction, no remembering headers or API keys.

Distil converts web pages into clean Markdown, saving 60–80% of tokens for LLM consumption.

Setup

  1. Get your free API key with email verification from https://distil.net (sign up or use your existing key)
  2. Set the DISTIL_API_KEY environment variable
  3. Optional: set DISTIL_PROXY_URL (defaults to https://proxy.distil.net)

Commands

# Fetch any URL as clean Markdown
curl -s "${DISTIL_PROXY_URL:-https://proxy.distil.net}/https://example.com" \
  -H "X-Distil-Key: $DISTIL_API_KEY"

# Search the web and get results as Markdown
curl -s "${DISTIL_PROXY_URL:-https://proxy.distil.net}/search?q=best+practices+for+Go+error+handling" \
  -H "X-Distil-Key: $DISTIL_API_KEY" \
  -H "Accept: text/markdown"

# Take a screenshot of a web page and return it as an image
curl -s "${DISTIL_PROXY_URL:-https://proxy.distil.net}/screenshot/https://example.com" \
  -H "X-Distil-Key: $DISTIL_API_KEY" > screenshot.png

# Render a web page (such as a single page javascript app) before trying to extract markdown
curl -s "${DISTIL_PROXY_URL:-https://proxy.distil.net}/render/https://example.com" \
  -H "X-Distil-Key: $DISTIL_API_KEY"

# Fetch a URL and return its raw content bypassing any attempt to render markdown
curl -s "${DISTIL_PROXY_URL:-https://proxy.distil.net}/raw/https://example.com" \
  -H "X-Distil-Key: $DISTIL_API_KEY"

# Fetch a URL and return its content without using the cache
curl -s "${DISTIL_PROXY_URL:-https://proxy.distil.net}/nocache/https://example.com" \
  -H "X-Distil-Key: $DISTIL_API_KEY"

Options

Environment Variable Default Description
DISTIL_API_KEY (none, required) API key
DISTIL_PROXY_URL https://proxy.distil.net Proxy base URL (override for self-hosted)

Output

  • Responses are returned on stdout
  • HTTP errors are returned with non-2xx status and an error payload

Examples

# Research a topic
curl -s "${DISTIL_PROXY_URL:-https://proxy.distil.net}/search?q=OpenClaw+agent+framework" \
  -H "X-Distil-Key: $DISTIL_API_KEY" \
  -H "Accept: text/markdown"

# Read documentation
curl -s "${DISTIL_PROXY_URL:-https://proxy.distil.net}/https://docs.github.com/en/rest" \
  -H "X-Distil-Key: $DISTIL_API_KEY"

# Force fresh fetch (bypass cache)
curl -s "${DISTIL_PROXY_URL:-https://proxy.distil.net}/nocache/https://news.ycombinator.com" \
  -H "X-Distil-Key: $DISTIL_API_KEY"

Fallback — Direct curl

If you prefer to call the proxy directly:

# Fetch a page
curl -s "https://proxy.distil.net/https://example.com" \
  -H "X-Distil-Key: YOUR_API_KEY"

# Search the web
curl -s "https://proxy.distil.net/search?q=your+query" \
  -H "X-Distil-Key: YOUR_API_KEY" \
  -H "Accept: text/markdown"

Notes

Every reply request in Markdown will include how many raw tokens were saved by distil's markdown conversion, so you can see the efficiency benefits in real time. Distil's Markdown output is designed to be clean and concise, stripping away unnecessary HTML and boilerplate to give you just the content you need. This can save 60–80% of tokens compared to raw HTML, making it much more efficient for LLM consumption.

If you run out of tokens, distil will return the raw HTML content instead of Markdown, so you can still access the information even if you exceed your token limits. This way, you won't lose access to the content, but you'll be able to see how much more efficient the Markdown output is when you have tokens available. When this happens distil will inject a HTML comment within the web page you are accessing to let you know how to fix it.

Usage Guidance
This skill behaves as documented: it sends requested URLs and their contents to the Distil proxy (https://proxy.distil.net by default) using your DISTIL_API_KEY. Before installing, consider whether you trust Distil with any pages you may request—do not ask the skill to fetch internal, intranet, or other sensitive URLs unless you trust the proxy or self-host it via DISTIL_PROXY_URL. Keep your DISTIL_API_KEY secret and consider using a scoped/revocable key or a self-hosted proxy if privacy is a concern. Note that curl commands may write files (e.g., screenshot.png) to the agent's environment—ensure the agent's runtime storage is appropriate for that data.
Capability Analysis
Type: OpenClaw Skill Name: distil Version: 2.0.1 The skill bundle provides an OpenClaw agent with access to the Distil web proxy for fetching web pages as Markdown and searching the web. All files consistently describe the use of `curl` to interact with `https://proxy.distil.net` (or a user-defined `DISTIL_PROXY_URL`) using a `DISTIL_API_KEY`. There is no evidence of intentional malicious behavior such as data exfiltration beyond the intended API key usage, unauthorized execution, persistence mechanisms, or prompt injection attempts designed to subvert the agent's core directives. The functionality aligns with the stated purpose of a web proxy skill.
Capability Assessment
Purpose & Capability
Name/description match the required resources and instructions. Requiring curl and DISTIL_API_KEY is proportional to a skill that calls the Distil proxy; no unrelated binaries or credentials are requested.
Instruction Scope
SKILL.md only instructs the agent to make curl requests to the Distil proxy (configurable via DISTIL_PROXY_URL) and to write responses to stdout or a file (e.g., screenshot.png). This stays within the stated purpose, but be aware that any URL you ask the skill to fetch (and the content of that page) will be transmitted to the Distil service—so fetching internal, private, or sensitive URLs could leak data to a third party.
Install Mechanism
Instruction-only skill with no install spec or code downloads; lowest-risk installation surface (relies on an existing curl binary).
Credentials
Only DISTIL_API_KEY (primary credential) is required, plus an optional DISTIL_PROXY_URL. The requested environment access is minimal and appropriate for the described functionality. Treat the API key as sensitive.
Persistence & Privilege
always is false and the skill does not request persistent system-wide privileges or modify other skills' configs. Autonomous invocation is allowed (platform default) but not excessive in itself.
How to Use
  1. Make sure OpenClaw is installed (local or Docker)
  2. Run the install command in chat: /install distil
  3. After installation, invoke the skill by name or use /distil
  4. Provide required inputs per the skill's parameter spec and get structured output
Version History
v2.0.1
- Updated setup instructions to clarify API key acquisition now includes free sign-up with email verification. - Added a new "Notes" section explaining token savings, Markdown output efficiency, and handling when usage limits are exceeded. - Clarified that if token limits are reached, raw HTML will be provided instead of Markdown, along with an instructional HTML comment.
v2.0.0
Major update: Distil skill now uses direct curl commands instead of requiring the distil CLI. - Removed dependency on Node.js, npm, and the distil-proxy CLI; now requires only curl. - Updated all command examples and documentation to use curl for fetching and searching. - Simplified setup steps — just set your API key and (optionally) custom proxy URL. - Metadata reflects reduced dependencies: only curl is required. - Output and error handling instructions updated for direct HTTP responses.
v1.0.5
- Switched the Node package from distil-mcp to distil-proxy for CLI installation. - Updated install instructions to use distil-proxy. - No changes to commands or environment variables.
v1.0.4
- Version bumped from 1.0.2 to 1.0.4 - No changes to files or documentation content detected
v1.0.3
- Updated references from distil.ai to distil.net throughout the documentation. - Changed the install command to use global `-g` installation for the `distil` CLI. - Updated the default `DISTIL_PROXY_URL` to `https://proxy.distil.net`. - Clarified that `DISTIL_API_KEY` is required, and adjusted environment variable documentation. - Simplified and clarified setup and example instructions.
v1.0.1
- Updated description to reference the distil.net proxy. - Bumped version from 1.0.0 to 1.0.1. - No functional or command changes; documentation update only.
v1.0.0
Initial release of the Distil skill. - Provides commands to fetch web pages as clean Markdown and perform web searches via the Distil proxy. - Supports additional features like screenshots, full page renders, raw content retrieval, and cache bypassing. - Easy setup with environment variable configuration and npm package installation. - Allows passing extra HTTP headers for advanced control. - Offers direct curl examples as an alternative to the CLI.
Metadata
Slug distil
Version 2.0.1
License
All-time Installs 4
Active Installs 4
Total Versions 7
Frequently Asked Questions

What is Distil the web & PDFs into markdown?

Fetch web pages as clean Markdown and search the web via the distil.net proxy. It is an AI Agent Skill for Claude Code / OpenClaw, with 913 downloads so far.

How do I install Distil the web & PDFs into markdown?

Run "/install distil" in the OpenClaw or Claude Code chat to install it in one step — no extra setup required.

Is Distil the web & PDFs into markdown free?

Yes, Distil the web & PDFs into markdown is completely free (open-source). You can download, install and use it at no cost.

Which platforms does Distil the web & PDFs into markdown support?

Distil the web & PDFs into markdown is cross-platform and runs anywhere OpenClaw / Claude Code is available (cross-platform).

Who created Distil the web & PDFs into markdown?

It is built and maintained by Mikel Lindsaar (@mikel); the current version is v2.0.1.

💬 Comments