Web Crawl

Advanced web crawling and content extraction tool with multiple extraction modes

This skill appears to do what it says (crawl and extract web content), but take the following precautions before installing or allowing it to run autonomously: - Review the full web_crawl.py file in your environment (the provided manifest shows the file content truncated), because hidden code or unexpected behavior could be present in the omitted portion. - Be aware that crawling arbitrary URLs can access internal services (metadata endpoints, intranet, admin consoles) if the agent has network access — avoid allowing this skill to scan sensitive hosts or provide a restricted allowlist of target domains. - Examples include an exec-style command that runs Python from the skill workspace. Do not run shell execs or workspace-local scripts without explicit review and user consent. - Ensure required Python dependencies (requests, beautifulsoup4) are installed from trusted sources. If you want higher assurance, ask the skill author for a full unobfuscated source, or run it in a sandboxed environment and limit its outbound network access and allowed target domains.

Type: OpenClaw Skill Name: web-crawl Version: 1.0.0 The 'web-crawl' skill is a legitimate and well-structured tool designed for advanced web content extraction and research. The core logic in `web_crawl.py` uses standard libraries (requests, BeautifulSoup) to convert HTML to various formats like Markdown and JSON, including support for CSS selectors and parallel crawling. The `research.py` script provides templates and orchestration logic to help an AI agent perform multi-step research. No evidence of data exfiltration, malicious execution, or harmful prompt injection was found; the code and instructions are entirely consistent with the stated purpose of web crawling and information synthesis.