MCP Web Search Crawler
@tomas-hanzlik
About MCP Web Search Crawler
A lightweight MCP server for web searching via DuckDuckGo and extracting page content as Markdown.
Config
Add this server to your MCP-compatible client using the configuration below.
{
"mcpServers": {
"mcp-web-search-crawl": {
"command": "uvx",
"args": [
"--from",
"git+https://github.com/tomas-hanzlik/mcp-web-search-crawl",
"mcpsearchcrawl"
]
}
}
}Tools
No tools detected
Fetch the live tool list by running this server in a temporary sandbox using the button above.
Overview
What is MCP Web Search Crawler?
MCP Web Search Crawler is a Model Context Protocol server that provides web search and crawling capabilities. It enables AI assistants and other MCP clients to search the web using DuckDuckGo and crawl specific URLs to extract content in markdown format.
How to use MCP Web Search Crawler?
Clone the repository, then configure the server in your MCP client using the uv command with a directory argument, or use uvx from the GitHub repository. It exposes a tool called search_links that accepts a query parameter. The server can be run with --transport sse or --transport stdio.
Key features of MCP Web Search Crawler
- Web Search via DuckDuckGo returning links, titles, and body snippets
- URL crawling returning content as markdown
- Configurable via MCP client JSON
- Programmatic testing with Python client (fastmcp)
- Supports SSE and stdio transports
Use cases of MCP Web Search Crawler
- AI assistants searching the web for current information
- Crawling specific web pages to extract markdown content
- Combining search and crawl for research tasks
- Integrating with MCP-compatible clients for automated web data retrieval
FAQ from MCP Web Search Crawler
What dependencies are required?
Requires uv and Python. The server uses DuckDuckGo as its search backend and does not need API keys.
How do I test the server programmatically?
Use the provided Python script with the fastmcp Client, then call tools like search_links with a query parameter.
What transport options are available?
The server supports SSE and stdio transports. Run it with --transport sse or --transport stdio.
How can I debug the server?
Use MCP Inspector with the command: uv run fastmcp dev src/mcp_web_search_crawl/server.py:mcp --with-editable .
Frequently asked questions
What dependencies are required?
Requires `uv` and Python. The server uses DuckDuckGo as its search backend and does not need API keys.
How do I test the server programmatically?
Use the provided Python script with the `fastmcp` Client, then call tools like `search_links` with a query parameter.
What transport options are available?
The server supports SSE and stdio transports. Run it with `--transport sse` or `--transport stdio`.
How can I debug the server?
Use MCP Inspector with the command: `uv run fastmcp dev src/mcp_web_search_crawl/server.py:mcp --with-editable .`
Basic information
More Search MCP servers
Everything Search MCP Server
mamertofabian
Google Search MCP Server
mixelpixxMCP Server built for use with Claude Code, Claude Desktop, VS Code, Cline - enable google search and ability to follow links and research websites
Exa MCP Server 🔍
exa-labsExa MCP for web search and web crawling!
Google News MCP Server
ChanMeng666【Star-crossed coders unite!⭐️】Model Context Protocol (MCP) server implementation providing Google News search capabilities via SerpAPI, with automatic news categorization and multi-language support.
🚀 OneSearch MCP Server: Web Search & Crawl & Scraper & Extract
yokingma🚀 OneSearch MCP Server: Web Search & Scraper & Extract, Support agent-browser, SearXNG, Tavily, DuckDuckGo, Bing, etc.
Comments