WebScraping.AI MCP Server

@MCP-Mirror

About WebScraping.AI MCP Server

Mirror of

#webscraping #data-extraction #automation

Config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "webscraping-ai_webscraping-ai-mcp-server": {
      "command": "npx",
      "args": [
        "@modelcontextprotocol/inspector",
        "node",
        "src/index.js"
      ]
    }
  }
}

Tools

No tools detected

Fetch the live tool list by running this server in a temporary sandbox using the button above.

Overview

What is WebScraping.AI MCP Server?

WebScraping.AI MCP Server is a Model Context Protocol (MCP) server that integrates with the WebScraping.AI API to provide web data extraction capabilities to MCP-enabled LLMs. It allows AI agents to scrape web pages, answer questions about content, extract structured data, and retrieve HTML or text with JavaScript rendering and proxy support.

How to use WebScraping.AI MCP Server?

Install and run it via npx with your API key: env WEBSCRAPING_AI_API_KEY=your_api_key npx -y webscraping-ai-mcp. Configure it in Cursor (.cursor/mcp.json) or Claude Desktop (claude_desktop_config.json) by specifying the command and environment variables. All scraping tools accept parameters like url, timeout, js, proxy, country, device, and wait_for.

Key features of WebScraping.AI MCP Server

Question answering about web page content
Structured data extraction with custom fields
Full HTML and plain text retrieval
CSS selector–based content extraction
Support for datacenter and residential proxies with country selection
JavaScript rendering using headless Chrome/Chromium

Use cases of WebScraping.AI MCP Server

Ask an LLM to summarize or find specific information on a webpage
Extract product details (title, price, description) from e-commerce sites
Retrieve the full HTML of a page for further analysis or archiving
Monitor account usage (API requests remaining) via the account tool
Scrape content from multiple page sections using multiple CSS selectors

FAQ from WebScraping.AI MCP Server

What is the difference between WebScraping.AI MCP Server and direct web scraping?

This server wraps WebScraping.AI’s API into the MCP format, making it accessible to any MCP-compatible LLM. It handles rate limiting, retries, proxy selection, and JavaScript rendering automatically, so the LLM can focus on extraction tasks.

What are the runtime dependencies?

You need Node.js and an active WebScraping.AI API key. The server runs via npx or can be installed from npm; no other external services are required.

Where does scraped data live?

All web page data is fetched through the WebScraping.AI API. The server does not store data locally; responses are returned directly to the calling LLM.

Are there any rate limits or concurrency restrictions?

Yes. The default concurrency limit is 5 requests (configurable via WEBSCRAPING_AI_CONCURRENCY_LIMIT). The server includes automatic backoff for rate-limit errors (HTTP 429). Timeouts are capped at 30 s for page retrieval and 2 s for JavaScript rendering (configurable).

What transport and authentication does the server use?

The server communicates over stdio (standard input/output) using the MCP protocol. Authentication is via an API key passed as the environment variable WEBSCRAPING_AI_API_KEY.

Frequently asked questions

What is the difference between WebScraping.AI MCP Server and direct web scraping?

What are the runtime dependencies?

You need Node.js and an active WebScraping.AI API key. The server runs via `npx` or can be installed from npm; no other external services are required.

Where does scraped data live?

All web page data is fetched through the WebScraping.AI API. The server does not store data locally; responses are returned directly to the calling LLM.

Are there any rate limits or concurrency restrictions?

Yes. The default concurrency limit is 5 requests (configurable via `WEBSCRAPING_AI_CONCURRENCY_LIMIT`). The server includes automatic backoff for rate-limit errors (HTTP 429). Timeouts are capped at 30 s for page retrieval and 2 s for JavaScript rendering (configurable).

What transport and authentication does the server use?

The server communicates over stdio (standard input/output) using the MCP protocol. Authentication is via an API key passed as the environment variable `WEBSCRAPING_AI_API_KEY`.

Comments

Basic information

More Other MCP servers

Unity MCP ✨

justinpbarnett

Unity MCP acts as a bridge between AI assistants and your Unity Editor. Give your LLM tools to manage assets, control scenes, edit scripts, and automate tasks within Unity.

12.2K