Back to Browse

Web Research MCP Server

Search & WebModerate5.3MCP RegistryLocal
Free

Server data from the Official MCP Registry

Stealth Google Search and token-efficient web content extraction with undetected browser automation.

About

Stealth Google Search and token-efficient web content extraction with undetected browser automation.

Security Report

5.3
Moderate5.3Moderate Risk

Valid MCP server (1 strong, 1 medium validity signals). 7 known CVEs in dependencies (1 critical, 0 high severity) Imported from the Official MCP Registry.

6 files analyzed ยท 7 issues found

Security scores are indicators to help you make informed decisions, not guarantees. Always review permissions before connecting any MCP server.

Permissions Required

This plugin requests these system permissions. Most are normal for its category.

file_system

Check that this permission is expected for this type of plugin.

HTTP Network Access

Connects to external APIs or services over the internet.

env_vars

Check that this permission is expected for this type of plugin.

Shell Command Execution

Runs commands on your machine. Be cautious โ€” only use if you trust this plugin.

What You'll Need

Set these up before or after installing:

HTTP port for streamable MCP and health check endpoints (default: 3000)Optional

Environment variable: PORT

Optional HTTP/HTTPS proxy URI (e.g. http://user:pass@host:port)Required

Environment variable: HTTP_PROXY

Persistent directory storing Chrome profile, cookies, and cacheOptional

Environment variable: BROWSER_USER_DATA_DIR

Default navigation timeout in millisecondsOptional

Environment variable: BROWSER_TIMEOUT_MS

Maximum simultaneous browser tabs allowed by semaphoreOptional

Environment variable: MAX_CONCURRENT_PAGES

Lifetime requests before background browser restart to prevent RAM fragmentationOptional

Environment variable: MAX_REQUESTS_BEFORE_RECYCLE

Upper bound memory guard in bytes against massive HTML payloadsOptional

Environment variable: MAX_CONTENT_LENGTH

Maximum wait time in ms for initial network stabilization before scrollingOptional

Environment variable: NETWORK_IDLE_TIMEOUT_MS

Debounce delay in ms after each scroll and semantic unfolding stepOptional

Environment variable: SCROLL_DELAY_MS

Distance in pixels per scroll incrementOptional

Environment variable: SCROLL_STEP_PX

Final quiet period in ms after scroll loop completesOptional

Environment variable: SCROLL_SETTLE_TIMEOUT_MS

Maximum scroll iterations for lazy content hydration and DOM convergenceOptional

Environment variable: MAX_SCROLL_STEPS

Maximum wait time in ms for Google SERP cards to render before parsing resultsOptional

Environment variable: SEARCH_WAIT_TIMEOUT_MS

Minimum character length for Mozilla Readability article scoringOptional

Environment variable: READABILITY_CHAR_THRESHOLD

How to Install

Install command not verified. See the repository for install instructions.

View repository

Documentation

View on GitHub

From the project's GitHub README.

๐ŸŽญ MCP Web Research Server

Version: 2.2.0 Docker: Universal License: MIT TypeScript: 5.7+ Patchright: Undetected Shukoi: Enterprise

Enterprise-Grade, Ultra-Fast, and Production-Ready Web Research Model Context Protocol (MCP) Server.
Engineered for AI coding agents and autonomous workflows requiring high-accuracy Google Search and high-density, token-efficient webpage content extraction with undetectable anti-bot evasion.


๐ŸŒŸ Key Architectural Features

  • ๐ŸŽญ 100% Undetectable Headed Stealth on Display :99: Built on Patchright (patched driver eliminating Runtime.enable, --enable-automation, and console leaks). Enforces official Google Chrome in genuine headed mode (headless: false) inside an isolated X11 virtual framebuffer (:99) adhering strictly to Patchright's official Zero-Injection Best Practice (zero custom args, zero custom user-agent, zero header tampering). Guarantees 100% authentic consumer browser window metrics and extensions state with zero desktop window pop-ups or focus-stealing on the host OS.
  • โšก Pre-Warmed Persistent Pool (<100ms Tab Acquisition): Browser context is pre-warmed at server boot with a persistent user profile (launchPersistentContext), retaining DNS, TLS session tickets, V8 bytecode, and consent cookies for 38โ€“50% faster navigations.
  • ๐Ÿ”„ Universal Regex-Free Semantic DOM Unfolding & Auto-Scroll: Handles complex e-commerce, SPA, and XHR-heavy sites (e.g. Tokopedia, Shopee, Reddit). Progressively scrolls, triggers IntersectionObserver lazy-loads, unfolds collapsed sections using universal W3C WAI-ARIA standards (aria-expanded="false", data-state="closed", aria-controls) and native HTML5 <details> disclosures with zero language-locking, unclamps CSS line-clamping, and stops when DOM height converges.
  • ๐Ÿ“‰ Token-Efficient & High-Density Markdown: Strips token-wasting avatar images, promotional widgets, and redundant blank lines. Compiles clean GitHub Flavored Markdown via Unified AST (linkedom + @mozilla/readability + rehype-remark + remark-gfm).
  • ๐Ÿ›ก๏ธ Zero-Trust SSRF Defense: Rejects loopback (127.0.0.0/8, ::1), RFC 1918 private subnets (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16), link-local metadata (169.254.169.254, metadata.google.internal), and non-HTTP protocols.
  • ๐ŸŒ Universal Cross-Platform Architecture: Runs identically across macOS, Windows, and Linux via Docker with zero host dependencies and zero physical screen interference.
  • ๐Ÿฅ Real Liveness & Readiness Probing: Provides real subsystem health metrics over both MCP STDIO (healthCheck) and native HTTP REST (GET /health with HTTP 200/503 status codes for Kubernetes / Docker).

๐ŸŒ Universal Cross-Platform Execution

Thanks to hermetic Docker containerization, mcp-web-research runs with 100% feature parity across all operating systems without any host configuration:

Host PlatformSupport LevelDisplay & Stealth EngineWindow Behavior
macOS (Apple Silicon & Intel)โœ… UniversalVirtual Framebuffer (:99) + Zero-Injection StealthZero Desktop Pop-ups (Rendered in container)
Windows (10 / 11 / Server)โœ… UniversalVirtual Framebuffer (:99) + Zero-Injection StealthZero Desktop Pop-ups (Rendered in container)
Linux (Ubuntu / Debian / RHEL)โœ… UniversalVirtual Framebuffer (:99) + Zero-Injection StealthZero Desktop Pop-ups (Rendered in container)

Anti-Bot Invariant: The browser strictly runs in genuine headed mode (headless: false) inside the virtual framebuffer :99. End users will never see desktop pop-up windows, yet anti-bot inspection systems detect 100% authentic consumer desktop Chrome behavior.


โš™๏ธ 12-Factor Configuration Reference

All settings can be configured via environment variables. Universal defaults are applied automatically if unset:

VariableTypeDefaultDescription
HOSTstring0.0.0.0Network interface address to bind HTTP service on VPS/host.
PORTinteger3000HTTP port for streamable MCP and health check endpoints.
ALLOWED_HOSTSstring*Comma-separated list or wildcard (*) of allowed Host headers for DNS-rebinding protection.
ALLOWED_ORIGINSstring*Comma-separated list or wildcard (*) of allowed Origin headers for CORS security.
MCP_API_KEY / API_KEYstringundefinedOptional secret token(s) for HTTP API protection (comma-separated for key rotation). If unset, auth is disabled.
HTTP_PROXYstringundefinedOptional HTTP/HTTPS proxy URI (e.g. http://user:pass@host:port).
BROWSER_USER_DATA_DIRstring~/.cache/mcp-web-research/chrome-profilePersistent directory storing Chrome cache, cookies, and session state.
BROWSER_TIMEOUT_MSinteger30000Default navigation timeout in milliseconds.
MAX_CONCURRENT_PAGESinteger5Maximum simultaneous browser tabs allowed by semaphore.
MAX_REQUESTS_BEFORE_RECYCLEinteger1000Lifetime requests before background browser restart to prevent RAM fragmentation.
MAX_CONTENT_LENGTHinteger5242880 (5MB)Upper bound memory guard against massive HTML payloads.
NETWORK_IDLE_TIMEOUT_MSinteger5000Maximum wait time in ms for initial network stabilization before scrolling.
SCROLL_DELAY_MSinteger500Debounce delay in ms after each scroll step and semantic unfolding pass (increase to 1000โ€“2000 for slow networks).
SCROLL_STEP_PXinteger1000Distance in pixels per scroll increment.
SCROLL_SETTLE_TIMEOUT_MSinteger500Final quiet period in ms after scroll loop completes.
MAX_SCROLL_STEPSinteger6Maximum scroll loop iterations for lazy content hydration, semantic unfolding, and DOM convergence.
SEARCH_WAIT_TIMEOUT_MSinteger10000Maximum wait time in ms for Google SERP cards to render before parsing results.
READABILITY_CHAR_THRESHOLDinteger20Minimum character length for Mozilla Readability article scoring.

๐Ÿš€ Client Installation & Setup Guide

Option 1: Docker Streamable HTTP Service (Recommended for Remote Agents, VPS & Teams)

1. Quick Start (Default Settings)

Run as an isolated, persistent network service on any machine:

docker run -d -p 3000:3000 --shm-size=1gb --name web-research-mcp --restart unless-stopped ghcr.io/shukoilab/web-research-mcp:latest

Tip on Custom Host Ports: If port 3000 is occupied on your host, forward your desired host port to container port 3000 (e.g. -p 3031:3000), or override the internal port via environment variable (e.g. -p 3031:3031 -e PORT=3031).

2. Production VPS Deployment with Full Configuration Knobs

Customize any operational parameter directly at runtime via environment variables (-e). You do not need to modify or rebuild the Dockerfile:

docker run -d \
  --name web-research-mcp \
  --restart unless-stopped \
  --shm-size=1gb \
  -p 3000:3000 \
  -e HOST="0.0.0.0" \
  -e PORT="3000" \
  -e ALLOWED_HOSTS="*" \
  -e ALLOWED_ORIGINS="*" \
  -e MCP_API_KEY="sk-your-secret-api-key" \
  -e HTTP_PROXY="" \
  -e BROWSER_TIMEOUT_MS="30000" \
  -e MAX_CONCURRENT_PAGES="5" \
  -e MAX_REQUESTS_BEFORE_RECYCLE="1000" \
  -e MAX_CONTENT_LENGTH="5242880" \
  -e NETWORK_IDLE_TIMEOUT_MS="5000" \
  -e SCROLL_DELAY_MS="500" \
  -e SCROLL_STEP_PX="1000" \
  -e SCROLL_SETTLE_TIMEOUT_MS="500" \
  -e MAX_SCROLL_STEPS="6" \
  -e SEARCH_WAIT_TIMEOUT_MS="10000" \
  -e READABILITY_CHAR_THRESHOLD="20" \
  ghcr.io/shukoilab/web-research-mcp:latest
3. Using an Environment File (--env-file)

Keep sensitive keys and parameters organized in a local .env file:

docker run -d \
  -p 3000:3000 \
  --shm-size=1gb \
  --env-file .env \
  --name web-research-mcp \
  --restart unless-stopped \
  ghcr.io/shukoilab/web-research-mcp:latest
4. Docker Compose Deployment (docker-compose.yml)

For automated restarts, VPS orchestration, and persistent stacks:

version: '3.8'

services:
  web-research-mcp:
    image: ghcr.io/shukoilab/web-research-mcp:latest
    container_name: web-research-mcp
    restart: unless-stopped
    shm_size: 1gb
    ports:
      - "3000:3000"
    environment:
      # Network & Security
      - HOST=0.0.0.0
      - PORT=3000
      - ALLOWED_HOSTS=*
      - ALLOWED_ORIGINS=*
      - MCP_API_KEY=sk-your-secret-api-key
      # Browser Performance & Tuning
      - BROWSER_TIMEOUT_MS=30000
      - MAX_CONCURRENT_PAGES=5
      - MAX_REQUESTS_BEFORE_RECYCLE=1000
      - MAX_CONTENT_LENGTH=5242880
      - NETWORK_IDLE_TIMEOUT_MS=5000
      - SCROLL_DELAY_MS=500
      - SCROLL_STEP_PX=1000
      - SCROLL_SETTLE_TIMEOUT_MS=500
      - MAX_SCROLL_STEPS=6
      - SEARCH_WAIT_TIMEOUT_MS=10000
      - READABILITY_CHAR_THRESHOLD=20
  • MCP Endpoint: http://localhost:3000/mcp (Streamable HTTP transport)
  • Healthcheck Probe: GET http://localhost:3000/health (HTTP 200/503)
5. Connecting AI Clients with Optional Authentication

In Cursor / OpenCode (opencode.json / remote MCP settings):

{
  "type": "remote",
  "url": "http://your-vps-ip:3000/mcp",
  "headers": {
    "Authorization": "Bearer sk-your-secret-api-key"
  }
}

Testing via cURL:

# Public liveness healthcheck probe (always unauthenticated)
curl -s http://localhost:3000/health

# MCP request with Bearer Token (or X-API-Key header)
curl -s -X POST http://localhost:3000/mcp \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-your-secret-api-key" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"curl-client","version":"1.0.0"}}}'

Option 2: Docker Local STDIO Container (Recommended for Claude Desktop)

Runs inside the sealed container without needing Node.js or Chrome installed on the host machine:

Claude Desktop

Add to your claude_desktop_config.json:

  • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
  • Windows: %APPDATA%\Claude\claude_desktop_config.json
  • Linux: ~/.config/Claude/claude_desktop_config.json
{
  "mcpServers": {
    "web-research": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "--init",
        "--shm-size=1gb",
        "ghcr.io/shukoilab/web-research-mcp:latest",
        "node",
        "dist/entries/mcp-stdio.js"
      ]
    }
  }
}
Cursor IDE

Add to ~/.cursor/mcp.json (or via Cursor Settings > Features > MCP):

{
  "mcpServers": {
    "web-research": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "--init",
        "--shm-size=1gb",
        "ghcr.io/shukoilab/web-research-mcp:latest",
        "node",
        "dist/entries/mcp-stdio.js"
      ]
    }
  }
}
OpenCode

Add to your opencode.json (or profile config):

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "web-research": {
      "type": "local",
      "command": [
        "docker",
        "run",
        "-i",
        "--rm",
        "--init",
        "--shm-size=1gb",
        "ghcr.io/shukoilab/web-research-mcp:latest",
        "node",
        "dist/entries/mcp-stdio.js"
      ]
    }
  }
}
VS Code (Cline / Roo Code / Claude Dev)
{
  "mcpServers": {
    "web-research": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "--init",
        "--shm-size=1gb",
        "ghcr.io/shukoilab/web-research-mcp:latest",
        "node",
        "dist/entries/mcp-stdio.js"
      ],
      "disabled": false,
      "autoApprove": []
    }
  }
}

Option 3: Local Node.js Development & STDIO Execution

For local development or environments with Node.js 20+ installed:

# Build the project
pnpm install
pnpm run build

# Start the Streamable HTTP server on port 3000
pnpm run start

Or invoke STDIO locally:

{
  "mcpServers": {
    "web-research": {
      "command": "node",
      "args": ["/absolute/path/to/mcp-web-research/dist/entries/mcp-stdio.js"]
    }
  }
}

๐Ÿ› ๏ธ MCP Tools Catalog

1. googleSearch (googleSearchAction)

Executes organic web search on Google using stealth Chrome with automatic consent dialog bypass, breadcrumb canonicalization, and rich snippet extraction. Acts as Step 1 (Discovery & Triage) in agent research workflows, returning candidate URLs and preview snippets.

Input Parameters:

  • query (string, required, 1..500 chars): Search keywords or phrases. Supports Google Dorking operators for high-precision queries (e.g. site:github.com "patchright" inurl:releases, filetype:pdf "machine learning", intitle:"API docs").
  • limit (integer, optional, 1..50, default: 10): Maximum organic results.
  • language (string, optional, default: "en"): Language code (e.g. "en", "id").
  • country (string, optional, default: "us"): Geolocation region code (e.g. "us", "id").
  • safeSearch (boolean, optional, default: true): SafeSearch adult filter.

Example Tool Response:

{
  "query": "typescript documentation",
  "totalReturned": 2,
  "results": [
    {
      "title": "The starting point for learning TypeScript",
      "url": "https://www.typescriptlang.org/docs",
      "redirectUrl": "https://www.google.com/goto?url=...",
      "snippet": "TypeScript extends JavaScript by adding types to the language..."
    }
  ],
  "timestamp": "2026-11-20T00:00:00.000Z"
}

2. webExtract (webExtractAction)

Navigates to any target URL, executes intelligent progressive scrolling to trigger lazy XHR/SPA content (prices, reviews, specs), unfolds collapsed content using universal W3C ARIA standards and native HTML5 disclosures, unclamps visual CSS line-clamping, and serializes clean GitHub Flavored Markdown. Acts as Step 2 (Deep Content Reader) following googleSearch or as Step 1 for direct URLs.

Input Parameters:

  • url (string, required, format: "uri", 1..2048 chars): Target website URL (http:// or https://). Accepts direct URLs or canonical URLs discovered via googleSearch.
  • smartScroll (boolean, optional, default: true): Enable automated progressive scroll, semantic unfolding, and XHR stabilization.
  • ultraFast (boolean, optional, default: false): Block images, external CSS, fonts, media, and manifest (JS + XHR still run) for faster extraction on image-heavy pages.
  • scrollDelayMs (integer, optional, 100..5000, default: 500): Debounce wait time in milliseconds after each scroll step (increase to 1000โ€“2000ms for slow networks).
  • maxScrolls (integer, optional, 0..20, default: 6): Maximum scroll step iterations for lazy content hydration and DOM convergence.
  • timeoutMs (integer, optional, 1000..60000, default: 30000): Maximum navigation deadline.

Example Tool Response:

{
  "url": "https://www.tokopedia.com/kyo-official/ssd-kyo-kaizen-m-2-nvme-512gb-pcie-gen3-x4-2280-ssd-nvme-gen-3",
  "title": "SSD KYO KAIZEN M.2 NVMe 512GB PCIe Gen3 x4 2280 SSD NVMe Gen 3",
  "byline": "KYO Indonesia",
  "excerpt": "KAIZEN diambil dari bahasa Jepang untuk perbaikan berkelanjutan...",
  "markdown": "## SSD KYO KAIZEN M.2 NVMe 512GB...\n\n**Harga:** Rp1.626.050\n**Stok:** 451\n\n### Spesifikasi Lengkap:\n- Tipe: M.2 NVMe PCIe 3 x4\n- Kapasitas: 512GB\n\n## ULASAN PEMBELI\n- **Andri:** Barang diterima dengan baik dan cepat...",
  "timestamp": "2026-11-20T00:00:00.000Z"
}

3. healthCheck (healthAction)

Inspects real-time subsystem readiness, browser pool connectivity, active concurrency, and memory consumption.

Example Output:

{
  "status": "healthy",
  "timestamp": "2026-11-20T00:00:00.000Z",
  "uptimeSec": 1240,
  "browser": {
    "connected": true,
    "prewarmed": true,
    "activeTabs": 0,
    "maxCapacity": 5,
    "requestCount": 42,
    "userDataDir": "/home/shukoi/.cache/mcp-web-research/chrome-profile"
  },
  "display": ":99",
  "memory": {
    "heapUsedMb": 64.25,
    "rssMb": 182.10
  }
}

๐Ÿงช Testing & Verification

Run the full automated verification pipeline:

# Execute unit and live integration tests (68+ tests)
pnpm test

# Verify strict TypeScript type-safety (0 errors)
pnpm run typecheck

# Check formatting and static quality (Biome)
pnpm run lint

# Compile production build to ./dist
pnpm run build

๐Ÿ™ Acknowledgements & Open-Source Credits

This project is built on top of an incredible open-source ecosystem. Sincere gratitude and credits go to the creators and maintainers of the following foundations:

  • Patchright โ€” Undetected browser automation driver eliminating CDP leaks, --enable-automation, and runtime flags for genuine consumer fingerprints.
  • Shukoi Framework (@shukoi/core, @shukoi/mcp) โ€” Modular TypeScript runtime framework powering MCP action dispatching and Web-Standard transport.
  • Hono (hono, @hono/node-server) โ€” Ultra-fast, lightweight Web-Standard HTTP framework powering Streamable HTTP transport and native health probes.
  • Model Context Protocol SDK โ€” Official TypeScript SDK implementing the MCP specification across STDIO and Streamable HTTP transports.
  • Mozilla Readability โ€” Standalone content extraction engine from Mozilla Firefox's Reader View for clean, token-efficient article parsing.
  • LinkeDOM โ€” High-performance, lightweight server-side DOM implementation with near-zero memory footprint.
  • Unified Ecosystem (unified, rehype-parse, rehype-remark, remark-gfm, remark-stringify) โ€” Composable syntax tree (AST) pipeline transforming HTML into high-density GitHub Flavored Markdown.
  • TypeBox โ€” High-performance JSON Schema Type Builder with static type inference enforcing strict input/output contracts.
  • jose โ€” Universal Web Cryptography library for secure token validation and cryptographic operations.
  • Biome & Vitest โ€” Modern development toolchains delivering blazing-fast linting, formatting, and automated testing.

๐Ÿ“„ License

MIT ยฉ Shukoi Lab

Reviews

No reviews yet

Be the first to review this server!