ClawHub

Web Search Plus

Automation

Unified multi-provider web search and URL extraction skill with intelligent auto-routing across Serper, Brave, Tavily, Querit, Linkup, Exa, Firecrawl, Perplexity, You.com, SearXNG, and SerpBase.

Install

openclaw skills install web-search-plus

Web Search Plus

Stop choosing search providers. Let the skill do it for you.

This skill now connects you to 11 search providers and adds a companion extraction flow for pulling content from URLs. Broad web query? → Brave or Serper. Research question? → Tavily or Exa. Need citations and grounding? → Linkup. Want scrape-ready content? → Firecrawl. Prefer privacy? → SearXNG. Need low-cost Google SERP with prepaid credits? → SerpBase (explicit/fallback-only).

✨ What Makes This Different?

  • Just search — no need to think about which provider to use
  • Smart routing — query analysis picks the best provider automatically
  • 11 providers, 1 interface — general web, research, semantic discovery, direct answers, privacy-first, prepaid-credits, and extraction-capable providers together
  • URL extraction included — pull markdown/HTML content with fallback across five providers
  • Works with just 1 credential — start with any single provider, add more later
  • Free/self-hosted options available — SearXNG can run at $0 API cost

🚀 Quick Start

# Interactive setup (recommended for first run)
python3 scripts/setup.py

# Or manually
cp .env.example .env
python3 scripts/search.py -q "latest OpenClaw release"
python3 scripts/extract.py --url https://example.com

The wizard explains providers, collects keys, and sets defaults.

🔑 Providers

Search providers

  • Serper — shopping, prices, local, and general Google-style results; fast broad fallback
  • Brave — independent web index and generic current-web queries; strong non-Google complement
  • Tavily — research, explanations, and synthesis; strong research routing
  • Querit — multilingual and international updates; good for cross-language recency
  • Linkup — source-grounded/citation-heavy search; evidence-first queries
  • Exa — semantic discovery, similar sites, and deep research; supports deep + deep-reasoning
  • Firecrawl — search with scrape-ready metadata; also strong extraction provider
  • Perplexity — direct answers with citations; via PERPLEXITY_API_KEY or KILOCODE_API_KEY
  • You.com — current-web / RAG-friendly snippets; also supports extraction
  • SearXNG — private/self-hosted search; no API key, just instance URL
  • SerpBase — low-cost Google SERP with prepaid credits; explicit/fallback-only (opt-in via --provider serpbase or add to provider_priority in config.json)

Extraction providers

scripts/extract.py auto-falls back across:

  1. Firecrawl
  2. Linkup
  3. Tavily
  4. Exa
  5. You.com

🧠 Routing at a Glance

Default priority (SerpBase excluded by design — opt-in only):

tavily → linkup → querit → exa → firecrawl → perplexity → brave → serper → you → searxng

Examples:

python3 scripts/search.py -q "weather in Vienna today"
# generic current-web intent → Brave or Serper

python3 scripts/search.py -q "find credible sources for AI tutoring outcomes"
# citation/evidence intent → Linkup

python3 scripts/search.py -q "latest AI policy updates in Germany"
# multilingual + recency → Querit or Tavily

python3 scripts/search.py -p exa --exa-depth deep -q "LLM scaling laws research"
python3 scripts/search.py -p firecrawl -q "YC startups web scraping"
python3 scripts/search.py -p serpbase -q "best laptop 2026"   # explicit SerpBase

Debug routing:

python3 scripts/search.py --explain-routing -q "your query"

📖 Extraction Examples

python3 scripts/extract.py --url https://example.com
python3 scripts/extract.py --url https://docs.linkup.so --provider linkup
python3 scripts/extract.py --url https://example.com --url https://example.org --include-images
python3 scripts/extract.py --url https://example.com --format html --include-raw-html

⚙️ Configuration Notes

  • .env.example documents supported env vars
  • config.example.json includes provider priority and provider-specific defaults
  • config.json is your local runtime config
  • SearXNG still supports explicit URL config and docker-aware auto-detection
  • SerpBase is explicit/fallback-only by default; to include in auto-routing, append "serpbase" to auto_routing.provider_priority in config.json

🔒 Security

SearXNG SSRF protection:

  • Enforces http / https only
  • Blocks common cloud metadata endpoints
  • Blocks private/internal IP resolution unless SEARXNG_ALLOW_PRIVATE=1
  • Uses operator-controlled config/env only for the instance URL

✅ Verification

python3 -m unittest discover -s tests -p 'test_*.py'
python3 scripts/search.py --explain-routing -q "find credible sources for climate change impacts"
python3 scripts/extract.py --url https://example.com --provider auto --compact
More in Automation