search-cluster
par 1999azzar
Search & ResearchRésumé
Unified search aggregator for Google, Wikipedia, Reddit, NewsAPI, and RSS feeds with optional Redis caching.
Description
Ce que fait ce skill
Multi-provider search aggregator that queries Google CSE, Wikipedia, Reddit, GNews RSS, and DuckDuckGo (via headless scraping) in a single call. Returns structured JSON with source, title, link, and sanitized snippet per result. Supports optional Redis caching and runs the scraping provider in a subprocess-isolated virtual environment.
Querying five independent providers in one call reduces coverage gaps that occur when any single source is rate-limited, blocked, or returns poor results.
Quand l'utiliser
- Aggregating news and forum discussion results on a single topic across five sources
- Cross-referencing Wikipedia summaries with Reddit threads during research
- Building a search backend with automatic provider-level fallback
- Fetching DuckDuckGo results without triggering bot detection
- Caching repeated search queries to reduce API call volume
Exemple de workflow
User asks: "Find recent news and discussions about LLM fine-tuning"
- 1
Invokes scripts/search-cluster.py with provider flag 'all' and the query string
- 2
Dispatches requests to Google CSE, Wikipedia, Reddit, and GNews RSS concurrently
- 3
Spawns an isolated subprocess running stealth_fetch.py to query DuckDuckGo via Scrapling
- 4
Sanitizes and normalizes all provider responses into a shared schema
- 5
Writes aggregated results to Redis cache if REDIS_HOST is configured
Structured JSON array where each entry contains source, title, link, and sanitized snippet
Prérequis
- GOOGLE_API_KEY (optional, Google Custom Search)
- GOOGLE_CSE_ID (optional, Google Custom Search Engine ID)
- SCRAPLING_PYTHON_PATH (optional, path to scrapling venv python binary)
- REDIS_HOST and REDIS_PORT (optional, for result caching)
Installation
clawhub install 1999azzar/search-clusterExécute la commande CLI, ou colle l'instruction dans Claude Code, Cursor, Codex ou l'agent de code que tu utilises.