decodo-scraper
donatasdecodo tarafından
Web & Frontend DevelopmentÖzet
Search Google, scrape web pages, Amazon product pages, YouTube subtitles, or Reddit (post/subreddit)
Açıklama
Bu Beceri Ne Yapar
Connects to the Decodo Web Scraping API to search Google, scrape arbitrary web pages, fetch Amazon product data, retrieve YouTube transcripts, and read Reddit posts or subreddits. Each target returns structured output suited for programmatic use: JSON for search and Amazon results, markdown for pages, plain text for transcripts.
Handles JavaScript-rendered pages and structured parsers for Google, Amazon, and Reddit that raw HTTP requests cannot replicate.
Ne Zaman Kullanılır
- Looking up current prices and specs for a product on Amazon before buying
- Pulling a YouTube video transcript to summarize or translate
- Checking what a subreddit is currently discussing without opening a browser
- Running a Google search and extracting organic results for further processing
- Scraping a documentation page or article into clean markdown for an LLM prompt
Örnek İş Akışı
User asks: find the top Google results for 'open source LLM benchmarks' and summarize the snippets
- 1
Runs scrape.py with --target google_search --query 'open source LLM benchmarks'
- 2
Receives JSON with organic, related_questions, and related_searches sections
- 3
Extracts titles and snippets from the organic array
- 4
Compiles a summary from the top results
A list of the top organic results with titles, URLs, and snippets summarized in plain text
Gereksinimler
- DECODO_AUTH_TOKEN environment variable (Basic auth token from Decodo Dashboard under Scraping APIs)
Kurulum
clawhub install donatasdecodo/decodo-scraperCLI komutunu çalıştırın ya da istemi Claude Code, Cursor, Codex veya kullandığınız kodlama ajanına yapıştırın.