Перейти к содержимому
TakoAPI
Назад к навыкам

web-scraper-as-a-service

от seanwyngaard

Web & Frontend Development
1.5kЧисло загрузок по данным clawskills.sh
3Звёзды
0Просмотры
0 лайковВойдите, чтобы поставить лайк навыку
—Ещё нет оценок
Ваша оценка

Кратко

Build client-ready web scrapers with clean data output.

Описание

Что делает этот навык

Turns a scraping brief or target URL into a complete client-ready deliverable. Analyzes the target site, generates a Python scraper with rate limiting and resume support, cleans the output data, and packages everything with documentation.

Covers the full pipeline from site analysis to client handoff, including data cleaning and a quality report, rather than just generating a bare scraper script.

Когда его использовать

  • Extracting product catalogs from e-commerce sites
  • Aggregating real estate listings across multiple pages
  • Collecting job postings from a job board
  • Archiving news articles from a publication
  • Building a business directory dataset from a listings site

Пример сценария

ВВОД

User asks: Scrape all products from example-store.com — need name, price, description, images. CSV output.

АГЕНТ
  1. 1

    Fetches the target URL to determine if it is server-rendered or JavaScript-rendered

  2. 2

    Identifies pagination pattern, data structure, and visible anti-scraping measures

  3. 3

    Generates scrape.py with rate limiting, user-agent rotation, retry logic, and resume capability

  4. 4

    Runs the scraper, removes duplicates, and normalizes field values

  5. 5

    Packages data.csv, data.json, a data quality report, and a README into delivery/

ВЫВОД

2,487 product records in delivery/data.csv with a data quality report showing field completeness and any cleaned values

Установка

clawhub install seanwyngaard/web-scraper-as-a-service

Выполните команду CLI или вставьте промпт в Claude Code, Cursor, Codex — любой кодинг-агент, которым пользуетесь.

Ссылки