본문으로 건너뛰기
TakoAPI
스킬로 돌아가기

web-scraper-as-a-service

seanwyngaard 제작

Web & Frontend Development
1.5kclawskills.sh가 보고한 다운로드 수
3스타
0조회수
좋아요 0개로그인하면 이 스킬에 좋아요를 누를 수 있습니다
—아직 평가 없음
내 평가

요약

Build client-ready web scrapers with clean data output.

설명

이 스킬이 하는 일

Turns a scraping brief or target URL into a complete client-ready deliverable. Analyzes the target site, generates a Python scraper with rate limiting and resume support, cleans the output data, and packages everything with documentation.

Covers the full pipeline from site analysis to client handoff, including data cleaning and a quality report, rather than just generating a bare scraper script.

사용 시점

  • Extracting product catalogs from e-commerce sites
  • Aggregating real estate listings across multiple pages
  • Collecting job postings from a job board
  • Archiving news articles from a publication
  • Building a business directory dataset from a listings site

예시 워크플로

입력

User asks: Scrape all products from example-store.com — need name, price, description, images. CSV output.

에이전트
  1. 1

    Fetches the target URL to determine if it is server-rendered or JavaScript-rendered

  2. 2

    Identifies pagination pattern, data structure, and visible anti-scraping measures

  3. 3

    Generates scrape.py with rate limiting, user-agent rotation, retry logic, and resume capability

  4. 4

    Runs the scraper, removes duplicates, and normalizes field values

  5. 5

    Packages data.csv, data.json, a data quality report, and a README into delivery/

출력

2,487 product records in delivery/data.csv with a data quality report showing field completeness and any cleaned values

설치

clawhub install seanwyngaard/web-scraper-as-a-service

CLI 명령을 실행하거나, 프롬프트를 Claude Code, Cursor, Codex 등 사용하는 코딩 에이전트에 붙여넣으세요.

링크