跳到正文
TakoAPI
返回技能列表

web-scraper-as-a-service

由 seanwyngaard 提供

Web & Frontend Development
1.5k下载量由 clawskills.sh 提供
3星标
0浏览量
0 个赞登录后即可为这个技能点赞
—暂无评分
你的评分

简介

Build client-ready web scrapers with clean data output.

描述

该技能的功能

Turns a scraping brief or target URL into a complete client-ready deliverable. Analyzes the target site, generates a Python scraper with rate limiting and resume support, cleans the output data, and packages everything with documentation.

Covers the full pipeline from site analysis to client handoff, including data cleaning and a quality report, rather than just generating a bare scraper script.

何时使用

  • Extracting product catalogs from e-commerce sites
  • Aggregating real estate listings across multiple pages
  • Collecting job postings from a job board
  • Archiving news articles from a publication
  • Building a business directory dataset from a listings site

示例工作流

输入

User asks: Scrape all products from example-store.com — need name, price, description, images. CSV output.

智能体
  1. 1

    Fetches the target URL to determine if it is server-rendered or JavaScript-rendered

  2. 2

    Identifies pagination pattern, data structure, and visible anti-scraping measures

  3. 3

    Generates scrape.py with rate limiting, user-agent rotation, retry logic, and resume capability

  4. 4

    Runs the scraper, removes duplicates, and normalizes field values

  5. 5

    Packages data.csv, data.json, a data quality report, and a README into delivery/

输出

2,487 product records in delivery/data.csv with a data quality report showing field completeness and any cleaned values

安装

clawhub install seanwyngaard/web-scraper-as-a-service

运行 CLI 命令,或把这段提示词粘贴到 Claude Code、Cursor、Codex 或你在用的任意编程智能体里。

链接