본문으로 건너뛰기
TakoAPI
스킬로 돌아가기

crawl-for-ai

angusthefuzz 제작

Web & Frontend Development
2.4kclawskills.sh가 보고한 다운로드 수
5스타
0조회수
좋아요 0개로그인하면 이 스킬에 좋아요를 누를 수 있습니다
—아직 평가 없음
내 평가

요약

Web scraping using local Crawl4AI instance.

설명

이 스킬이 하는 일

Fetches full web page content using a local Crawl4AI instance with JavaScript rendering. Offers two endpoints: a proxy for clean markdown output and a direct endpoint for structured data including links, HTML, and media. No usage limits since it runs locally.

Runs on a local instance with no rate limits or API costs, and reliably handles dynamic JavaScript content that simpler fetch-based scrapers miss.

사용 시점

  • Scraping JavaScript-rendered single-page applications
  • Extracting article text from JS-heavy or dynamic news sites
  • Collecting product data from e-commerce pages
  • Archiving web pages with full link and media metadata
  • Pulling content from documentation portals that require JS execution

예시 워크플로

입력

User asks: Get the full content from https://example.com/docs/getting-started

에이전트
  1. 1

    Identify the target URL from the user request

  2. 2

    Call crawl4ai.js with the URL via the proxy endpoint for clean output

  3. 3

    Receive the page rendered as structured markdown with metadata

  4. 4

    Return the extracted content to the user

출력

Full page content as clean markdown, including any dynamically loaded text

요구 사항

  • Self-hosted Crawl4AI instance running locally or on a server
  • CRAWL4AI_URL environment variable set to the instance URL
  • CRAWL4AI_KEY environment variable if the instance requires authentication

설치

clawhub install angusthefuzz/crawl-for-ai

CLI 명령을 실행하거나, 프롬프트를 Claude Code, Cursor, Codex 등 사용하는 코딩 에이전트에 붙여넣으세요.

링크