تخطَّ إلى المحتوى
TakoAPI
العودة إلى المهارات

crawl-for-ai

بواسطة angusthefuzz

Web & Frontend Development
2.4kالتنزيلات وفق ما يذكره clawskills.sh
5النجوم
0المشاهدات
لا إعجاباتسجّل الدخول للإعجاب بهذه المهارة
—لم يُقيَّم بعد
تقييمك

موجز

Web scraping using local Crawl4AI instance.

الوصف

ما الذي تفعله هذه المهارة

Fetches full web page content using a local Crawl4AI instance with JavaScript rendering. Offers two endpoints: a proxy for clean markdown output and a direct endpoint for structured data including links, HTML, and media. No usage limits since it runs locally.

Runs on a local instance with no rate limits or API costs, and reliably handles dynamic JavaScript content that simpler fetch-based scrapers miss.

متى تستخدمها

  • Scraping JavaScript-rendered single-page applications
  • Extracting article text from JS-heavy or dynamic news sites
  • Collecting product data from e-commerce pages
  • Archiving web pages with full link and media metadata
  • Pulling content from documentation portals that require JS execution

مثال على سير العمل

الإدخال

User asks: Get the full content from https://example.com/docs/getting-started

الوكيل
  1. 1

    Identify the target URL from the user request

  2. 2

    Call crawl4ai.js with the URL via the proxy endpoint for clean output

  3. 3

    Receive the page rendered as structured markdown with metadata

  4. 4

    Return the extracted content to the user

الإخراج

Full page content as clean markdown, including any dynamically loaded text

المتطلبات

  • Self-hosted Crawl4AI instance running locally or on a server
  • CRAWL4AI_URL environment variable set to the instance URL
  • CRAWL4AI_KEY environment variable if the instance requires authentication

التثبيت

clawhub install angusthefuzz/crawl-for-ai

شغّل أمر واجهة سطر الأوامر، أو الصق النص في Claude Code أو Cursor أو Codex أو أي وكيل برمجة تستخدمه.

الروابط