تخطَّ إلى المحتوى
TakoAPI
العودة إلى المهارات

screen-vision

بواسطة ls18166407597-design

AI & LLMs
—لم يُقيَّم بعد
تقييمك

موجز

macOS Local OCR & Automation Tool using Vision Framework.

الوصف

ما الذي تفعله هذه المهارة

A macOS tool that runs OCR on screen content using Apple's built-in Vision framework. It extracts text and [X, Y] coordinates locally, sending only the recognized text to the AI rather than full screenshots. Supports mixed Chinese and English recognition.

Running OCR locally via the Vision framework avoids sending screenshots to an AI API, which eliminates token costs and keeps screen content private.

متى تستخدمها

  • Reading text from a non-standard desktop app that has no API
  • Monitoring a balance or progress indicator on screen without manual checking
  • Clicking a button in a GUI app by locating its label text
  • Automating repetitive UI interactions across any macOS application
  • Extracting text from a screen region that can't be copy-pasted

مثال على سير العمل

الإدخال

User asks: help me click the Confirm button in this app

الوكيل
  1. 1

    Runs vision_ocr.swift to scan the current screen

  2. 2

    Receives recognized text with [X, Y] coordinates for each element

  3. 3

    Locates the text matching 'Confirm' and retrieves its coordinates

  4. 4

    Passes coordinates to click.swift to perform a physical mouse click

الإخراج

The Confirm button is clicked at the detected screen coordinates

المتطلبات

  • macOS Screen Recording permission enabled for your terminal app (System Settings > Privacy & Security > Screen Recording)
  • macOS Accessibility permission enabled for your terminal app (System Settings > Privacy & Security > Accessibility)

التثبيت

clawhub install ls18166407597-design/screen-vision

شغّل أمر واجهة سطر الأوامر، أو الصق النص في Claude Code أو Cursor أو Codex أو أي وكيل برمجة تستخدمه.

الروابط