تخطَّ إلى المحتوى
TakoAPI
العودة إلى المهارات

midscene-android-automation

بواسطة quanru

Web & Frontend Development
1.1kالتنزيلات وفق ما يذكره clawskills.sh
0المشاهدات
لا إعجاباتسجّل الدخول للإعجاب بهذه المهارة
—لم يُقيَّم بعد
تقييمك

موجز

Vision-driven Android device automation using Midscene.

الوصف

ما الذي تفعله هذه المهارة

Controls Android devices using screenshot-based vision, with no DOM access or accessibility labels required. Accepts natural language commands to tap, swipe, type, launch apps, and capture screenshots. Works across any Android app regardless of the underlying framework.

Unlike Appium or UI Automator, it requires no accessibility labels or view hierarchy access, so it works on any app including games and fully custom-rendered UIs.

متى تستخدمها

  • Automating repetitive tasks in mobile apps that have no scriptable API
  • Extracting data from Android apps without a web or API equivalent
  • Testing a mobile UI without writing platform-specific test code
  • Running multi-step workflows in messaging or social apps
  • Capturing screenshots of specific app states for documentation or bug reports

مثال على سير العمل

الإدخال

User asks: 'Open Settings, go to Wi-Fi, and tell me the connected network name'

الوكيل
  1. 1

    Connect to the Android device via ADB

  2. 2

    Launch the Settings app using adb shell am start and take a screenshot to confirm it is in the foreground

  3. 3

    Run act with the prompt 'navigate to Wi-Fi settings and read the connected network name'

  4. 4

    Take a screenshot to verify the current screen state

  5. 5

    Disconnect and report the network name extracted from the screen

الإخراج

Connected Wi-Fi network name read from the device screen and reported to the user

المتطلبات

  • MIDSCENE_MODEL_API_KEY environment variable from a supported AI vision model provider
  • MIDSCENE_MODEL_NAME, MIDSCENE_MODEL_BASE_URL, and MIDSCENE_MODEL_FAMILY environment variables
  • Android SDK Platform Tools (provides adb)
  • Android device with USB debugging enabled in Developer Options

التثبيت

clawhub install quanru/midscene-android-automation

شغّل أمر واجهة سطر الأوامر، أو الصق النص في Claude Code أو Cursor أو Codex أو أي وكيل برمجة تستخدمه.

الروابط