تخطَّ إلى المحتوى
TakoAPI
العودة إلى المهارات

pinchbench

بواسطة olearycrew

Coding Agents & IDEs
439التنزيلات وفق ما يذكره clawskills.sh
0المشاهدات
لا إعجاباتسجّل الدخول للإعجاب بهذه المهارة
—لم يُقيَّم بعد
تقييمك

موجز

Run PinchBench benchmarks to evaluate OpenClaw agent performance across real-world tasks.

الوصف

ما الذي تفعله هذه المهارة

PinchBench measures LLM model performance as the brain of an OpenClaw agent across 23 standardized tasks. Tasks span calendar, email, research, coding, file management, and multi-step workflows. Results are graded automatically and can be submitted to a public leaderboard at pinchbench.com.

Provides 23 reproducible, auto-graded tasks with a public leaderboard, making model comparisons objective rather than anecdotal.

متى تستخدمها

  • Comparing two models on email triage and calendar tasks before picking one
  • Running the automated-only suite for a quick performance check before a release
  • Submitting benchmark scores to the PinchBench public leaderboard
  • Testing a specific subset of tasks like market research or PDF summarization
  • Verifying a new OpenClaw setup handles multi-step API workflows correctly

مثال على سير العمل

الإدخال

User asks: How well does claude-sonnet-4 handle email and calendar tasks?

الوكيل
  1. 1

    Register for a PinchBench API token using the --register flag

  2. 2

    Run the benchmark targeting specific tasks: --suite task_01_calendar,task_16_email_triage,task_17_email_search

  3. 3

    Wait for tasks to complete and results to save as JSON in the results/ directory

  4. 4

    Inspect scores with jq to extract per-task grading means

  5. 5

    Results upload automatically to the leaderboard using the registered token

الإخراج

Per-task scores for calendar and email tasks, plus an overall average, visible on the pinchbench.com leaderboard

المتطلبات

  • PinchBench API token for leaderboard submission (register once via --register flag)

التثبيت

clawhub install olearycrew/pinchbench

شغّل أمر واجهة سطر الأوامر، أو الصق النص في Claude Code أو Cursor أو Codex أو أي وكيل برمجة تستخدمه.

الروابط