pinchbench
olearycrew 作
Coding Agents & IDEs概要
Run PinchBench benchmarks to evaluate OpenClaw agent performance across real-world tasks.
説明
このスキルでできること
PinchBench measures LLM model performance as the brain of an OpenClaw agent across 23 standardized tasks. Tasks span calendar, email, research, coding, file management, and multi-step workflows. Results are graded automatically and can be submitted to a public leaderboard at pinchbench.com.
Provides 23 reproducible, auto-graded tasks with a public leaderboard, making model comparisons objective rather than anecdotal.
使いどころ
- Comparing two models on email triage and calendar tasks before picking one
- Running the automated-only suite for a quick performance check before a release
- Submitting benchmark scores to the PinchBench public leaderboard
- Testing a specific subset of tasks like market research or PDF summarization
- Verifying a new OpenClaw setup handles multi-step API workflows correctly
ワークフロー例
User asks: How well does claude-sonnet-4 handle email and calendar tasks?
- 1
Register for a PinchBench API token using the --register flag
- 2
Run the benchmark targeting specific tasks: --suite task_01_calendar,task_16_email_triage,task_17_email_search
- 3
Wait for tasks to complete and results to save as JSON in the results/ directory
- 4
Inspect scores with jq to extract per-task grading means
- 5
Results upload automatically to the leaderboard using the registered token
Per-task scores for calendar and email tasks, plus an overall average, visible on the pinchbench.com leaderboard
要件
- PinchBench API token for leaderboard submission (register once via --register flag)
インストール
clawhub install olearycrew/pinchbenchCLIコマンドを実行するか、Claude Code、Cursor、Codexなどお使いのコーディングエージェントにプロンプトを貼り付けてください。