본문으로 건너뛰기
TakoAPI
스킬로 돌아가기

pinchbench

olearycrew 제작

Coding Agents & IDEs
439clawskills.sh가 보고한 다운로드 수
0조회수
좋아요 0개로그인하면 이 스킬에 좋아요를 누를 수 있습니다
—아직 평가 없음
내 평가

요약

Run PinchBench benchmarks to evaluate OpenClaw agent performance across real-world tasks.

설명

이 스킬이 하는 일

PinchBench measures LLM model performance as the brain of an OpenClaw agent across 23 standardized tasks. Tasks span calendar, email, research, coding, file management, and multi-step workflows. Results are graded automatically and can be submitted to a public leaderboard at pinchbench.com.

Provides 23 reproducible, auto-graded tasks with a public leaderboard, making model comparisons objective rather than anecdotal.

사용 시점

  • Comparing two models on email triage and calendar tasks before picking one
  • Running the automated-only suite for a quick performance check before a release
  • Submitting benchmark scores to the PinchBench public leaderboard
  • Testing a specific subset of tasks like market research or PDF summarization
  • Verifying a new OpenClaw setup handles multi-step API workflows correctly

예시 워크플로

입력

User asks: How well does claude-sonnet-4 handle email and calendar tasks?

에이전트
  1. 1

    Register for a PinchBench API token using the --register flag

  2. 2

    Run the benchmark targeting specific tasks: --suite task_01_calendar,task_16_email_triage,task_17_email_search

  3. 3

    Wait for tasks to complete and results to save as JSON in the results/ directory

  4. 4

    Inspect scores with jq to extract per-task grading means

  5. 5

    Results upload automatically to the leaderboard using the registered token

출력

Per-task scores for calendar and email tasks, plus an overall average, visible on the pinchbench.com leaderboard

요구 사항

  • PinchBench API token for leaderboard submission (register once via --register flag)

설치

clawhub install olearycrew/pinchbench

CLI 명령을 실행하거나, 프롬프트를 Claude Code, Cursor, Codex 등 사용하는 코딩 에이전트에 붙여넣으세요.

링크