Pular para o conteúdo
TakoAPI
Voltar às skills

pinchbench

por olearycrew

Coding Agents & IDEs
439Downloads informados pelo clawskills.sh
0Visualizações
0 curtidaEntre para curtir esta skill
—Ainda sem avaliações
Sua avaliação

Resumo

Run PinchBench benchmarks to evaluate OpenClaw agent performance across real-world tasks.

Descrição

O que esta skill faz

PinchBench measures LLM model performance as the brain of an OpenClaw agent across 23 standardized tasks. Tasks span calendar, email, research, coding, file management, and multi-step workflows. Results are graded automatically and can be submitted to a public leaderboard at pinchbench.com.

Provides 23 reproducible, auto-graded tasks with a public leaderboard, making model comparisons objective rather than anecdotal.

Quando usá-la

  • Comparing two models on email triage and calendar tasks before picking one
  • Running the automated-only suite for a quick performance check before a release
  • Submitting benchmark scores to the PinchBench public leaderboard
  • Testing a specific subset of tasks like market research or PDF summarization
  • Verifying a new OpenClaw setup handles multi-step API workflows correctly

Exemplo de fluxo de trabalho

ENTRADA

User asks: How well does claude-sonnet-4 handle email and calendar tasks?

AGENTE
  1. 1

    Register for a PinchBench API token using the --register flag

  2. 2

    Run the benchmark targeting specific tasks: --suite task_01_calendar,task_16_email_triage,task_17_email_search

  3. 3

    Wait for tasks to complete and results to save as JSON in the results/ directory

  4. 4

    Inspect scores with jq to extract per-task grading means

  5. 5

    Results upload automatically to the leaderboard using the registered token

SAÍDA

Per-task scores for calendar and email tasks, plus an overall average, visible on the pinchbench.com leaderboard

Requisitos

  • PinchBench API token for leaderboard submission (register once via --register flag)

Instalação

clawhub install olearycrew/pinchbench

Execute o comando da CLI ou cole o prompt no Claude Code, Cursor, Codex ou no agente de código que você usa.

Links