कंटेंट पर जाएँ
TakoAPI
स्किल्स पर वापस

api-benchmark

polarjunction द्वारा

Coding Agents & IDEs
—अभी तक कोई रेटिंग नहीं
आपकी रेटिंग

संक्षिप्त

Benchmark token generation speed across multiple LLM API providers.

विवरण

यह स्किल क्या करती है

Benchmarks token generation speed across multiple LLM API providers by measuring TTFT, tokens-per-second throughput, and total generation time. Configuration is read from ~/.openclaw/openclaw.json, with support for anthropic-messages, openai-completions, and openai-responses API formats.

Measures TTFT separately from total generation time, giving a more accurate picture of perceived latency than simple end-to-end timing scripts.

इसे कब इस्तेमाल करें

  • Comparing TTFT between OpenAI and Anthropic for a latency-sensitive app
  • Verifying a new API key and endpoint are reachable before deploying
  • Measuring throughput differences between two versions of the same model
  • Checking whether a self-hosted LLM endpoint matches a hosted provider's speed
  • Running repeated benchmark runs to catch API performance regressions

उदाहरण वर्कफ़्लो

इनपुट

User asks: Compare token generation speed across all configured providers

एजेंट
  1. 1

    Reads provider definitions from ~/.openclaw/openclaw.json

  2. 2

    Runs preflight connectivity check on all targets

  3. 3

    Sends short, medium, and long prompts to each provider

  4. 4

    Records TTFT, TPS, and total time per run

  5. 5

    Outputs results as a formatted comparison table

आउटपुट

Table showing TTFT, tokens/sec, and total time for each configured provider and model

आवश्यकताएँ

  • API keys for each LLM provider to benchmark (e.g., ANTHROPIC_API_KEY, OPENAI_API_KEY)

इंस्टॉलेशन

clawhub install polarjunction/api-benchmark

CLI कमांड चलाएँ, या प्रॉम्प्ट को Claude Code, Cursor, Codex या जो भी कोडिंग एजेंट आप इस्तेमाल करते हैं उसमें पेस्ट करें।

लिंक