Bỏ qua tới nội dung
TakoAPI
Quay lại kỹ năng

azure-ai-evaluation-py

bởi thegovind

DevOps & Cloud
1.7kLượt tải do clawskills.sh công bố
1Sao
0Lượt xem
0 lượt thíchĐăng nhập để thích kỹ năng này
—Chưa có đánh giá
Đánh giá của bạn

Tóm tắt

Azure AI Evaluation SDK for Python.

Mô tả

Kỹ năng này làm gì

Python SDK for evaluating generative AI applications using Azure OpenAI. Supports quality metrics (groundedness, relevance, coherence), NLP-based scoring (F1, BLEU, ROUGE), and safety evaluations (violence, hate, self-harm). Results can be logged to Azure AI Foundry for tracking across runs.

Combines quality, NLP, and safety evaluators in one SDK with direct Azure AI Foundry integration, eliminating the need to wire together separate scoring libraries.

Khi nào nên dùng

  • Scoring RAG pipeline responses for groundedness against source documents
  • Running safety checks on chatbot outputs before production deployment
  • Batch evaluating a dataset of query/response pairs with multiple metrics
  • Logging evaluation runs to Azure AI Foundry for regression tracking
  • Building custom domain-specific evaluators for specialized content

Quy trình ví dụ

ĐẦU VÀO

User asks: evaluate my RAG app responses for groundedness and safety

AGENT
  1. 1

    Load model config from AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, and AZURE_OPENAI_DEPLOYMENT env vars

  2. 2

    Initialize GroundednessEvaluator with model config and ContentSafetyEvaluator with Azure AI project scope

  3. 3

    Call evaluate() with test_data.jsonl, mapping query/context/response columns to both evaluators

  4. 4

    Print result metrics and studio_url for Foundry dashboard link

ĐẦU RA

Groundedness and safety scores per row plus aggregate metrics, with a link to view results in Azure AI Foundry

Yêu cầu

  • AZURE_OPENAI_ENDPOINT environment variable
  • AZURE_OPENAI_API_KEY environment variable
  • AZURE_OPENAI_DEPLOYMENT environment variable
  • AIPROJECT_CONNECTION_STRING environment variable (for Foundry logging and safety evaluators)
  • Azure OpenAI resource
  • Azure AI Foundry project (for safety evaluators and result logging)

Cài đặt

clawhub install thegovind/azure-ai-evaluation-py

Chạy lệnh CLI, hoặc dán prompt vào Claude Code, Cursor, Codex hay coding agent bạn đang dùng.

Liên kết