Usage and Examples

This page provides practical guidance on how to use the Azure AI Evaluation SDK through hands-on examples and common evaluation scenarios. It’s designed to help users quickly get started and build confidence in applying the SDK to real-world tasks.

1. Basic Evaluation Workflow

• Step-by-step guide:
• Load dataset
  • Configure evaluation pipeline
  • Run evaluation
  • View results
• 📘 Quickstart: Evaluate a Model Locally

2. Sample Code Snippets

• Text Classification Example:
• RAG Evaluation Example:

  from azure.ai.evaluation import EvaluationPipeline
  pipeline = EvaluationPipeline.from_config("config.yaml")
  pipeline.run()
  

📘 Evaluating RAG Applications

3. Configuration Options

• YAML Config Structure:
• Define model endpoints
• Specify metrics
• Set dataset paths
• 📘 Configuration Reference

4. Common Scenarios

• Evaluating Azure OpenAI Models
• Comparing Multiple Models
• Evaluating Fairness Across Demographics
• 📘 Fairness Evaluation Guide

5. Troubleshooting Tips

• Common Errors:
• Invalid config paths
• Unsupported dataset formats
• Metric computation failures
• Debugging Tools:
• Verbose logging
• Evaluation reports
• 📘 Troubleshooting Evaluation SDK