Examples Overview
Fair Forge provides comprehensive examples for different deployment scenarios.Available Examples
Jupyter Notebooks
Interactive notebooks for each metric with visualizations
AWS Lambda
Serverless deployment for generators and runners
Example Categories
Metrics Examples
Interactive Jupyter notebooks demonstrating each metric:| Metric | Notebook | Description |
|---|---|---|
| Toxicity | toxicity.ipynb | DIDT framework with Frequentist and Bayesian modes |
| Bias | bias.ipynb | Guardian-based bias detection with visualization |
| Context | context.ipynb | LLM judge for context alignment |
| Conversational | conversational.ipynb | Grice’s Maxims evaluation with radar charts |
| Humanity | humanity.ipynb | NRC emotion analysis with entropy |
| BestOf | bestof.ipynb | Tournament comparison between assistants |
Deployment Examples
Serverless functions for production use:| Module | Lambda | Description |
|---|---|---|
| Generators | generators/aws-lambda | Generate test datasets from context |
| Runners | runners/aws-lambda | Execute tests against AI systems |
Quick Links
Jupyter Notebooks
AWS Lambda
Getting Started
- Choose your metric - Pick the metric that matches your evaluation needs
- Try the notebook - Run the interactive example to understand the API
- Deploy to production - Use Lambda for serverless execution