Skip to content

[AI Evaluation] Support evaluation of JSONL dataset files in the dotnet aieval command line tool #6482

@shyamnamboodiripad

Description

@shyamnamboodiripad

The idea would be to allow evaluation of question-answer pairs without needing to author unit tests for them. In addition to question-answer pairs to be evaluated, the JSONL dataset could also include the list of evaluators to be executed + the configuration for setting up LLM / Azure AI Foundry evaluation service connection. It could then internally execute the configured evaluators against each question-answer pair and produce a report containing all the evaluation scores.

Metadata

Metadata

Labels

area-ai-evalMicrosoft.Extensions.AI.Evaluation and related

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions