Azure function returns tuple

[JaroslawMargasDev]

1. Initialize model:
   model = AzureChatOpenAI(
   openai_api_version="",
   azure_deployment="",
   azure_endpoint="",
   openai_api_key="",
   model="",
   validate_base_url=False,
   )

2. Create Instance of the DeepEvalBaseLLM
   deepEvalModel = GPTModelAzure(model=model)

3. Create metric
   metric = AnswerRelevancyMetric(model=deepEvalModel, include_reason=True)

4. Measure
   await metric.a_measure(test_case)

In this case the response returns not a string but tuple (score and Json string) , so trimAndLoadJson() method fails on find() metod because of tuple.

This fix check the instance of the input_string.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
evals-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 24, 2025 9:09am

[penguine-ip]

Hey @JaroslawMargasDev how is GPTModelAzure implemented? It should just return a string

[JaroslawMargasDev]

Hey @JaroslawMargasDev how is GPTModelAzure implemented? It should just return a string

@penguine-ip

GPTModelAzure(model=model) its same as GPTModel(DeepEvalBaseLLM) class:

instead of load_model() which uses a KEY_FILE_HANDLER I have got:
def load_model(self):
if self.should_use_azure_openai():
return self.model

Each initialize_model() in the metric returns native model:
return model, True

As a model I'm using AzureChatOpenAI: https://oai.azure.com
Name: gpt-4o
Model version: 2024-08-06