[azure-ai-inference] (BadRequest) Invalid input format when input messages contain tool calling history

[dunalduck0]

• Package Name: azure-ai-inference
• Package Version: 1.0.0b9
• Operating System: Ubuntu
• Python Version: 3.12.10

Describe the bug
Tried the example code sample_chat_completions_with_tools.py. Replacing the endpoint and key with my GPT-4o deployment. Getting error as follows:

azure.core.exceptions.HttpResponseError: (BadRequest) Invalid input format.
Code: BadRequest
Message: Invalid input format.

To Reproduce
Steps to reproduce the behavior:

1. Download the sample code as shown in the link
2. Update endpoint and key
3. Run it locally

Expected behavior
The code should run without error

Screenshots

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @dargilco @jhakulin @trangevi.

[dargilco]

@dunalduck0 can you share your endpoint URL, while redacting your resource name from it? Depending on how you deployed your gpt-4o model (Foundry Project endpoint or Azure OpenAI service endpoint), you may need to add some additional input arguments to the constructor of the ChatCompletionsClient as shown here https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_chat_completions_azure_openai.py

[dunalduck0]

I have two versions of code. When following sample_chat_completions_with_tools.py, I used these code to create the client

    endpoint = "https://mais-aoai.openai.azure.com/openai/deployments/gpt-4o"
    client = ChatCompletionsClient(endpoint=endpoint, credential=AzureKeyCredential(key))

In my project which uses AutoGen, I followed this example and here is my code

import os
from typing import Optional

from autogen_core.models import ModelFamily, ModelInfo
from autogen_ext.models.azure import AzureAIChatCompletionClient
from azure.ai.agents.aio import AgentsClient
from azure.identity.aio import DefaultAzureCredential


def get_autogen_model_client_by_model_name(endpoint: str, model_name: str) -> AzureAIChatCompletionClient:
    if not ModelFamily.is_openai(model_name):
        raise ValueError(f"Model name '{model_name}' is not a valid OpenAI model family, and it's not supported!")

    model_info = ModelInfo(
        vision=True,
        function_calling=True,
        family=model_name,
        json_output=False,  # it means json_output=None when calling the model
        structured_output=False,
        multiple_system_messages=False,
    )

    return AzureAIChatCompletionClient(
        endpoint=endpoint,
        credential=DefaultAzureCredential(),  # CodeQL [SM05139] Environment variable AZURE_TOKEN_CREDENTIALS is set to "prod" on the server.
        credential_scopes=["https://cognitiveservices.azure.com/.default"],  # type: ignore
        model_info=model_info,
    )

[dargilco]

@dunalduck0 please make sure you pass an "api_version" value into the Constructor of ChatCompletionClient, as shown in the sample I provided (https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-inference/samples/sample_chat_completions_azure_openai.py#L57). See comment about setting api-version at the top of the sample. I would try api_version="2025-04-01-preview" first to make sure you try with latest. Since in autogen you have an object of type AzureAIChatCompletionClient that you construct (and you don't construct ChatCompletionsClient directly) I don't know if you have the option to pass in additional keywords. I hope whatever you pass into AzureAIChatCompletionClient as keyword args gets passed directly into ChatCompletionsClient. But I would first try to run the sample that I shared directly, to see if that works.

Setting credential_scope does not apply to api key authentication. So you can remove it. It only applies to Entra ID authentication.

[github-actions]

Hi @dunalduck0. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[dunalduck0]

Thank you @dargilco for the suggestion. I tried several api_version but they didn't make a difference. But I think I finally cracked the mystery. All message types are based on Model, which filter out None data field. In other words, when AssistantMessage has no textual message (i.g content=None), the request to endpoint will send in a message without the data filed "content". Unfortunately, the endpoint didn't like it and errored out with "Invalid input format".

To fix it, I forced an empty field "content" to AssistantMessage by adding this line before Line 136 in sample_chat_completions_with_tools.py, and it fixed the crash

messages[2].content = ""

In summary, sending an AssistantMessage without data field "content" will crash as indicated in the first post.

Enforce an empty data field "content" will pass

I wish the error message could be more productive by saying "Invalid input format: missing 'content'"