WatsonxAiChatModel is using wrong APIs

[FabioAng71]

Bug description
While using WatsonxAiChatModel I see the request is sent to the following endpoint:
/ml/v1/text/generation?version=2023-05-29
https://cloud.ibm.com/apidocs/watsonx-ai#text-generation

while I expect it should use the following one:
/ml/v1/text/chat?version=2023-05-29
https://cloud.ibm.com/apidocs/watsonx-ai#text-chat

and consequently all the parameters associated with the second API (such as tools)

watsonx-ai.text-chat is very close to opeai.completions

Environment
Spring AI 1.0.0-M6, Java version 21

Steps to reproduce
add the following fragment to the pom.xml

        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-watsonx-ai-spring-boot-starter</artifactId>
        </dependency>

Expected behavior
I think that the WatsonxAiChatModel should be changed and be more close to the OpenAiChatModel

[apappascs]

Hi @FabioAng71,

Both the /ml/v1/text/generation and /ml/v1/text/chat endpoints are valid Watsonx AI APIs. The current WatsonxAiChatModel in Spring AI is designed to use the text generation API /ml/v1/text/generation as you mentioned already. WatsonxAi Connection Properties

If your use case requires OpenAI API-compatible behavior the recommended approach is to use the spring-ai-openai-spring-boot-starter instead. You can configure it to point to Watsonx AI’s text-chat API by setting:

spring.ai.openai.chat.base-url=https://your-watsonx-instance-url
spring.ai.openai.chat.completions-path=/ml/v1/text/chat?version=2023-05-29

same approach as Deepseek and Groq

Let me know if this works for you or if you have any further questions!?

[FabioAng71]

Hi @apappascs.

I followed you seggustion but it doesn't work for several reasons:
1) spring.ai.openai.chat.base-url doesn't override for the spring.ai.openai.base-url, so I used spring.ai.openai.base-url

2. the autentication token for watsonx.ai is generated from the IAM Token using the following api:
   https://cloud.ibm.com/docs/account?topic=account-iamtoken_from_apikey&code=curl#iamtoken
   so we need to generate a Bearer using any watsonx.ai endpoint

3. in the openai chat endpoint (https://platform.openai.com/docs/guides/text-generation) the model is specified in the 'model' field while in the watsonx.ai chat endpoint (https://cloud.ibm.com/apidocs/watsonx-ai#text-chat) the model is specified in the 'model_id' field

4. in the openai chat endpoint project_id is optional and it is in the header (https://platform.openai.com/docs/api-reference/authentication) while in the watsonx.ai chat endpoint the project_id is mandatory and it is in the body as 'project_id' field

5. most important, if WatsonxAiChatModeluse https://cloud.ibm.com/apidocs/watsonx-ai#text-generation it is not possibile to use Tools (https://docs.spring.io/spring-ai/reference/api/tools.html )and Advisors (https://docs.spring.io/spring-ai/reference/api/advisors.html) beacuse that api doesn't support them.

The right solution should be to re-implement WatsonxAiChatModel using https://cloud.ibm.com/apidocs/watsonx-ai#text-chat to fully satisfy the ChatClient interface (https://docs.spring.io/spring-ai/reference/api/chatclient.html).

Let me know your thoughts

[apappascs]

@FabioAng71 Thank you for looking into this. Based on your findings, it seems that it's not compatible with the OpenAI API. However, to be honest, I'm not sure which of these endpoints is preferable or why.

I couldn't find any documentation comparing the two:

• Text Generation API
• Chat API

[FabioAng71]

@apappascs
If you look at WatsonxAi Chat API you will see it is very close to OpenAi Chat API apart the differences I mentioned at point 2) 3) 4) of my previous post. The request has the same structure as for the response.

On the other hand WatsonxAi Text Generation API is just a simple text request and a simple text response.

Looking into IBM documentaion for Text Generation API it is cleary stated that the correct API is WatsonxAi Chat API
For chat use cases, use the Chat API. See Adding generative chat function to your applications with the chat API.