AI Proxy - Custom upstream URL trigger "no host allowed while connecting to upstream"

[kdubuc]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

3.9

Current Behavior

When I want to setup AI proxy with a custom upstream url, I got an error

[lua] init.lua:1401:
balancer(): failed to set the current peer (address: api.scaleway.ai port: 443)
no host allowed while connecting to upstream
client: 172.17.0.1
server: kong
request: "POST /ai-proxy HTTP/1.1"
subrequest: "/kong_buffered_http"
host: "localhost:7110"
request_id: "58bd14166a68303a88699c54ea97ec62"

Expected Behavior

Kond should be running, and ready to proxy request to AI model

Steps To Reproduce

1- Setup kong like this

_format_version: "3.0"
services:
- url: http://0.0.0.0:8000
  plugins:
  - name: ai-proxy
    config:
      route_type: "llm/v1/chat"
      auth:
        header_name: Authorization
        header_value: "Bearer xxxxxx"
      model:
        provider: mistral
        name: mistral-nemo-instruct-2407
        options:
          mistral_format: openai
          upstream_url: "https://api.scaleway.ai/v1"
          max_tokens: 512
          temperature: 0.3
          top_p: 1.0

2- Launch kong

Anything else?

Bug possibly related : #12201, #14244

[kdubuc]

Update : DNS resolution is ok only if the service upstream url is equals to the model upstream url :

_format_version: "3.0"
services:
- - url: http://0.0.0.0:8000
+ - url: https://api.scaleway.ai
  plugins:
  - name: ai-proxy
    config:
      route_type: "llm/v1/chat"
      auth:
        header_name: Authorization
        header_value: "Bearer xxxxxx"
      model:
        provider: mistral
        name: mistral-nemo-instruct-2407
        options:
          mistral_format: openai
          upstream_url: "https://api.scaleway.ai/v1"
          max_tokens: 512
          temperature: 0.3
          top_p: 1.0

But the docs says :

Each of these AI-enabled routes must point to a null service. This service doesn’t need to map to any real upstream URL, it can point somewhere empty (for example, http://localhost:32000), because the AI Proxy plugin overwrites the upstream URL. This requirement will be removed in a later Kong revision.

https://docs.konghq.com/hub/kong-inc/ai-proxy/how-to/llm-provider-integration-guides/mistral/

[chobits]

I think this is an unimplemented feature in kong, although kong's doc said it could be done as what you have set in ur custom plugin. We alraedy have this ticket internally, but we did not implement it currently.