Python: Bug: Agent as the kernel function doesn't pass arguments

[HuskyDanny]

Please note the below is AI generated for better readability:

Okay, let's break down the problem you're encountering when trying to invoke your Nl2sqlAgent as a kernel function while passing custom arguments and a thread object.

Context

1. Agent Creation and Registration (@semantic_kernel_manager.py):

   • You correctly create a ChatCompletionAgent named Nl2sqlAgent.
   • You then add this agent instance as a plugin to the kernel using kernel.add_plugin(agent, "Agents").

   # src/wukong/semantic_kernel_manager/semantic_kernel_manager.py
   
   # ... (inside _create_nl2sql_agent)
       # Create agent with the configured template
       agent = ChatCompletionAgent(
           kernel=kernel,
           name="Nl2sqlAgent",  # The agent gets a name
           prompt_template_config=prompt_config,
           arguments=arguments,
           plugins=["TextMemoryPlugin", "PromptPlugin"],
       )
   
       # Add the agent instance itself as a plugin named "Agents"
       # This relies on the agent's model_post_init to create a kernel function
       kernel.add_plugin(agent, "Agents")

2. Automatic Kernel Function Creation (@agent.py):

   • The Agent base class uses model_post_init to automatically create a kernel function.
   • This function is decorated with @kernel_function and uses the agent's name (Nl2sqlAgent in your case) as the function name within the plugin.
   • Crucially, the signature of this automatically generated function (_as_kernel_function) only accepts messages and instructions_override. It then calls the agent's get_response method internally.

   # \home\allenpan\anaconda3\envs\wukongtest\lib\python3.10\site-packages\semantic_kernel\agents\agent.py
   
   def model_post_init(self, __context: Any) -> None:
       """Post initialization: create a kernel_function that calls this agent's get_response()."""
   
       # The function created has the agent's name
       @kernel_function(name=self.name, description=self.description or self.instructions)
       async def _as_kernel_function(
           # Note the parameters: only 'messages' and 'instructions_override'
           messages: Annotated[str | list[str], "The user messages for the agent."],
           instructions_override: Annotated[str | None, "Override agent instructions."] = None,
       ) -> Annotated[Any, "Agent response."]:
           """A Minimal universal function for all agents.
           # ... documentation ...
           """
           if isinstance(messages, str):
               messages = [messages]
   
           # It calls the agent's get_response method internally
           response_item = await self.get_response(
               messages=messages,  # type: ignore
               instructions_override=instructions_override if instructions_override else None,
           )
           return response_item.content
   
       setattr(self, "_as_kernel_function", _as_kernel_function)

3. Attempted Invocation (@nl2sql.py):

   • In your Nl2sql scenario, you try to invoke the agent using kernel.invoke.
   • You provide plugin_name="Agents" (where you registered the agent) and function_name="Nl2sqlAgent" (the name of the automatically generated function).
   • You attempt to pass arguments (containing query, chat_id, database_type, etc.) and a thread object to this invocation. You also pass messages=latest_question.

   # src/wukong/semantic_kernel_manager/scenario/nl2sql.py
   
   async def invoke(
       self,
       kernel: Kernel,
       agent: ChatCompletionAgent, # The actual agent instance
       messages: List[Message],
       database_type: str,
       chat_id: str,
   ):
       # ... setup arguments and agent_thread ...
   
       arguments = KernelArguments(
           query=latest_question,
           chat_id=chat_id,
           database_type=database_type,
           entity_container=entity_container,
           chart_container=chart_container,
       )
   
       # Attempting to invoke the agent *as a kernel function*
       response = await kernel.invoke(
           plugin_name="Agents",          # Correct plugin name
           function_name="Nl2sqlAgent", # Correct function name (agent's name)
           arguments=arguments,       # PROBLEM: Target function doesn't accept arbitrary arguments
           thread=agent_thread,       # PROBLEM: Target function doesn't accept 'thread'
           messages=latest_question,    # PROBLEM: Target function expects message content directly in 'messages', not via invoke's 'messages' kwarg.
       )
       # ... rest of the code ...

Problem Description

The core issue is a mismatch between the parameters you are trying to pass via kernel.invoke and the actual signature of the underlying kernel function (_as_kernel_function) that gets called.

1. Limited Function Signature: The automatically generated kernel function for the agent (Agent._as_kernel_function) is designed for simple invocation and only accepts messages (the user input) and an optional instructions_override.
2. Unsupported Parameters: When you call kernel.invoke(..., arguments=arguments, thread=agent_thread, messages=...), you are passing parameters (arguments object, thread object) that the target function (_as_kernel_function) is not defined to accept. The kernel's invocation mechanism doesn't automatically know how to map these extra parameters to the agent's internal methods (get_response or invoke) when the agent is called as a kernel function.
3. Incorrect messages Usage: The messages parameter in the kernel.invoke call in nl2sql.py is also likely incorrect. The _as_kernel_function expects the actual message content to be passed to its messages parameter, which usually happens through the primary input mechanism of kernel.invoke (often mapping the input argument or specific arguments defined in the function signature), not through a separate messages keyword argument to kernel.invoke itself.

In essence, you are trying to use the kernel's function invocation system (kernel.invoke) to pass complex objects (thread, KernelArguments) to a function that was automatically generated with a much simpler signature, only intended to handle basic message input. This direct invocation path doesn't support passing the necessary thread context or the detailed KernelArguments required by your Nl2sqlAgent's logic when interacting via its native methods.

------ User question:
So the final arugment I have is only having the messages as the parameters:

My question is I want to unify the way we call either plugin, agent or function through kernel in my project management, is my attempt aligned with the practices in SK? If so, can we let the agent accept the reserved arguments? Otherwise, my agent cannot continue the previous context.

[sahiladit]

but the kernel function can take multiple parameters , is this issue related with function calling ?

[TaoChenOSU]

Hi @HuskyDanny,

Thank you for creating the issue!

Looks like you are trying to invoke an agent as a kernel function via the kernel, where the function only accepts one argument which is messages.

In SK, you need to provide the arguments as key-value pairs through an KernelArguments object. For example: https://github.com/microsoft/semantic-kernel/blob/main/python/samples/getting_started/04-kernel-arguments-chat.ipynb

In the case of the agent as a function, you only need the messages argument and anything else will be ignored.

At this point, when you are using agent as a function, it's really meant to be a fire-and-forget process.

Could you share more about your use case on why you need to retain the agent context between calls when it's used as a function?

[HuskyDanny]

but the kernel function can take multiple parameters , is this issue related with function calling ?

I would say it is related to kernel function use for the agent, even though the kernel arguments can have multiple, but the kernel function parameters for agent doens't include the arguments keyword.

[HuskyDanny]

Hi @HuskyDanny,

Thank you for creating the issue!

Looks like you are trying to invoke an agent as a kernel function via the kernel, where the function only accepts one argument which is messages.

In SK, you need to provide the arguments as key-value pairs through an KernelArguments object. For example: https://github.com/microsoft/semantic-kernel/blob/main/python/samples/getting_started/04-kernel-arguments-chat.ipynb

In the case of the agent as a function, you only need the messages argument and anything else will be ignored.

At this point, when you are using agent as a function, it's really meant to be a fire-and-forget process.

Could you share more about your use case on why you need to retain the agent context between calls when it's used as a function?

Hi @TaoChenOSU , yes your observation is right on. The reason why I want to do this is mainly because I want to manage the agent the same way as the plugin and functions. In my application, I invoke SK capability through kernel, so I want to stick to this pattern. Right now I am seeing the agent as an object passing arounds, I feel it is more clean we manage all things through kernel & name. And my application is more like stateless server so it wants to take extra context for each request, so that why I need to pass the arguments.

Speaking of fire & forget, I think even with the arguments, the pattern is still fire & forget just with extra context from outside? May I know why it is designed in this way?