Skip to content

Python: Improve agent integration tests #11475

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 10, 2025

Conversation

moonbox3
Copy link
Contributor

Motivation and Context

The current integration tests for agents work fine; however, if any errors are encountered due to server-side hiccups, the instant failure cause the whole test run to fail. This PR creates an agent_test_base that provides some retry logic, as well as a standardized way for agent test classes to use the underlying methods/protocol.

Description

Improve the agent integration tests.

  • The OpenAI Responses API is throwing 500s for file search right now. Adding xfail for the time being. Filed an issue on them.
  • Handle errors when calling for a response for the OpenAI Responses API -- throw a specific agent except or content exception, if received from Azure OpenAI.
  • Removed the OpenAIResponsesAgent invoke and invoke_stream web_search tests. We have coverage for get_responses. The web search is flaky sometimes.

Contribution Checklist

@moonbox3 moonbox3 requested a review from a team as a code owner April 10, 2025 04:39
@moonbox3 moonbox3 self-assigned this Apr 10, 2025
@moonbox3 moonbox3 added docs and tests Improvements or additions to documentation agents labels Apr 10, 2025
@markwallace-microsoft markwallace-microsoft added the python Pull requests for the Python Semantic Kernel label Apr 10, 2025
@markwallace-microsoft
Copy link
Member

markwallace-microsoft commented Apr 10, 2025

Python Unit Test Overview

Tests Skipped Failures Errors Time
3418 5 💤 0 ❌ 0 🔥 1m 41s ⏱️

@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue Apr 10, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Apr 10, 2025
@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue Apr 10, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Apr 10, 2025
@moonbox3 moonbox3 added this pull request to the merge queue Apr 10, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Apr 10, 2025
@moonbox3 moonbox3 enabled auto-merge April 10, 2025 10:58
@moonbox3 moonbox3 added this pull request to the merge queue Apr 10, 2025
@eavanvalkenburg eavanvalkenburg removed this pull request from the merge queue due to a manual request Apr 10, 2025
@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue Apr 10, 2025
Merged via the queue into microsoft:main with commit 81e89f0 Apr 10, 2025
28 checks passed
@github-project-automation github-project-automation bot moved this to Sprint: Done in Semantic Kernel Apr 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
agents docs and tests Improvements or additions to documentation python Pull requests for the Python Semantic Kernel
Projects
Status: Sprint: Done
Development

Successfully merging this pull request may close these issues.

3 participants