Python: Improve agent integration tests #11475
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
The current integration tests for agents work fine; however, if any errors are encountered due to server-side hiccups, the instant failure cause the whole test run to fail. This PR creates an
agent_test_base
that provides some retry logic, as well as a standardized way for agent test classes to use the underlying methods/protocol.Description
Improve the agent integration tests.
xfail
for the time being. Filed an issue on them.OpenAIResponsesAgent
invoke
andinvoke_stream
web_search tests. We have coverage forget_responses
. The web search is flaky sometimes.Contribution Checklist