Skip to content

telemetry(amazonq): toolUseSuggested #1014

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 16, 2025
Merged

Conversation

chungjac
Copy link
Contributor

Problem

  • Want to see the distribution of tools that LLM either used directly (no acceptance needed) or suggested (user acceptance required)

Solution

  • Add metric toolUseSuggested

Notes

  • This only takes into account what tool the LLM uses/suggests. This does not take into account whether that tool was actually accepted/rejected.
    • This is because a user can accept/reject at a later time, or even ignore. For example, a generated shell command can be rejected 30 minutes later or dozens of messages later in the conversation.
    • The goal is to possibly streamline this, likely with the amazonq_interactWithAgenticChat metric
  • For now, we want info in the distribution of tools that LLM uses/suggests

License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@chungjac chungjac requested a review from a team as a code owner April 16, 2025 19:29
@justinmk3 justinmk3 merged commit d1cf36e into aws:main Apr 16, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants