Skip to content

Commit d1cf36e

Browse files
authored
telemetry(amazonq): toolUseSuggested #1014
## Problem - Want to see the distribution of tools that LLM either used directly (no acceptance needed) or suggested (user acceptance required) ## Solution - Add metric `toolUseSuggested` ## Notes - This only takes into account what tool the LLM uses/suggests. This does not take into account whether that tool was actually accepted/rejected. - This is because a user can accept/reject at a later time, or even ignore. For example, a generated shell command can be rejected 30 minutes later or dozens of messages later in the conversation. - The goal is to possibly streamline this, likely with the `amazonq_interactWithAgenticChat` metric - For now, we want info in the distribution of tools that LLM uses/suggests
1 parent fa4916e commit d1cf36e

File tree

1 file changed

+33
-1
lines changed

1 file changed

+33
-1
lines changed

telemetry/definitions/commonDefinitions.json

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1022,7 +1022,7 @@
10221022
"Assign",
10231023
"Transform",
10241024
"AgenticChat",
1025-
"AgenticChatWithToolUse"
1025+
"AgenticChatWithToolUse"
10261026
],
10271027
"description": "Identifies the type of conversation"
10281028
},
@@ -1269,6 +1269,16 @@
12691269
"type": "int",
12701270
"description": "Truncated length of workspace context"
12711271
},
1272+
{
1273+
"name": "cwsprToolName",
1274+
"type": "string",
1275+
"description": "Client side tool name (ex: fsWrite, executeBash)"
1276+
},
1277+
{
1278+
"name": "cwsprToolUseId",
1279+
"type": "string",
1280+
"description": "The id for when a client side tool is used."
1281+
},
12721282
{
12731283
"name": "databaseCredentials",
12741284
"type": "string",
@@ -2820,6 +2830,28 @@
28202830
}
28212831
]
28222832
},
2833+
{
2834+
"name": "amazonq_toolUseSuggested",
2835+
"description": "When the LLM either used a tool directly (no acceptance needed) or suggested a tool (user acceptance required)",
2836+
"metadata": [
2837+
{
2838+
"type": "credentialStartUrl",
2839+
"required": false
2840+
},
2841+
{
2842+
"type": "cwsprChatConversationId"
2843+
},
2844+
{
2845+
"type": "cwsprChatConversationType"
2846+
},
2847+
{
2848+
"type": "cwsprToolName"
2849+
},
2850+
{
2851+
"type": "cwsprToolUseId"
2852+
}
2853+
]
2854+
},
28232855
{
28242856
"name": "amazonq_unitTestGeneration",
28252857
"description": "Client side invocation of the Amazon Q Unit Test Generation with build and execute loop",

0 commit comments

Comments
 (0)