feat(amazonq): Move inline completion to Flare #7480
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
) ## Problem We want to enable inline suggestions from flare on this branch ## Solution - enable it - leave the toggle setting so its easy to turn off/on to compare behaviours --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
…7233) ## Problem the isSuggestionsEnable setting isn't respected ## Solution if someone disabled suggestions via a setting or from the status bar then don't return anything for automatic triggers --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
## Problem - we're moving to inline via flare ## Solution - deprecate amazon q inline ## Notes #### deprecation steps: 1. removed recommendation handler and recommendation service and all regular dependencies, since those are the bulk of inline suggestions 2. removed command registrations for onacceptance 3. removed tests that are no longer relevant to the vscode implementation since they are already in flare 4. modified the lineAnnotationController and activeStateControllers to comment out any missing imports, since those still need to be there in the new implementation 5. removed pagination calls, since those are now done through flare 6. remove keystroke handler, since that's now done by the vscode api 7. removed old cloud9 compatability code for inline #### Future PRs: - Refactor lineAnnoationController and activeStateControllers - re-implement the `aws.amazonq.refreshAnnotation` command - fix the inline e2e tests, since now they will fully go through flare instead of the recommendation handler - fix any unit tests that are now failing - updating the status bar when a request is in progress --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
## Problem deprecation for inline ## Solution deprecates: 1. supplemental context + its tests, since that was only passed into codewhisperer 2. codewhisperer coverage tracker, since thats [already in flare](https://github.com/aws/language-servers/blob/main/server/aws-lsp-codewhisperer/src/language-server/inline-completion/codePercentage.ts) 3. [classifier trigger](https://github.com/aws/language-servers/blob/main/server/aws-lsp-codewhisperer/src/language-server/inline-completion/auto-trigger/coefficients.json), since that's in flare 4. inlineCompletionProvider is no longer used and should have been in my other PR 5. editorContext utils, since that was used with supplemental context --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
…gh Flare. (#7235) ## Problem This is the initial set of work required to get inline chat running through flare. ## Solution - **add a feature flag for inline chat**: this allows testing of the two implementations side-by-side by flipping the feature flag. - **move general utils out of chat**: stuff like encryption and editorState can all be reused. - **render full diff response from inline chat**: this does not include progress updates from the language server. ## Testing and Verification https://github.com/user-attachments/assets/0dff58b7-40f7-487d-9f9e-d58610201041 ## Future Work / Next Steps - ensure telemetry is still being emitted. - add tests for new flow. (there aren't any for the existing one) - handle partial events from the language server. ## Known Bugs - selecting part of a line will cause the text to insert mid-line - running inline-chat without a selection causes the entire file to be copied (This is in JB, Eclipse Prod, but IMO it makes the feature unusable). --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
…gh Flare. (#7235) ## Problem This is the initial set of work required to get inline chat running through flare. ## Solution - **add a feature flag for inline chat**: this allows testing of the two implementations side-by-side by flipping the feature flag. - **move general utils out of chat**: stuff like encryption and editorState can all be reused. - **render full diff response from inline chat**: this does not include progress updates from the language server. ## Testing and Verification https://github.com/user-attachments/assets/0dff58b7-40f7-487d-9f9e-d58610201041 ## Future Work / Next Steps - ensure telemetry is still being emitted. - add tests for new flow. (there aren't any for the existing one) - handle partial events from the language server. ## Known Bugs - selecting part of a line will cause the text to insert mid-line - running inline-chat without a selection causes the entire file to be copied (This is in JB, Eclipse Prod, but IMO it makes the feature unusable). --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
## Problem Follow up to #7235 (comment). ## Solution - extract all decrypting and encrypting logic to a single location. - add a simple test for this logic (encrypt and decrypt are inverses). - refactor existing implementations. ## Verification Used agentic chat with some tools, as well as inline chat and didn't notice a difference. If encryption were broken, I would expect this to fail immediately. --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
…n metric (#7244) ## Problem vscode keeps track of `codewhispererImportRecommendationEnabled` inside of the `codewhisperer_serviceInvocation` event but flare doesn't ## Solution add it before emitting telemetry, since this is purely a client side feature --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
## Problem - "Amazon Q is generating ..." does not show with the lsp mode ## Solution - re-add the line tracker with tests - re-implement the activeState tracker ## Notes In master the active state tracker decides whether or not to show "Amazon Q is generating ..." by the following: - When a change is made, the auto trigger decides whether or not to start a recommendation request. When a recommendation requests eventually starts, an event is sent to the active state tracker to tell it to start showing the "Amazon Q is generating ..." message. When the first recommendation starts loading and the results are shown to the user another event is sent telling it to hide the message. It de-bounces this message showing every 1000ms so that the message is not constantly toggling on/off In this implementation its slightly different: - VSCode decides when to trigger the inline completion through their inline completion provider. From here we show the "Amazon Q is generating ... " message until the first recommendation is received from the language server and shown to the user. It still de-bounces this message every 1000ms so that users aren't shown it too often --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
This new name more accurately represents what this class is for. It is just a util to create the "Amazon Q generating" inline message. - Class is renamed - File is renamed and move out of the "stateTracker" folder --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. Signed-off-by: nkomonen-amazon <[email protected]>
I saw while testing that the "Amazon Q is generating..." got stuck at some point. I think this fix should avoid that Signed-off-by: nkomonen-amazon <[email protected]>
This was a regression that appeared while doing the port to flare, now we will show the spinning symbol when generating a suggestion. Additionally the file was more appropriately named since it now only has the status bar related code. Signed-off-by: nkomonen-amazon <[email protected]>
## Problem the tutorial trackers aren't implemented when using the language server ## Solution - re-add the inlineLineAnnotationController (inlineChatTutorialAnnotation) for adding hints with inline chat - re-add the lineAnnotationController (inlineTutorialAnnotation) for adding the inline suggestions tutorial ## Notes in a future PR I'll fully deprecate the old trackers --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
There was a regression that appeared while doing the port to flare where the status bar would not show the spinning icon when generating a suggestion. Now we will show the spinning symbol when generating a suggestion. Additionally the file was more appropriately named since it now only has the status bar related code.
…7234) ## Problem the only metric it looks like we're missing for inline on the vscode side is `codewhisperer_clientComponentLatency` ## Solution codewhisperer_clientComponentLatency uses a very similar implementation as before the only differences are: 1. codewhispererCredentialFetchingLatency is no longer relevant because the token is always injected into the language server and it doesn't need to build the client on demand like before. - This causes the preprocessing latency to decrease, because that used to contain the time it takes to fetch the credentials 2. postProcessing latency is way lower because once we get the result vscode instantly displays it -- we no longer have control of that example metric now: ``` 2025-05-06 11:53:59.858 [debug] telemetry: codewhisperer_clientComponentLatency { Metadata: { codewhispererAllCompletionsLatency: '792.7122090000048', codewhispererCompletionType: 'Line', codewhispererCredentialFetchingLatency: '0', codewhispererCustomizationArn: 'arn:aws:codewhisperer:us-east-1:12345678910:customization/AAAAAAAAAA', codewhispererEndToEndLatency: '792.682249999998', codewhispererFirstCompletionLatency: '792.6440000000002', codewhispererLanguage: 'java', codewhispererPostprocessingLatency: '0.019500000002153683', codewhispererPreprocessingLatency: '0.007166999996115919', codewhispererRequestId: 'XXXXXXXXXXXXXXXXXXXXXXXXXXX', codewhispererTriggerType: 'AutoTrigger', credentialStartUrl: 'https://XXXXX.XXXXX.com/start', awsAccount: 'not-set', awsRegion: 'us-east-1' }, Value: 1, Unit: 'None', Passive: true } ``` --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
## Problem
When typing a decent amount of text in quick succession, the language
server will get throttled in its requests to the backend. This is
because we send a request for recommendations on every key stroke,
causing the language server to make a request on each key stroke. This
is rightfully getting throttled by the backend.
## Solution
- The ideal behavior is that we only make a request to the language
server, and thus to the backend, when typing stops. Therefore, this is
an ideal use case for `debounce`. However, we need to extend debounce
slightly outlined below.
- Apply `debounce` to the recommendations such that we wait 20 ms after
typing stops before fetching the results.
- By applying this at the recommendation level, none of the inline
latency metrics are affected.
### Debounce Changes
-Let f be some debounced function that takes a string argument, our
current debounce does the following:
```
f('a') f('ab') f('abc') (pause for debounce delay) -> f would be called with 'a'
```
The issue is is that for suggestions, this means the language server
request will be made with stale context (i.e. not including our most
recent content). What we want instead is for the case above to call f
with `'abc'` and not with `'a'` or `'ab'`.
We can accomplish this by adding a flag to `debounce` allowing us to
choose whether we call it with the first args of the debounce interval
(default, and 'a' in the example above), or the most recent args ('abc'
in the example above).
## Verification
- I did not notice the added latency when testing inline. However it
does seem slower than prod, with and without this change.
- I was not able to get a throttling exception.
---
- Treat all work as PUBLIC. Private `feature/x` branches will not be
squash-merged at release time.
- Your code changes must meet the guidelines in
[CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines).
- License: I confirm that my contribution is made under the terms of the
Apache 2.0 license.
## Problem Adding logging statements to the top level is extremely noisy, since that part still triggers on each key stroke. This could also improve latency since any computation at the top-level will be redone on each keystroke. ## Solution - move the debounce up a layer. - remove outdated tests, since the debounce util already has tests for all this logic. - added new tests for new behavior. ## Verification - I tested this side-by-side with previous version and didn't notice a difference. --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
## Automatic merge failed - Resolve conflicts and push to this PR branch. - **Do not squash-merge** this PR. Use the "Create a merge commit" option to do a regular merge. ## Command line hint To perform the merge from the command line, you could do something like the following (where "origin" is the name of the remote in your local git repo): ``` git stash git fetch --all git checkout origin/feature/flare-inline git merge origin/master git commit git push origin HEAD:refs/heads/autoMerge/feature/flare-inline ``` --------- Signed-off-by: nkomonen-amazon <[email protected]> Co-authored-by: aws-toolkit-automation <> Co-authored-by: Tom Zu <[email protected]> Co-authored-by: Nikolas Komonen <[email protected]> Co-authored-by: Will Lo <[email protected]> Co-authored-by: Tai Lai <[email protected]> Co-authored-by: nkomonen-amazon <[email protected]> Co-authored-by: Josh Pinkney <[email protected]> Co-authored-by: Josh Pinkney <[email protected]> Co-authored-by: Adam Khamis <[email protected]> Co-authored-by: Na Yue <[email protected]> Co-authored-by: Laxman Reddy <[email protected]> Co-authored-by: Hweinstock <[email protected]> Co-authored-by: Jiatong Li <[email protected]> Co-authored-by: Avi Alpert <[email protected]> Co-authored-by: Brad Skaggs <[email protected]> Co-authored-by: hkobew <[email protected]>
## Automatic merge failed - Resolve conflicts and push to this PR branch. - **Do not squash-merge** this PR. Use the "Create a merge commit" option to do a regular merge. ## Command line hint To perform the merge from the command line, you could do something like the following (where "origin" is the name of the remote in your local git repo): ``` git stash git fetch --all git checkout origin/feature/flare-inline git merge origin/master git commit git push origin HEAD:refs/heads/autoMerge/feature/flare-inline ``` --------- Signed-off-by: nkomonen-amazon <[email protected]> Co-authored-by: aws-toolkit-automation <> Co-authored-by: Tom Zu <[email protected]> Co-authored-by: Will Lo <[email protected]> Co-authored-by: Tai Lai <[email protected]> Co-authored-by: Josh Pinkney <[email protected]> Co-authored-by: Nikolas Komonen <[email protected]> Co-authored-by: Adam Khamis <[email protected]> Co-authored-by: Na Yue <[email protected]> Co-authored-by: Laxman Reddy <[email protected]> Co-authored-by: Jiatong Li <[email protected]> Co-authored-by: Avi Alpert <[email protected]> Co-authored-by: Brad Skaggs <[email protected]> Co-authored-by: Zoe Lin <[email protected]> Co-authored-by: chungjac <[email protected]> Co-authored-by: Lei Gao <[email protected]> Co-authored-by: hkobew <[email protected]>
## Problem There are two edge cases discovered during testing that warrant some tests. 1. the language server will respond without a range, causing the result to not be rendered. 2. the language server COULD respond with a `StringValue` instead of string (currently doesn't, but valid in the type contract). ## Solution - add tests for these cases, and do minor refactoring to make this easier. --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
## Problem References are not added to the reference log at the bottom. The root cause is that the reference tracker log is expecting CW shaped references, but we are passing "Flare" shaped ones. ## Solution - Update the reference log to standardize the shape of incoming references. ## Testing and Verification - added unit test. - Verified e2e: https://github.com/user-attachments/assets/620f6f3d-1205-405e-96a2-322d5e8220c4 - --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
… invoke (#7318) ## Problem When manually invoking inline suggestions, it is possible for the backend to still not reply with results. Since the user took an action to invoke this, we should let them know when it fails. ## Solution - add a message with same timeout as original implementation. ## Testing and Verification - added unit test. - E2E flow demo (notice it doesn't trigger on automatic failures, but does on manual invoke failures): https://github.com/user-attachments/assets/7cf34178-e91e-4d38-8875-1259c5552a53 --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Co-authored-by: Josh Pinkney <[email protected]>
## Automatic merge failed - Resolve conflicts and push to this PR branch. - **Do not squash-merge** this PR. Use the "Create a merge commit" option to do a regular merge. ## Command line hint To perform the merge from the command line, you could do something like the following (where "origin" is the name of the remote in your local git repo): ``` git stash git fetch --all git checkout origin/feature/flare-inline git merge origin/master git commit git push origin HEAD:refs/heads/autoMerge/feature/flare-inline ``` --------- Signed-off-by: nkomonen-amazon <[email protected]> Co-authored-by: aws-toolkit-automation <> Co-authored-by: Tom Zu <[email protected]> Co-authored-by: Will Lo <[email protected]> Co-authored-by: Tai Lai <[email protected]> Co-authored-by: Josh Pinkney <[email protected]> Co-authored-by: Nikolas Komonen <[email protected]> Co-authored-by: Adam Khamis <[email protected]> Co-authored-by: Na Yue <[email protected]> Co-authored-by: Laxman Reddy <[email protected]> Co-authored-by: Jiatong Li <[email protected]> Co-authored-by: Avi Alpert <[email protected]> Co-authored-by: Brad Skaggs <[email protected]> Co-authored-by: Zoe Lin <[email protected]> Co-authored-by: chungjac <[email protected]> Co-authored-by: Lei Gao <[email protected]> Co-authored-by: Hweinstock <[email protected]> Co-authored-by: invictus <[email protected]> Co-authored-by: hkobew <[email protected]>
…nds. (#7323) ## Problem The current implementation overwrites the `editor.action.inlineSuggest.showPrevious` and `editor.action.inlineSuggest.showNext` commands to swap out the inlineProvider. These custom overwrites cause a few issues: - forces us to maintain a ton of state [leads to bugs, some of which were observed in bug bash]. - causes the suggestions to "refresh" when going through them since we are re-instantiating the provider on "scrolls". - doesn't natively support "accept word"/"accept line". ## Solution - Remove the overwrite, remove reliance of `session` state. - Simplify implementation to use native VSC commands. ## Fixed Bugs - Scrolling through suggestions now works and doesn't "refresh". - Accept Word/Accept Line now work properly. - Significantly faster to scroll through suggestions. - Inline Reference Codelense no longer flashes when scrolling through suggestions. ## Tests and Verification: Testing this change it feels significantly 'smoother'. https://github.com/user-attachments/assets/631ee822-3cab-46b5-a07c-b3d4b8eb7d3b ## Future Work - with some more aggressive refactoring, and once telemetry is mostly in Flare, we can completely delete the sessionManager. - revisit reference codelense, and see if its the right choice. Maybe a message instead? --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
builds off #7323 ## Problem - Code reference codelense doesn't open the code reference panel, it opens problems. - Sometimes the codelense never goes away. (Note: both of these bugs are on prod) ## Solution - Hide the codelense after 5 seconds. - Properly link the codelense to code reference log panel. ## Verification https://github.com/user-attachments/assets/3a231ab4-1e25-4746-864a-d66f5e852686 --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
deps(amazonq): merge master into feature/flare-inline
deps(amazonq): remove unused code in Flare inline migration
|
leigaol
commented
Jun 12, 2025
## Problem ## Solution --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
https://github.com/aws/aws-toolkit-vscode/pull/7450/files is too convoluted, hard to review and we also found a couple of regressions.
Solution
Move inline completion to Flare first in this PR. Then we plan on how to move auth to Flare. After this merge, https://github.com/aws/aws-toolkit-vscode/pull/7450/files will be a lot easier to debug and review.
feature/xbranches will not be squash-merged at release time.