feat(amazonq): implement basic server for standard LSP completion API

[viktorsaws]

Problem

Q/codewhisperer does not support LSP standard "textDocument/completion" method #724

Solution

• Added simple experimental AmazonQCompletionServer implementation to support textDocument/completion protocol. More docs available in server/aws-lsp-codewhisperer/src/experimental/README.md.
• Added new testing app in app/aws-lsp-q-dev-completions, which includes AmazonQCompletionServer and device SSO token provider among other servers.

To test the server: fetch this PR or https://github.com/viktorsaws/language-servers/tree/experimental_lsp_completions branch and rebuild the project:

npm install && npm run package

Bundled language server is available at ./app/aws-lsp-q-dev-completions/build/amazon-q-completions-and-sso.js path.

License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[justinmk3]

given the layout of the rest of this project, the experimental/ directory seems unintuitive. how does it map to the existing pattern of the repo (which has client/ and language-server/ at the top level)?

[viktorsaws]

Yeah, I was not sure where to place it, given it's simple implementation. Moving to language-server would be more appropriate.

[justinmk3]

why doesn't this inherit from some other part of this repo? does each "server" have its own webpack config?

[viktorsaws]

Yeah, every package like this in app directory could be built using it's own config. Moving it out to common shared config would be an optimization, I decided to keep it more standalone for now.

[justinmk3]

why isn't this part of the existing codewhispererserver.ts ? can clients opt-in to the experiment by sending a settings flag on init?

[viktorsaws]

I wanted to avoid polluting already overloaded codewhispererserver.ts with this implementation. I'd look into toggling this completion behind a flag.

[justinmk3]

I wanted to avoid polluting already overloaded codewhispererserver.ts

I'm more worried about the pollution of numerous duplicate modules, configs, sub-packages, which interact in a way that must be imagined, because code-navigation tools can't make sense of all these sub-packages and cross-configs. That's much harder to cognitively consume than 1 package with lots of code (which can use shared base-classes/modules).

But maybe I missed what you meant :)

[kmile]

Can we just support both approaches at the same time? Clients that send us textdocument/completion requests will get answers with code-with-references filtered out. Clients that ask for a richer response including references, will get the enhanced response. They could use exactly the same handler without any duplication. We only need a small additional filter and transformation to make sure the params and suggestions match the right shape.

Are we worried clients might be sending both at the same time? They shouldn't, at least not to the same LSP server. While we could try to avoid this behavior with turning capabilities on/off dynamically, or adding some additional settings, it seems to me that the LSP server is responsible for answering client's requests, not determining whether the client's request is valid or not.

[kmile]

Would this (untested) code work instead?

        const onCompletionHandler = async (params: CompletionParams, token: CancellationToken) => {
            const inlineCompletionParams: InlineCompletionWithReferencesParams = {
                ...params,
                context: {
                    ...params.context,
                    triggerKind: InlineCompletionTriggerKind.Invoked,
                },
            }

            const inlineCompletionResult = await onInlineCompletionHandler(inlineCompletionParams, token)

            return {
                isIncomplete: false,
                items: inlineCompletionResult.items.filter(item => !item.references).map(item => {
                    return {
                        label: item.insertText.toString()
                    }
                }),
            }
        }

[viktorsaws]

My main concern with using same handler as for inlineSuggestionWithReferences is that it's stateful - we have notion of shared completion session shared between Client and Server. Session lifecycle has been already quite complicated, and adding extra logic to handle standard completion could overcomplicate things.

I think it would be better alternative to move this separate handler to CodeWhispererServer, and reuse shared handler and LSP configuration state, as Justin suggested, but I would avoid mixing 2 calls.

[justinmk3]

Discussion with Neovim plugin authors leads me to believe that "snippet" is most appropriate for multiline completions.

I'm not sure if that means we should usekind: CompletionItemKind.Snippet here, but at least it means we should set insertTextFormat = InsertTextFormat.Snippet.

[justinmk3]

Actually based on feedback from blink.nvim users, usekind: CompletionItemKind.Snippet seems to work better for them, while insertTextFormat: InsertTextFormat.Snippet risks activating snippet "placeholder" handling (snippet mode).

[viktorsaws]

I'm not sure if using Snippet type would be correct, given it expects that returned text has richer syntax, defined in LSP https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#snippet_syntax.

I don't have clear overview of how different IDE/LSP Client handle Snippets, I suspect it might cause issues in some clients.

[justinmk3]

Further feedback from Neovim users has indicated that Text or Reference work better, because completion engines often filter-out Snippet (and sometimes Text, too) by default.

[viktorsaws]

Need to implement support for cancellationTokens before merging