Integrated a new c/c++ language server: cquery

[saibing]

I'm try a new c/c++ language server: cquery

CQuery is a Low-latency language server for large C++ code-bases, powered by libclang. It is much more powerful than clangd.

But I ran into some trouble that cquery need add some initializationOptions:

{
        "initializationOptions": {
                "cacheDirectory": "/tmp/cquery",
		"resourceDirectory": "yourCQueryDirectory/clang_resource_dir"
        }
}

How to use cquery with LanguageClient-neovim

May be Oni need to support config language server's initializationOptions?

[bryphe]

Thanks for logging the issue, @saibing !

Yes, unfortunately our configuration doesn't support this currently... But it would make sense to have something like this, ie:

"language.cpp.languageServer.initializationOptions": {
                 "cacheDirectory": "/tmp/cquery",
		"resourceDirectory": "yourCQueryDirectory/clang_resource_dir"
}

Will use this to track setting the initializationOptions.

[bryphe]

A good place to start looking is here:

oni/browser/src/Services/Language/LanguageConfiguration.ts

Line 99 in 45f51c5

const languageClient = new LanguageClient(language, new LanguageClientProcess(serverRunOptions, initializationOptions, configuration))

Which is the code responsible for taking the configuration settings and handing them off to the language client.

[MaskRay]

FYI
See https://github.com/autozimu/LanguageClient-neovim/wiki/cquery
"resourceDirectory" is auto detected and can be removed now.

[MaskRay]

I'd like to ask how you would like to detect the project root. When there are subpackages, the root compile_commands.json contains entries for subproject files and you do not want files in subprojects to be associated with subprojects, but rather the root project.

proj
 compile_commands.json
 .git
 third_party
  proj0
   .git
   a.cc       // you want this file to be associated with the root project
  proj1
   .git

Sorry for pasting the link of the Emacs lsp-mode discussion thread. But they have the same issue
emacs-lsp/lsp-mode#293 and I think it is beneficial to draw on the wisdom of the masses

[saibing]

@MaskRay
Thank you very much

Now I can solve this problem by add follow config into config.tsx

    "language.cpp.languageServer.rootFiles": ["compile_commands.json", ".cquery"],
    "language.cpp.languageServer.arguments": ["--init", '{"cacheDirectory": "/tmp/oni_cquery"}'],

[chenlijun99]

@saibing Sorry, is cquery working for you with Oni?
I've tried but it doesn't get any suggestions, nor any other lsp features seem to work.

By inspecting the debugger console, there is written

No grammar found for language: cpp
u @ bundle.js:1

[MaskRay]

For cquery users, you may consider ccls. It differs a lot from cquery now (5000 lines fewer, fewer third-party dependencies, IMO simpler but better pipeline, slightly better memory index). You can find some rationale for the fork at emacs-lsp/emacs-ccls#1

https://github.com/MaskRay/ccls/wiki/Getting-started

[MagBad]

@Free-Easy
I'm having a similar issue with the configuration of cquery for oni. Currently this is my config for
for cquery:
"language.cpp.languageServer.command": "/usr/bin/cquery", "language.cpp.languageServer.rootFiles": ["compile_commands.json"], "language.cpp.languageServer.arguments": ["--init", '{cacheDirectory": "/home/magb/.cache/cquery"}']

I'm receiving the same error in the console as @Free-Easy. @saibing could you provide your configuration? And maybe a mention of cquery in the wiki would be nice.

[paulie-g]

Indeed, a working cquery/ccls config would be great to see - can't seem to get either to work properly.

[feltech]

I just gave ccls (fork of cquery) a go, after clangd failed miserably, giving lots of spurious errors (despite the c++ command that clangd was supposedly running, evidenced from Oni's STDERR log, giving no errors when run independently...).

ccls, on the other hand, is working beautifully. My Oni configuration is as below (with the binary path altered for brevity)

    "language.cpp.languageServer.command": "/path/to/ccls",
    "language.cpp.languageServer.arguments": [
        "--log-file=/tmp/cc.log"
    ],
    "language.cpp.languageServer.rootFiles": [
        "compile_commands.json"
    ],

Initially it didn't work, so I added the --log-file. That pointed me to a failure to find compile_commands.json, because it was looking in my src directory (the location of the first source file I opened) rather than the project root. The logs also hint at a compilationDatabaseDirectory that can be passed as part of the --init option. I wasn't keen on setting that to a fixed directory, since I would have to change Oni config when switching projects. Then I noticed other language servers are configured with a language.<lang>.languageServer.rootFiles option in Oni. I added that line, and voila! ccls working nicely.

Oh, there was one other custom thing I did. I modified the build of ccls to use clang 7 (the current master defaults to clang 6), but I doubt that would make much difference to Oni integration.

[badosu]

@feltech Would you mind adding your lsp config to https://github.com/onivim/oni/wiki/Language-Support#cc?

[feltech]

OK, done. I didn't realise we could edit the wiki so easily! I'll have to think of some more useful stuff to add...

[feltech]

Also, the No grammar found for language: cpp is a separate issue. Oni does not come bundled with a TextMate grammar file for c++. I don't actually know what these files are supposed to do(?), but the warning was bothering me. So I lifted one from Theia-IDE, put it in oni/extensions/cpp/syntaxes (to fit with the existing grammar files), and referenced it in my config with

"language.cpp.textMateGrammar": "/path/to/oni/extensions/cpp/syntaxes/cpp.tmLanguage.json"

That stopped the warning in the console, at least.

[akinsho]

@feltech re. the grammar issue the text mate grammars allow oni use textmate syntax highlighting for different file types which tend to be richer than vim ones, a good source tends to be vscode or atom but any good one will do, you should see better highlighting with that change? also we are actually trying to amass as many bundled grammars as possible as they tend to be quite small files and massively improve the appearance of different file types

[chenlijun99]

@Akin909 Does this mean that Oni has its own engine for syntax highlighting, instead of the (slow) engine provided by Vim/Neovim? Does it run in another thread?

[akinsho]

@Free-Easy we use textmate grammars to parse and tokenize a file this happens in js (on the main thread atm although were looking at offloading this using webworkers but thats still a WIP see things like #2617), I don't actually know how the tokenization (via vscode-textmate) compares to vim's regex engine, but we then use nvim's msgpack api to tell nvim to highlight the tokens we've parsed, all in all I'm not sure how it all compares, but on oni's side of things we definitely have room to optimise the highlighting.

As a random side note/ tangent I personally am quite keen on looking into adding an option to use treesitter (created by one of atoms developers and now being trialled in atom) instead of textmate for syntax highlighting, theres a couple of good talks by the author on how it differs but the long and short is it seems to provide more relevant and consistent highlighting than textmate but is still fairly new

[feltech]

Some more on ccls/cquery with Oni... I was a bit premature in my optimism. In particular, completions are a pain. I gave up on ccls and moved to cquery to see if that was any better, but they seem to suffer the same problems. In particular I have discovered (so far):

• Critically: Oni's language server integration completely hangs when requesting completionItem/resolve from cquery. The server simply does not respond to the request, so the promise is never fulfilled and the request queue just keeps building up forever.
  The fix is simple, however. Oni already queries for the language server "capabilities", and cquery responds saying it doesn't support completionItem/resolve (completionProvider.resolveProvider is false), but Oni ignores them when it comes to completionItem/resolve. PR incoming.
• Oni only supports single trigger characters for the completion popup, i.e. . works, but not -> nor ::. You can use > and : as a workaround, but it's a bit weird.
• cquery requires more configuration that ccls - it just quits out unless you give it a cache directory to use. My config for cquery looks like (most of it is for logging, the key bit is the --init):

    "language.cpp.languageServer.command": "/path/to/cquery",
    "language.cpp.languageServer.arguments": [
        "--log-file=/tmp/cquery.log", `--init={"cacheDirectory": "/tmp/cquery.cache"}`
        "--record", "/tmp/cquery"
    ],

[feltech]

Updated wiki with config for cquery and a warning about code completion.

[MaskRay]

Some more on ccls/cquery with Oni... I was a bit premature in my optimism. In particular, completions are a pain. I gave up on ccls and moved to cquery to see if that was any better, but they seem to suffer the same problems. In particular I have discovered (so far):

• Critically: Oni's language server integration completely hangs when requesting completionItem/resolve from cquery. The server simply does not respond to the request, so the promise is never fulfilled and the request queue just keeps building up forever.
  The fix is simple, however. Oni already queries for the language server "capabilities", and cquery responds saying it doesn't support completionItem/resolve (completionProvider.resolveProvider is false), but Oni ignores them when it comes to completionItem/resolve. PR incoming.
• Oni only supports single trigger characters for the completion popup, i.e. . works, but not -> nor ::. You can use > and : as a workaround, but it's a bit weird.
• cquery requires more configuration that ccls - it just quits out unless you give it a cache directory to use. My config for cquery looks like (most of it is for logging, the key bit is the --init):

    "language.cpp.languageServer.command": "/path/to/cquery",
    "language.cpp.languageServer.arguments": [
        "--log-file=/tmp/cquery.log", `--init={"cacheDirectory": "/tmp/cquery.cache"}`
        "--record", "/tmp/cquery"
    ],

The issue is that ccls/cquery do not set completionProvider.resolveProvider. oni should not try resolving completion items (they are already resolved in the textDocument/completion responses)

In particular, completions are a pain.

If you find snippets $1 $2 inserted, you may check emacs-lsp/emacs-ccls#15
I've written something about expected responses in the thread.

Oni only supports single trigger characters for the completion popup, i.e. . works, but not -> nor ::. You can use > and : as a workaround, but it's a bit weird.

If this is ever an issue, there must be something wrong on oni's side. They work pretty well in other language clients (lsp-mode, LanguageClient-neovim, vscode-languageclient)

[feltech]

The issue is that ccls/cquery do not set completionProvider.resolveProvider

Like I say, I found that cquery does set it, and sets it to false, but Oni ignores it. This is fixed in the PR above.

Oni only supports single trigger characters for the completion popup, i.e. . works, but not -> nor ::. You can use > and : as a workaround, but it's a bit weird.

If this is ever an issue, there must be something wrong on oni's side.

Yeah I found the code in Oni responsible for that and started work on a fix, but the logic is a bit fiddly, to say the least!

[MaskRay]

@feltech For the single character triggerCharacters: microsoft/language-server-protocol#138

The client sends > as triggerCharacter in textDocument/completion. While it can check its own buffer lines and send more characters, or the server can check its view of the buffer, but the specification or the reference implementation (vscode) does not do the sensible thing.