Add tests and test support commands for visual editor

[vezwork]

---

This PR builds on-top of #757 to add tests, supporting functionality in test-utils, and registers visual editor commands specifically for tests. Two tests were added, they test:

1. opening the visual editor, and
2. roundtripping a qmd file through the visual editor.

Unfortunately these tests rely on waiting an approximate amount of time for the visual editor to open and for other related commands to complete. They are slow and potentially flaky. I think this could be improved, but I'm not yet sure how. We may be able to return things from commands that help us understand if the visual editor has finished opening (maybe we will have to do some polling?). I wonder how other teams do this...

The tests pass on my machine and its kind of fun to watch :)

edit: I just noticed the tests fail on this PR. It seems to be because the extension is not built before the tests are ran, resulting in the commands that were added in this PR not being present at the time of running tests. I attempted a fix (in now-reverted commits) that:

• ran the test-extension job after the package-extension job but that did not help;
• I also tried putting the test workflow steps at the end of the package-extension job, but that did not help;
• I also tried registering the added testing-specific commands into the extension's package.json, but that did not help.

[cscheid]

I'm pretty out of my depth here, but it looks like activateEditor (quarto/apps/vscode/src/providers/editor/editor.ts) isn't getting called, or those ids are not being set as one would expect. Is there a way to test this hypothesis?

[vezwork]

I'm pretty out of my depth here, but it looks like activateEditor (quarto/apps/vscode/src/providers/editor/editor.ts) isn't getting called, or those ids are not being set as one would expect. Is there a way to test this hypothesis?

I added a log in activateEditor to attempt to test this hypothesis. The log does not show up in the workflow logs but it does show locally. This test confirms, at least, that the code of activateEditor introduced in this PR is not being run. Unfortunately it does not help us understand why. I believe that activateEditor is being run, just not the version introduced in this PR, but I have no way to confirm that (I haven't found any way to access data from the running extension other than through commands).

---

Locally:

In the workflow:

---

[juliasilge]

The tests pass on my machine and its kind of fun to watch

I agree entirely!

I can dig into this a bit more later, but I am wondering if the extension is not getting activated with its LSP and similar. We might want to look at how the air extension runs tests to dig into this a bit. Here is there GH action (they bundle a binary in their extension FYI):

https://github.com/posit-dev/air/blob/main/.github/workflows/test-code.yml

And here is where they have some code they use for the extension to be activated and the LSP to be running:

https://github.com/posit-dev/air/blob/main/editors/code/src/test/extension.ts

[vezwork]

I was able to verify that the extension's commands are not present at all. Here's how: I removed the runCommands for the commands I added in this PR and only left in runCommands for existing Quarto commands. Here's what I got:

 Error: command 'quarto.editInVisualMode' not found

So there seems to be no Quarto commands available right now!

I'm gonna check out what @juliasilge sent! That second link looks really promising for ensuring the extension is activated.

[vezwork]

Some progress. I tried to explicitly activate the extension in the tests and got this error in the logs:

 Activating extension 'quarto.quarto' failed: Cannot find module '/home/runner/work/quarto/quarto/apps/vscode/out/main.js'

[vezwork]

Ok dang... I added a test workflow step to cd app/vscode, yarn install and yarn run build. Activating the extension no longer fails, but we're back to this error:

Error: command 'quarto.test_setkVisualModeConfirmedTrue' not found

edit: and if I remove calls to the new test commands, and add some long waits, I still get this error:

Error: command 'quarto.editInVisualMode' not found

so activateEditor in apps/vscode/src/providers/editor/editor.ts is still not being called somehow.

[vezwork]

More progress! I added a bunch of console.logs into activate in apps/vscode/src/main.ts (where activateEditor is called) and I found that the console.logs inside of this if statement are not called in the workflow:

if (quartoContext.available) { 

So...

@cscheid I have narrowed down the issue with tests in the github workflow: quartoContext.available is not true while the extension is being activated.
I think this makes sense! Because we don't install quarto-cli in the workflow and I don't believe it comes bundled with the extension.
Now, isn't the quarto-cli install pretty bulky? Would it be too expensive to install it in the workflow? Can we avoid installing it somehow (we need it for the extension so I don't see how we could)?

[juliasilge]

Ah, of course! We should definitely install Quarto in this workflow! I definitely should have done that already 🙈

I think a good place to start is just the regular install that you can do via:
https://github.com/quarto-dev/quarto-actions

[vezwork]

Nice! That helps. More progress! I used the quarto action you linked to install quarto-cli in the test workflow and now we have a new error!

 Error: Failed to start Quarto LSP Server
  	at /home/runner/work/quarto/quarto/apps/vscode/out/main.js:395:9526
  	at LS.invoke (out/main.js:81:11593)
  	at E0.fire (out/main.js:81:12362)
  	at set $state (out/main.js:329:111045)
  	at lk.handleConnectionClosed (out/main.js:331:13596)
  	at process.processTicksAndRejections (node:internal/process/task_queues:105:5)

which may have to do with some other errors I have been seeing in the workflow logs? These logs don't show up locally for me, such as:

[3125:0723/174614.167244:ERROR:bus.cc(408)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
[3125:0723/174614.167253:ERROR:object_proxy.cc(576)] Failed to call method: org.freedesktop.DBus.NameHasOwner: object_path= /org/freedesktop/DBus: unknown error type: 
[main 2025-07-23T17:46:14.509Z] update#setState disabled
[main 2025-07-23T17:46:14.510Z] update#ctor - updates are disabled by the environment
[3125:0723/174614.511033:ERROR:bus.cc(408)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
[3153:0723/174617.044493:ERROR:viz_main_impl.cc(183)] Exiting GPU process due to errors during initialization
workbench#open()

edit: I tried removing the explicit extension activation. Back to the original error about not recognizing quarto commands.

edit 2: I tried adding additional logging next to the Error: Failed to start Quarto LSP Server error and saw that the server goes from starting status to startFailed but I was not able to understand anything further than that.

edit 3: I added some logging to various spots in apps/lsp/src/index.ts and none of the logs show up. Not even one at the top of the file. Is the lsp code not being run at all? Why not? I am running out of ideas.

[vezwork]

@lionel- I am getting a bit stuck on debugging the quarto extension test github workflow. Do you have any advice or guidance? Do you know why our LSP server might not run in the workflow?

[juliasilge]

@lionel- I notice that in air, you do this to wait for the LSP to be fully started up in tests:

https://github.com/posit-dev/air/blob/7125f07fb88ea9b79387b04d0a8fa131b3cc5044/editors/code/src/test/extension.ts#L46

Could you say more about that in a testing context? And how we might apply that to the Quarto extension?

[vezwork]

Skipping the added in tests in the workflow/CI for now. This PR is ready for review and merge. We can come back to getting the extension, more specifically the LSP, working in CI later. Any advice on that is still appreciated!

[cscheid]

Looks good to me!

[juliasilge]

Looks great!

  Quarto basics
    ✔ Can open a Quarto document (199ms)
    - Can edit in visual mode
    - Roundtrip doesn't change hello.qmd
  1 passing (261ms)
  2 pending

Let's see if we can continue to iterate on these tests so we can run them in CI.

[lionel-]

@juliasilge

I notice that in air, you do this to wait for the LSP to be fully started up in tests:

https://github.com/posit-dev/air/blob/7125f07fb88ea9b79387b04d0a8fa131b3cc5044/editors/code/src/test/extension.ts#L46

Could you say more about that in a testing context? And how we might apply that to the Quarto extension?

This starts the LSP and only resolves when fully started (the heavy lifting is done by vscode-languageclient's start() method). It also ensures the quarto extension has been activated, as relying on activation events can be flaky. Tests that rely on LSP features should generally call something like this on init to ensure proper synchronisation.

However from the error messages above it looks like the LSP is getting started but fails to do so.

[vezwork]

I tried one last thing this morning (and reverted it): doing yarn install and yarn run build at the top level. This runs into an issue where yarn run build currently exits with an error because of build errors in Quarto writer.