JupyterServer: opening content in other extensions

[oliver-sanders]

The Cylc UI Server allows browsing workflows, cycles, tasks and whatnot.

We can open log files, but, the Cylc app lacks tooling to inspect and analyse task output. This is where tools like Jupyter Lab come into it.

We should make it easier to open resources associated with Cylc (e.g. data created by tasks) in other server extensions (e.g. Jupyter Lab).

E.G. We could have a button which opens a task's work directory in Jupyter Lab.

This is actually kinda possible, Jupyter Lab has a URL argument to open the server at a specific location, however, because this is done in the URL you get a new Lab instance every time you follow the link. Ideally we would like the ability to open tabs in an existing Lab session from within Cylc.

There are two ways to go about this problem:

1. Send the message down into the server, and from the server back up to the Lab app.
2. Use LocalStorage to send the message sideways to active Lap app sessions.

Ideally Jupyter Server would provide an interface for doing this so that all extensions can communicate in this way. Have a chat with the Jupyter folks and see what they think.

[oliver-sanders]

Have made some progress towards opening Jupyter Lab from the Cylc UI with these two PRs:

• List Jupyter Server extensions in the userprofile - add jupyter extensions to userprofile handler #641
• Add a Jupyter Lab button on the Cylc UI dashboard - jupyter lab dashboard link cylc-ui#1982

We can take this further and open Jupyter Lab in a specified directory, these two branches allow opening Jupyter Lab in a workflow's source directory:

• Add workflow's source directories into the GraphQL schema - schema: add source workflows #512
• Add an "open source" button in Cylc UI (WIP) - https://github.com/oliver-sanders/cylc-ui/pull/new/jupyterlab

Opening content in existing Jupyter Lab sessions is trickier, however, it looks like there is now a viable option in the form of jupyter_events. This is a global Jupyter event bus implemented in jupyter_server and available to all extensions. Currently there are only a couple of events going into this. It should be possible for us to register an event in cylc-ui requesting jupyter-lab to open a file, and have jupyter-lab subscribe to this event and handle it when fired. This would require upstream changes in jupyter-lab and a bit of dialogue as I think the event bus is more geared towards telemetry at the moment.

[oliver-sanders]

Here's a POC with three components:

• cylc-ui
• cylc-uiserver
• jupyterlab extension

It uses the Jupyter Events package. This is relatively new, but a dependency of Jupyter Server.

Questions remain over:

• The Jupyter Lab Eventlistener package:
  • This is currently a personal project with a single maintainer.
  • However, I expect Jupyter will absorb this project into one of their organisations in due course.
  • We will likely require this project to release on Conda Forge.
• Packaging:
  • It would be good to make the Jupyter Lab extension a dependency of cylc-uiserver so that it is always installed.
  • However, this might bring in Jupyter Lab as a dependency (which we don't want).