I have a bunch of features to contribute

[benlubas]

Hi Quarto maintainers. I apologize in advance for dumping a bunch of review/discussion onto the project at once. I have a few (>20 depending on how you count) features/fixes that I've implemented over the past ~4 months, and I'd like to contribute them. I started with two PRs for features I think fit really well and don't require much review:

• feat: support for q/kdb programming language #13936
• feat(html): section-collapse metadata #13941

But I'm creating this post to discuss some of the rest, I'll break them into sections below. To be clear, I've already implemented all of these features, and have used and tested them extensively (albeit on an older version of quarto 1.6, and only on Linux)

I'd like to know if you would accept the below features as contributions before I take the time to open a bunch of PRs and test each one individually.

HTML

zoom-content

Along with #13941, I've added zoom-content metadata which adds a button to the Code Tools menu which:

• hides website navigation, table of contents, and other "side-bar" elements
• makes the main content as wide as possible (while maintaining some margin to look nice)

Justification: Adding plots to a quarto report is really common, and sometimes you want the extra space, especially if you're panning around something in Plotly.

anchor-click

- name: anchor-click
  schema:
    enum: [jump, copy, copy-and-jump]
  default: jump
  tags:
    formats: [$html-doc]
  description:
    short: "Change behavior when clicking on a section anchor."
    long: |
      Change behavior when clicking on a section anchor.
      - `jump` (default): navigate to anchor destination
      - `copy`: copy the destination to clipboard, don't change the window location
      - `jump-and-copy`: navigate to the destination, and copy it to clipboard

Justification, confluence anchor copy works like "copy" option above. If people are used to that, they prefer this behavior.

fix: hide all code misses output: asis cells

Currently output: asis code cells aren't folded by "fold all" button in code tools menu.

Similarly, this type of code cell isn't folded, but this is probably intentional, right?

```{{python}}
# these code cells
```

Confluence Pandoc

• support for HTML macro which allows for {=html} and text/html mime outputs to render in confluence

  • this macro is opt-in and off by default on the confluence side. So we may need to put this behind metadata. They way I've implemented it, it's always on

• the HTML macro also enables the following:

  • code-tools-like menu which expands and collapses all code
  • support for mermaid diagrams
  • table of contents on the side of the page similar to HTML (using Confluence TOC macro)
  • latex using mathjax
  • max-width metadata to set the width of a page

• improved callout support

  • fixed bug with including headings in callouts
  • adding expand support

• improved codeblock support

  • code-line-numbers
  • code-fold

• latex macro support

• tabset support

• jira shortcode {{< jira TEAM-1234 >}} becomes the jira macro for that ticket

• user shortcode {{< user lubasben >}} becomes the user macro @lubasben

• fixed un-escaped cdata bug

Confluence Server Support

The way I implemented this for now, I've just totally replaced confluence cloud support. But I'll happily re-write to allow both to work together if you'll accept the contribution. I only had to change the Confluence API Client. Pretty much everything else worked the same way. As far as I can tell confluence storage format works the same between server and cloud, and most of the rest API works the same, with slightly different endpoints in some cases.

My question for the devs: How should we expose this to the user? An initial setup question that's stored along side the url/PAT seems like the best option. I think quarto publish confluence-server is more annoying to type out. And environment variables are not visible enough.

I've also fixed some bugs related to publishing projects (specifically related to links between pages, nested pages being unrecognized for updates/deletions, and the _site folder not being cleared before re-rending the project). Some of these could be bugs specific to confluence server, I've no way to test on confluence cloud.

General Confluence CLI

--create-url flag

Allows the user to pass the URL on the command line.

set value of browser based on $DISPLAY

Having to constantly pass --no-browser when you're running over ssh was annoying

various --no-prompt logic fixes

Currently it's possible to use --no-prompt and still get prompted for input. I've changed these cases to instead throw useful error messages. When the user doesn't specify an ID and they're updating a page we crash after printing up to 5 previous publish targets for the user to chose, often this lets the user immediately fix their command without having to run without --no-prompt.

Removed email address field

Replaced with "profile name". The email wasn't used, and was confusing. Email validation meant specifying two different 'valid' emails if you have a production and dev confluence instance to publish to. foo and foo-dev are better names that [email protected] and [email protected].

Removed server validator

This step was also a cause for confusion. Users commonly used foo.confluence.com/path/to/something instead of the root domain. The validator would succeed b/c foo.confluence.com/path/to/something actually existed and returns a 200 when curled, so the "Verified domain" text pops up, only for the next rest endpoint request to fail. Leading to confusion about the source of the error.

Jupyter Kernel

Fully resolve kernel path

JupyterQ kernel requires this to function. It could be a bug in jupyterQ, but nbconvert is able to execute the same file with the same kernel, so I think it makes sense to fix it in quarto.

--- a/src/resources/jupyter/notebook.py
+++ b/src/resources/jupyter/notebook.py
@@ -209,6 +209,13 @@
     nb = nbformat.read(input, as_version=NB_FORMAT_VERSION)
 
     trace("notebook was read")
+    if not nb.metadata.kernelspec.get("path"):
+        full_spec = kernelspec.get_kernel_spec(nb.metadata.kernelspec.name)
+        if full_spec.resource_dir:
+            # (this value needs to be set for jupyterQ)
+            nb.metadata.kernelspec.path = full_spec.resource_dir

Conditional Execution

The idea here is similar to how KnitR allows you to include cell output and decide weather or not to execute it based on some value evaluated by the R kernel. But this works with any Jupyter kernel (with the required language-by-language support)

Implementation details: modify src/resources/jupyter/notebook.py > notebook_execute function, a lot.

• Conditional metadata is evaluated by user_expressions (similar to the inline code execution feature) before the cell is sent to the kernel.
• The expression is first wrapped in a language specific string which will convert it to a boolean (in python this is simply bool(%s), in q it's $[%s;1b;0b])
• The full expression is evaluated and the value returned by the kernel is compared against the known "true" value for the language (python: "True", q: "1b")
• Based on the result we essentially modify values of echo eval

There are some more details here. I'm not going to waste time if you think this feature is too out-there.

```{python}
#| cond: os == "Linux"
...
```
```{python}
#| cond: os == "Windows"
...
```
```{python}
#| cond: os == "MacOS"
...
```

I've also implemented conditional markdown fences. I'm mis-using the div syntax a little here, I know. If you're interested in this feature, I think we should give the syntax a little more thought. This worked well enough for me, and it's more or less invisible to other markdown engines that don't understand it.

::: {.cond `os == "Linux"`}
Regular markdown content.

```{python}
# even additional code cells which are only executed & included if the
# outer conditional is true
assert os == "Linux"
```
:::

Metadata

--default-metadata-file

Metadata in this file is overwritten by any other specified metadata. Allows for easy "team defaults" to create a consistent look across different users' reports without users having to copy around a ton of frontmatter (plus easy updates to the shared defaults).

Absolute path support for metadata-files field

I know you may not like this one, b/c you like quarto projects to be self contained. But, it makes it so much easier when you have multiple projects that you want to feel similar.

---

And that is all. Mostly, I'm sure I'm forgetting some one-liners. I also have a patch to preserve inline comments between confluence updates, but I'll save that for now.

Thank you for considering these contributions, and I look forward to hopefully working together to get these features upstreamed!