notebook markdown rendering improvements

[dhruvisompura]

Addresses #10291

This updates some of the markdown rendering styles to better match up with the styles used in other notebooks.

What did get fixed:

• tables
• block quotes
• inline code
• code blocks

What did not get fixed:

• embedded images (there is a separate issue Notebook - images displayed within a markdown cell do not render #10594)
• math expressions (there is a separate issue MathJax/KaTeX initialization potentially missing in (new) Positron notebooks leads to inline math not being rendered in markdown cells #10479)

NOTE: text content spanning multiple lines works differently for positron notebooks than built-in notebooks: Our markdown rendering logic converts single newlines to spaces instead of treating them as line breaks. Built-in notebooks treat newlines as line breaks. This can be changed but I've left it as is for now. This means that for sentences in block quotes to be on separate lines you need to do:

# this text renders on separate lines but there is an empty line separating them.
> This is a blockquote
>
> It can span multiple lines.

instead of:

# this text will actually render on a single line
> This is a blockquote
> It can span multiple lines.

Code Blocks Whitespace Fix

Understanding the cause of the extra whitespace at the bottom of code blocks in Positron notebooks was kind of a pain. I'm including an explanation in case someone has a better suggestion. My fix included modifying the code block renderer in markdownRenderer.ts but I don't know why the newline was added in the first place, so maybe there's a better solution?

The root cause of the extra whitespace at the bottom of code blocks in Positron notebooks is caused by:

1. markdownRenderer.ts:111 adding a trailing \n to code blocks when rendering them to HTML

return `<pre><code${classAttr}>${escaped ? text : escape(text)}\n</code></pre>`;

2. renderHtml.tsx:75 which wraps ALL text nodes (including newlines and spaces) in <span> elements

return React.createElement('span', {}, node.content);

The renderHtml function in renderHtml.tsx looks like it intentionally preserves all whitespace text nodes because they can be "semantically significant" but in this case the trailing \n inside the <span> is not significant and causes layout issues.

Code blocks are <code> wrapped in a <pre>. The <pre> element has built-in white-space: pre style, which preserves the \n inside the <span>, causing visible whitespace at the bottom of code blocks.

Built-in notebooks don't have this issue because they use markdown-it instead of marked for markdown rendering which doesn't add the trailing \n to code blocks from what I can tell.

Table Rendering Fix

Similar to the code block styling issue with newlines; the html returned by markdownRenderer.ts includes newlines between all the table markup. The html looks something like:

'<table>\n<thead>\n<tr>\n<th>Header 1</th>\n<th>Header 2</th>\n<th>Header 3</th>\n</tr>\n</thead>\n<tbody><tr>\n<td>Cell 1</td>\n<td>Cell 2</td>\n<td>Cell 3</td>\n</tr>\n<tr>\n<td>Cell 4</td>\n<td>Cell 5</td>\n<td>Cell 6</td>\n</tr>\n</tbody></table>\n'

When we go to parse this html prior to rendering (htmlParser.ts:307), we would keep all of these newlines. These newlines then got wrapped in <span> elements. Having <span> elements between the table specific elements breaks the table html spec rules.

Per HTML specification:

• table can only contain: caption, colgroup, thead, tbody, tfoot, tr
• thead, tbody, tfoot can only contain: tr
• tr can only contain: td, th

Breaking this spec prevented existing css from working on tables, such as border-collapse: collapse, which prevents double borders.

To fix this, we now filter out whitespace-only text nodes that are between the table elements listed above when we parse the html string.

AFTER

Untitled.mov

Release Notes

New Features

• N/A

Bug Fixes

• N/A

QA Notes

@:positron-notebooks

For testing, I used the following markdown:

# H1
## H2
### H3
#### H4
##### H5
###### H6

### Bold Text

**The quick brown fox jumps over the lazy dog.**

### Italic Text

*The quick brown fox jumps over the lazy dog.*

### Strikethrough Text
~~The quick brown fox~~ jumps over the lazy dog.

### Inline Code

`print("Hello, world!")`

### Combined Formatting

This is **bold** and *italic* and `code` with spaces between them.

Multiple links: [link one](url1) and [link two](url2) next to each other.

Text with **bold *nested italic*** and `inline code` elements.

This paragraph has **bold**, *italic*, `code`, and [link](url) elements mixed together with normal text.

here is some inline code `print("Hello, world!")` surrounded by plain text.

**here is some inline code `print("Hello, world!")` surrounded by bolded text.**

*here is some inline code `print("Hello, world!")` surrounded by italic text.*

***here is some inline code `print("Hello, world!")` surrounded by bold and italic text.***

### Code block

```python
# This is a sample Python function
def hello_world():
    print("Hello, world!")
hello_world()
```

```javascript
// This is a sample JavaScript function
function greet(name) {
  console.log(`Hello, ${name}!`);
}
greet("Alice");
```

```
// This has no language specified
function greet(name) {
  console.log(`Hello, ${name}!`);
}
greet("Alice");
```

### Lists

- Bullet list item 1
- Bullet list item 2
    - Sub-item 1
    - Sub-item 2

* Bullet list item A
* Bullet list item B
    * Sub-item A
    * Sub-item B

1. Numbered list
2. Numbered list item 2
    1. Sub-item 1
    2. Sub-item 2

### Mixed Lists
1. Ordered item
   - Unordered nested
   - Another nested
2. Second ordered
   - More nesting

### Lists with Complex Content
- Item with **bold** text
- Item with [link](url)
- Item with `code`
- Item with multiple
  lines of text

### Links

[Link to posit.co](https://posit.co)

### Math

$x^2 + y^2 = z^2$

When $x = 2$, then $f(x) = x^2 + 3x + 1$.


### Horizontal Rule

Text before rule

---

Text after first rule

***

Text after second rule

___

Text after third rule

### Block quote

> This is a blockquote
> It can span multiple lines.

> This is a blockquote
> It can span multiple lines.
>
> and a blank line

> Nested quote
>> Inner quote

### Embed image

<img src="https://posit.co/wp-content/themes/Posit/assets/images/posit-logo-2024.svg" alt="Posit Logo" style="width:30%;">

### Multiple Markdown Tables

| Header 1 | Header 2 | Header 3 |
|----------|----------|----------|
| Cell 1   | Cell 2   | Cell 3   |
| Cell 4   | Cell 5   | Cell 6   |


| Header 1 | Header 2 | Header 3 |
|----------|----------|----------|
| Cell 1   | Cell 2   | Cell 3   |
| Cell 4   | Cell 5   | Cell 6   |

### Table with Alignment

| Left | Center | Right |
|:-----|:------:|------:|
| L1   | C1     | R1    |
| L2   | C2     | R2    |

### HTML Table

<table>
  <thead>
    <tr>
      <th>Header 1</th>
      <th>Header 2</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Cell 1</td>
      <td>Cell 2</td>
    </tr>
  </tbody>
</table>

### HTML Table (Incorrect HTML Structure)

<table>
  <tr>
    <th>Header 1</th>
    <th>Header 2</th>
  </tr>
  <tr>
    <td>Cell 1</td>
    <td>Cell 2</td>
  </tr>
</table>

### Multiple  HTML Tables

<table>
  <thead>
    <tr>
      <th>Header 1</th>
      <th>Header 2</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Cell 1</td>
      <td>Cell 2</td>
    </tr>
  </tbody>
</table>

<table>
  <thead>
    <tr>
      <th>Header 1</th>
      <th>Header 2</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Cell 1</td>
      <td>Cell 2</td>
    </tr>
  </tbody>
</table>

### Table with Complex Content

| **Bold** | *Italic* | `Code` |
|----------|----------|--------|
| [Link](https://example.com) | ![Image](image.png) | Text with spaces |


### Table with Empty Cells

| A | B | C |
|---|---|---|
|   | Empty | |
| Data | | More |

### List with Table
- List item with table:

  | A | B |
  |---|---|
  | 1 | 2 |

- Another item


# Edge Cases

### Pre-formatted Text
<pre>
  Whitespace
    should be
      preserved
</pre>

### Self-closing Tags
Line break:<br>
Another line

Horizontal rule:<hr>

### Special Characters

&lt; &gt; &amp; &quot; &#39;

[github-actions]

E2E Tests 🚀
This PR will run tests tagged with: @:critical @:positron-notebooks

^{readme  valid tags}

[seeM]

This looks great ty!

[nstrayer]

All looks good to me and the example content renders much better now.

I'm surprised these edge cases exist but glad you got to the bottom of them!

If we need to add much more logic in here in the future we should probably see if there's a way we can rethink the whole problem.

[nstrayer]

What about using a set. E.g.

const tableElements = new Set([
	'table',
	'thead',
	'tbody',
	'tfoot',
	'tr',
	'colgroup'
]);

Then you can do:

tableElements.has(node.name)

Probably actually has worse performance because of how javascript handles short objects with string keys but maybe is more clear?

[nstrayer]

NVM I see that it's copying the pattern from above. No need to change!

[dhruvisompura]

All looks good to me and the example content renders much better now.

I'm surprised these edge cases exist but glad you got to the bottom of them!

If we need to add much more logic in here in the future we should probably see if there's a way we can rethink the whole problem.

I agree! I think the core of the issue is that we turn every whitespace character into a span. Normally, newlines in html strings are okay because they get collapsed. I don't think this is the ideal fix but the other options are a heavier lift. Something to improve in the future!