Skip to content

[WEB-3888] Introduce support for MDX templating #2527

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 2, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Ignore all mdx files in src/pages - we need to be able to render components on one line for inlining
src/pages/**/*.mdx
src/pages/**/*.md
2 changes: 1 addition & 1 deletion content/chat/index.textile
Original file line number Diff line number Diff line change
@@ -40,4 +40,4 @@ h3(#reactions). Room reactions

h2. Demo

Take a look at a "livestream basketball game":https://ably-livestream-chat-demo.vercel.app with some simulated users chatting built using the Chat SDK. The "source code":https://github.com/ably/ably-chat-js/tree/main/demo is available in GitHub.
Take a look at a "livestream basketball game":https://ably-livestream-chat-demo.vercel.app with some simulated users chatting built using the Chat SDK. The "source code":https://github.com/ably/ably-chat-js/tree/main/demo is available in GitHub.
2 changes: 1 addition & 1 deletion content/chat/rooms/messages.textile
Original file line number Diff line number Diff line change
@@ -573,4 +573,4 @@ Applying an action to a message produces a new version, which is uniquely identi

The @Message@ object also has convenience methods "@isOlderVersionOf@":https://sdk.ably.com/builds/ably/ably-chat-js/main/typedoc/interfaces/chat-js.Message.html#isolderversionof, "@isNewerVersionOf@":https://sdk.ably.com/builds/ably/ably-chat-js/main/typedoc/interfaces/chat-js.Message.html#isnewerversionof and "@isSameVersionAs@":https://sdk.ably.com/builds/ably/ably-chat-js/main/typedoc/interfaces/chat-js.Message.html#issameversionas which provide the same comparison.

Update and Delete events provide the full message payload, so may be used to replace the entire earlier version of the message.
Update and Delete events provide the full message payload, so may be used to replace the entire earlier version of the message.
54 changes: 45 additions & 9 deletions data/onCreatePage.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
import { GatsbyNode } from 'gatsby';
import path from 'path';
import fs from 'fs';

export type LayoutOptions = { sidebar: boolean; searchBar: boolean; template: string };

const mdxWrapper = path.resolve('src/components/Layout/MDXWrapper.tsx');

const pageLayoutOptions: Record<string, LayoutOptions> = {
'/docs': { sidebar: false, searchBar: false, template: 'index' },
'/docs/api/control-api': { sidebar: false, searchBar: true, template: 'control-api' },
@@ -11,17 +15,49 @@ const pageLayoutOptions: Record<string, LayoutOptions> = {
'/docs/404': { sidebar: false, searchBar: false, template: '404' },
};

export const onCreatePage: GatsbyNode['onCreatePage'] = ({ page, actions }) => {
const { createPage } = actions;
// Function to extract code element classes from an MDX file
const extractCodeLanguages = async (filePath: string): Promise<Set<string>> => {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A first go at something a little less "runtime". Since the syntax for defining codeblocks is set by Markdown which is quite rigid itself - we can dig through MDX files for present languages and build a representative superset that way. Textile files are still determined at runtime as they are still tied into the half-Textile-half-React world.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nice! It would have been super convenient if the raw content of the file was available in onCreatePage, then you could forego all the duplicate IO (just a wishlist item)

try {
// Check if the file exists
if (!fs.existsSync(filePath)) {
return new Set();
}

const pathOptions = Object.entries(pageLayoutOptions).find(([path]) => page.path === path);
// Read the file content
const fileContent = fs.readFileSync(filePath, 'utf8');

if (pathOptions) {
page.context = {
...page.context,
layout: pathOptions[1],
};
// Find all instances of code blocks with language specifiers (```language)
const codeBlockRegex = /```(\w+)/g;
let match;
const languages = new Set<string>();

while ((match = codeBlockRegex.exec(fileContent)) !== null) {
if (match[1] && match[1].trim()) {
languages.add(match[1].trim());
}
}
return languages;
} catch (error) {
console.error(`Error extracting code element classes from ${filePath}:`, error);
return new Set();
}
};

export const onCreatePage: GatsbyNode['onCreatePage'] = async ({ page, actions }) => {
const { createPage } = actions;
const pathOptions = Object.entries(pageLayoutOptions).find(([path]) => page.path === path);
const isMDX = page.component.endsWith('.mdx');
const detectedLanguages = isMDX ? await extractCodeLanguages(page.component) : new Set();

createPage(page);
if (pathOptions || isMDX) {
createPage({
...page,
context: {
...page.context,
layout: pathOptions ? pathOptions[1] : { sidebar: true, searchBar: true, template: 'base' },
...(isMDX ? { languages: Array.from(detectedLanguages) } : {}),
},
component: isMDX ? `${mdxWrapper}?__contentFilePath=${page.component}` : page.component,
});
}
};
35 changes: 34 additions & 1 deletion gatsby-config.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import dotenv from 'dotenv';
import remarkGfm from 'remark-gfm';

dotenv.config({
path: `.env.${process.env.NODE_ENV}`,
@@ -38,6 +39,8 @@ export const siteMetadata = {

export const graphqlTypegen = true;

const headerLinkIcon = `<svg aria-hidden="true" height="20" version="1.1" viewBox="0 0 16 16" width="20"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg>`;

export const plugins = [
'gatsby-plugin-postcss',
'gatsby-plugin-image',
@@ -51,7 +54,37 @@ export const plugins = [
'how-tos': `${__dirname}/how-tos`,
},
},
'gatsby-plugin-mdx',
{
resolve: 'gatsby-plugin-mdx',
options: {
gatsbyRemarkPlugins: [
{
resolve: `gatsby-remark-autolink-headers`,
options: {
offsetY: `100`,
icon: headerLinkIcon,
className: `gatsby-copyable-header`,
removeAccents: true,
isIconAfterHeader: true,
elements: [`h2`, `h3`],
},
},
],
mdxOptions: {
remarkPlugins: [
// Add GitHub Flavored Markdown (GFM) support
remarkGfm,
],
},
},
},
{
resolve: `gatsby-source-filesystem`,
options: {
name: `pages`,
path: `${__dirname}/src/pages`,
},
},
// Images
{
resolve: 'gatsby-source-filesystem',
6 changes: 4 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -40,7 +40,7 @@
"validate-llms-txt": "ts-node bin/validate-llms.txt.ts"
},
"dependencies": {
"@ably/ui": "16.1.1",
"@ably/ui": "16.2.0",
"@codesandbox/sandpack-react": "^2.20.0",
"@codesandbox/sandpack-themes": "^2.0.21",
"@mdx-js/react": "^2.3.0",
@@ -59,11 +59,12 @@
"gatsby-plugin-image": "^3.3.0",
"gatsby-plugin-layout": "^4.14.0",
"gatsby-plugin-manifest": "^5.3.0",
"gatsby-plugin-mdx": "^5.12.0",
"gatsby-plugin-mdx": "^5.14.0",
"gatsby-plugin-react-helmet": "^6.3.0",
"gatsby-plugin-root-import": "^2.0.9",
"gatsby-plugin-sharp": "^5.8.1",
"gatsby-plugin-sitemap": "^6.12.1",
"gatsby-remark-autolink-headers": "^6.14.0",
"gatsby-source-filesystem": "^5.12.0",
"gatsby-transformer-remark": "^6.12.0",
"gatsby-transformer-sharp": "^5.3.0",
@@ -81,6 +82,7 @@
"react-helmet": "^6.1.0",
"react-medium-image-zoom": "^5.1.2",
"react-select": "^5.7.0",
"remark-gfm": "^1.0.0",
"textile-js": "^2.1.1",
"turndown": "^7.1.1",
"typescript": "^4.6.3",
10 changes: 7 additions & 3 deletions src/components/Layout/Layout.tsx
Original file line number Diff line number Diff line change
@@ -12,8 +12,12 @@ import Header from './Header';
import LeftSidebar from './LeftSidebar';
import RightSidebar from './RightSidebar';

type PageContextType = {
export type PageContextType = {
layout: LayoutOptions;
languages?: string[];
frontmatter?: {
title: string;
};
};

type LayoutProps = PageProps<unknown, PageContextType>;
@@ -26,7 +30,7 @@ const Layout: React.FC<LayoutProps> = ({ children, pageContext }) => {
<Header searchBar={searchBar} />
<div className="flex pt-64 md:gap-48 lg:gap-64 xl:gap-80 justify-center ui-standard-container mx-auto">
{sidebar ? <LeftSidebar /> : null}
<Container as="main" className="flex-1">
<Container as="main" className="flex-1 overflow-x-auto">
{sidebar ? <Breadcrumbs /> : null}
{children}
<Footer />
@@ -38,7 +42,7 @@ const Layout: React.FC<LayoutProps> = ({ children, pageContext }) => {
};

const WrappedLayout: React.FC<LayoutProps> = (props) => (
<LayoutProvider>
<LayoutProvider pageContext={props.pageContext}>
<Layout {...props} />
</LayoutProvider>
);
154 changes: 154 additions & 0 deletions src/components/Layout/MDXWrapper.test.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
import React, { ReactNode } from 'react';
import { render, screen } from '@testing-library/react';
import If from './mdx/If';
import CodeSnippet from '@ably/ui/core/CodeSnippet';

// Mock the dependencies we need for testing
jest.mock('./MDXWrapper', () => {
return {
__esModule: true,
default: ({ children, pageContext }: { children: ReactNode; pageContext: any }) => (
<div data-testid="mdx-wrapper">
{pageContext?.frontmatter?.title && <h1>{pageContext.frontmatter.title}</h1>}
<div data-testid="mdx-content">{children}</div>
</div>
),
};
});

// Mock the layout context
jest.mock('src/contexts/layout-context', () => ({
useLayoutContext: () => ({
activePage: { language: 'javascript' },
setLanguage: jest.fn(),
}),
LayoutProvider: ({ children }: { children: ReactNode }) => <div data-testid="layout-provider">{children}</div>,
}));

// We need to mock minimal implementation of other dependencies that CodeSnippet might use
jest.mock('@ably/ui/core/Icon', () => {
return {
__esModule: true,
default: ({ name, size, additionalCSS, color }: any) => (
<span
data-testid={`icon-${name}`}
className={`${additionalCSS || ''} ${color || ''}`}
style={{ width: size, height: size }}
>
{name}
</span>
),
};
});

// Mock Code component used by CodeSnippet
jest.mock('@ably/ui/core/Code', () => {
return {
__esModule: true,
default: ({ language, snippet }: any) => (
<pre data-testid={`code-${language}`}>
<code>{snippet}</code>
</pre>
),
};
});

describe('MDX component integration', () => {
it('renders basic content correctly', () => {
render(
<div>
<h1>Test Heading</h1>
<p>Test paragraph</p>
</div>,
);

expect(screen.getByText('Test Heading')).toBeInTheDocument();
expect(screen.getByText('Test paragraph')).toBeInTheDocument();
});

it('conditionally renders content with If component', () => {
render(
<div>
<If lang="javascript">This should be visible</If>
<If lang="ruby">This should not be visible</If>
</div>,
);

expect(screen.getByText('This should be visible')).toBeInTheDocument();
expect(screen.queryByText('This should not be visible')).not.toBeInTheDocument();
});

it('renders code snippets with different languages (JavaScript active)', () => {
render(
<div>
<CodeSnippet>
<pre>
<code className="language-javascript">
{`var ably = new Ably.Realtime('API_KEY');
var channel = ably.channels.get('channel-name');

// Subscribe to messages on channel
channel.subscribe('event', function(message) {
console.log(message.data);
});`}
</code>
</pre>
<pre>
<code className="language-swift">
{`let realtime = ARTRealtime(key: "API_KEY")
let channel = realtime.channels.get("channel-name")

// Subscribe to messages on channel
channel.subscribe("event") { message in
print(message.data)
}`}
</code>
</pre>
</CodeSnippet>
</div>,
);

const javascriptElement = screen.queryByTestId('code-javascript');
const swiftElement = screen.queryByTestId('code-swift');

expect(javascriptElement).toBeInTheDocument();
expect(swiftElement).not.toBeInTheDocument();
});

it('renders code snippets (TypeScript active)', () => {
render(
<div>
<CodeSnippet lang="typescript">
<pre>
<code className="language-javascript">
{`var ably = new Ably.Realtime('API_KEY');
var channel = ably.channels.get('channel-name');

// Subscribe to messages on channel
channel.subscribe('event', function(message) {
console.log(message.data);
});`}
</code>
</pre>
<pre>
<code className="language-typescript">
{`const ably = new Ably.Realtime('API_KEY');
const channel = ably.channels.get('channel-name');

// Subscribe to messages on channel
channel.subscribe('event', (message: Ably.Types.Message) => {
console.log(message.data);
});`}
</code>
</pre>
</CodeSnippet>
</div>,
);

const javascriptElement = screen.queryByTestId('code-javascript');
const typescriptElement = screen.queryByTestId('code-typescript');

expect(javascriptElement).not.toBeInTheDocument();
expect(typescriptElement).toBeInTheDocument();
});
});
Loading