add a `id` attribute while visiting a tree using `unist-util-visit`? #88

deadcoder0904 · 2022-10-28T06:29:01Z

deadcoder0904
Oct 28, 2022

in this blog, it shows this code:

visit(tree, 'element', function (node) {
  if ( node.tagName === 'h2' ) {
    const id = parameterize(node.children[0].value);
    node.properties.id = id;
  }
});

i tried it, it doesn't work. mabye because it was for previous versions.

i also tried doing node.children = [{properties: {id: '' }}] but that didn't work either.

files.forEach((file) => {
	const tree = unified().use(remarkParse).parse(file.content)

	visit(tree, (node) => {
		if (node.type === 'heading') {
			const slug = slugger.slug(node.children[0].value)
			node.children = [
				{
					type: 'element',
					tagName: `h${node.depth}`,
					properties: { id: slug },
					children: [{ type: 'text', value: title }],
				},
			]
			console.log(JSON.stringify(node.children))
			const noOfHastags = '#'.repeat(node.depth)
			const href = `#${slug}`
			const value = `${noOfHastags} ${node.children[0].value}`
			lis.push({ href, value })
		}
	})
})

what should i do to edit a single attribute id?

the reason i need this is because github-slugger appends -1 if you visit the same node twice & i visit it using this & rehype-rewrite. lmk if there's a way to disable that in github-slugger which i think can be feature :)

Answered by deadcoder0904

Oct 29, 2022

used unist-util-visit to add id to headers. this blog was useful in understanding mdast & hast structure.

import { visit } from 'unist-util-visit'

const processor = await unified()
	.use(remarkParse, { fragment: true })
	.use(remarkRehype)
	.use(rehypeDocument, {
		title,
		link,
	})
	.use(() => (tree) => {
		visit(tree, 'element', (node) => {
			const headers = ['h1', 'h2', 'h3', 'h4', 'h5', 'h6']
			if (headers.includes(node.tagName)) {
				const title = toString(node.children)
				const slug = slugger.slug(title)
				node.properties.id = slug
			}
		})
	})

View full answer

wooorm · 2022-10-28T07:45:36Z

wooorm
Oct 28, 2022
Maintainer

You might notice from the reference code, and your code, that they are different.
You might also notice from the reference post’s dependencies, and your dependencies, that they are different too.
The difference is that you are working on markdown, with mdast and remark.
They are working on HTML, with hast and rehype.
This is mdast: https://github.com/syntax-tree/mdast.
This is hast: https://github.com/syntax-tree/hast.
More on why there are different ecosystems and ASTs is described here: https://github.com/remarkjs/remark-rehype#what-is-this.

If you’re goal is to add HTML attributes, you are dealing with HTML, and you need to do things in that space: in rehype, working on hast.

You are also doing some other things:

Serializing a node: please don’t do that with node.children[0].value. It breaks when: a) the heading starts with emphasis/strong/links/images/etc b) there are multiple children. The solution is to use mdast-util-to-string or hast-util-to-string
Creating a slug: you should likely use github-slugger, and benefit from its support of generating different slugs, because ids must be unique. If you do want duplicate IDs, github-slugger supports that too.
Manually generating a string of markdown, which will fail for code, escapes, images, links, etc in headings: use mdast-util-to-markdown to turn an AST into a string of markdown

Last, I really recommend to stop using rehype-rewrite. Use unist-util-visit in your own plugins instead. visit gives you much more power, which you seem to need.

8 replies

deadcoder0904 Oct 29, 2022
Author

my final code...lmk if u still find any issues.

/**
 * node imports
 */
import fs from 'node:fs'
import path from 'node:path'

/**
 * unified imports
 */
import { unified } from 'unified'
import { read, write } from 'to-vfile'
import { visit } from 'unist-util-visit'

/**
 * remark imports
 */
import { toString } from 'mdast-util-to-string'
import { toMarkdown } from 'mdast-util-to-markdown'
import remarkRehype from 'remark-rehype'
import remarkParse from 'remark-parse'

/**
 * rehype imports
 */
import rehypeParse from 'rehype-parse'
import rehypeStringify from 'rehype-stringify'
import rehypeDocument from 'rehype-document'
import rehypeFormat from 'rehype-format'
import rehypeAddClasses from 'rehype-add-classes'

/**
 * npm imports
 */
import GithubSlugger from 'github-slugger'
import { copyFile } from 'cp-file'
import fg from 'fast-glob'

/**
 * local imports
 */
import bookConfig from '../book.config.js'

const readMarkdownFiles = async (mdFilePaths, root) => {
  return await Promise.all(
    mdFilePaths.map(async (mdPath) => {
      const content = await read(path.join(root, mdPath), {
        encoding: 'utf8',
      })
      return { path: mdPath, content: content.value }
    })
  )
}

const main = async () => {
  const { title, author, cover, tocTitle, root, entry, output } = bookConfig
  const slugger = new GithubSlugger()

  const DEST_PATH = path.join(root, '..', 'public')

  // copy all images from `_book/` folder to `public/` folder
  const images = fg.sync([`${root}/**/*.{jpg,png}`], {
    ignore: ['**/node_modules/**'],
  })

  images.forEach(async (image) => {
    const filename = image.split('/').pop()
    await copyFile(image, path.join(DEST_PATH, filename))
  })

  // copy `index.css` to `public/` folder
  const cssFileName = 'index.css'
  const css = fg.sync([path.join(root, '..', cssFileName)], {
    ignore: ['**/node_modules/**'],
  })[0]

  await copyFile(css, path.join(DEST_PATH, cssFileName))

  const files = await readMarkdownFiles(entry, root)

  const markup = files.map((file) => file.content).join('\n\n')

  const link = []
  let lis = []
  let n = 1

  const processor = await unified()
    .use(remarkParse, { fragment: true })
    .use(remarkRehype)
    .use(rehypeDocument, {
      title,
      link,
    })
    .use(() => (tree) => {
      visit(tree, 'element', (node) => {
        const headers = ['h1', 'h2', 'h3', 'h4', 'h5', 'h6']
        if (headers.includes(node.tagName)) {
          const title = toString(node.children)
          const slug = slugger.slug(title)
          const noOfHastags = '#'.repeat(Number(node.tagName.replace('h', '')))
          const href = `#${slug}`
          const value = `${noOfHastags} ${title}`
          lis.push({ href, value })
          node.properties.id = slug
        }
      })
    })
    .use(() => (tree) => {
      visit(tree, 'element', (node) => {
        if (node.tagName === 'body') {
          node.children = [
            {
              type: 'element',
              tagName: 'h1',
              properties: {},
              children: [{ type: 'text', value: title }],
            },
            {
              type: 'element',
              tagName: 'nav',
              properties: {
                id: 'toc',
                role: 'doc-toc',
              },
              children: [
                {
                  type: 'element',
                  tagName: 'h2',
                  properties: {},
                  children: [
                    { type: 'text', value: tocTitle || 'table of contents' },
                  ],
                },
                {
                  type: 'element',
                  tagName: 'ol',
                  properties: {},
                  children: lis.map((li) => {
                    const text = li.value.replaceAll('#', '').trim()
                    const depth = li.value.match(/#/g || []).length

                    let properties = {
                      href: li.href,
                      class: '',
                    }

                    if (depth > 1) {
                      properties = {
                        href: li.href,
                        class: `ml-${(depth - 1) * 3}`,
                      }
                    }

                    return {
                      type: 'element',
                      tagName: 'li',
                      properties: {},
                      children: [
                        {
                          type: 'element',
                          tagName: 'a',
                          properties,
                          children: [
                            {
                              type: 'text',
                              value: text,
                            },
                          ],
                        },
                      ],
                    }
                  }),
                },
              ],
            },
            ...node.children,
          ]
        }
      })
    })
    .use(rehypeFormat)
    .use(rehypeStringify)

  try {
    const file = await processor.process(markup)
    if (!fs.existsSync(DEST_PATH)) {
      fs.mkdirSync(DEST_PATH)
    }
    file.path = path.join(DEST_PATH, 'index')
    file.extname = '.html'
    await write(file)
  } catch (error) {
    throw error
  }
}

main()

wooorm Oct 29, 2022
Maintainer

Your read file function can be:

const readMarkdownFiles = async (filePaths, root) => {
  return await Promise.all(filePaths.map(filePath => read(path.join(root, filePath))))
}

That reads all files with vfile.read. And returns the files. You need to change the other code to not get fileObj.content anymore though, instead getting file.value.

You likely don’t need copyFile. Node’s fs can also copy files.

You can use hast-util-heading-rank to check if something is a heading. You use that rank later anyway.

toMarkdown is still better at generating markdown than `${noOfHastags} ${title}`.
But see my note a few down: you don’t need hashes here.

You can use hastscript to more easily create elements. Objects for elements are fine as well though. It’s more a question of how often you do it, when it becomes useful.

Looks like there is some code missing that defines title, tocTitle?

Your lis.map code undos the work you did before. If you don’t need the hashes, don’t put them in.

DO NOT USE class. And don’t set it to a single string. hast uses className, and it is an array:

let properties = {
    href: li.href,
    className: depth > 1 ? [`ml-${(depth - 1) * 3}`] : [],
}

You can replace your final file writing code with:

file.path = path.join(DEST_PATH, 'index.html')
await fs.promises.mkdir(file.dirname, {recursive: true})
await write(file)

You could use top-level await, the main wrapper is not needed.

deadcoder0904 Oct 30, 2022
Author

implemented all your suggestions. didn't need toMarkdown at all. title & tocTitle are in a file named book.config.js with a default export. absolutely loved hastscript as it reduced so much code.

/**
 * node imports
 */
import fs from 'node:fs'
import path from 'node:path'

/**
 * unified imports
 */
import { unified } from 'unified'
import { read, write } from 'to-vfile'
import { visit } from 'unist-util-visit'

/**
 * remark imports
 */
import { toString } from 'mdast-util-to-string'
import remarkRehype from 'remark-rehype'
import remarkParse from 'remark-parse'

/**
 * rehype imports
 */
import { h } from 'hastscript'
import { headingRank } from 'hast-util-heading-rank'
import rehypeParse from 'rehype-parse'
import rehypeStringify from 'rehype-stringify'
import rehypeDocument from 'rehype-document'
import rehypeFormat from 'rehype-format'
import rehypeAddClasses from 'rehype-add-classes'

/**
 * npm imports
 */
import GithubSlugger from 'github-slugger'
import fg from 'fast-glob'

/**
 * local imports
 */
import bookConfig from '../book.config.js'

const { slug } = GithubSlugger

const copyFile = async (src, dest) => {
  await fs.promises.copyFile(src, dest)
}

const readMarkdownFiles = async (filePaths, root) => {
  return await Promise.all(
    filePaths.map(async (filePath) => read(path.join(root, filePath)))
  )
}

const { title, author, cover, tocTitle, root, entry, output } = bookConfig

const DEST_PATH = path.join(root, '..', 'public')

// copy all images from `_book/` folder to `public/` folder
const images = fg.sync([`${root}/**/*.{jpg,png}`], {
  ignore: ['**/node_modules/**'],
})

images.forEach(async (image) => {
  const filename = image.split('/').pop()
  await copyFile(image, path.join(DEST_PATH, filename))
})

// copy `index.css` to `public/` folder
const cssFileName = 'index.css'
const css = fg.sync([path.join(root, '..', cssFileName)], {
  ignore: ['**/node_modules/**'],
})[0]

await copyFile(css, path.join(DEST_PATH, cssFileName))

const files = await readMarkdownFiles(entry, root)

const markup = files.map((file) => file.value).join('\n\n')

const link = [
  {
    rel: 'stylesheet',
    href: 'https://rsms.me/inter/inter.css',
  },
  {
    rel: 'stylesheet',
    href: './index.css',
  },
]

let lis = []
let n = 1

const processor = await unified()
  .use(remarkParse, { fragment: true })
  .use(remarkRehype)
  .use(rehypeDocument, {
    title,
    link,
  })
  .use(() => (tree) => {
    visit(tree, 'element', (node) => {
      const headers = ['h1', 'h2', 'h3', 'h4', 'h5', 'h6']
      if (headers.includes(node.tagName)) {
        const title = toString(node.children)
        const depth = headingRank(node)
        const noOfHastags = '#'.repeat(depth)
        lis.push({ title, depth })
        node.properties.id = `#${slug(title)}`
      }
    })
  })
  .use(() => (tree) => {
    visit(tree, 'element', (node) => {
      if (node.tagName === 'body') {
        node.children = [
          h('h1', { className: 'frontcover' }, title),
          h('nav#toc', { role: 'doc-toc' }, [
            h('h2', tocTitle || 'table of contents'),
            h(
              'ol',
              lis.map((li) => {
                return h(
                  'li',
                  {
                    className: li.depth > 1 ? [`ml-${(li.depth - 1) * 3}`] : [],
                  },
                  [
                    h(
                      'a',
                      {
                        href: `#${slug(li.title)}`,
                      },
                      li.title
                    ),
                  ]
                )
              })
            ),
          ]),
          ...node.children,
        ]
      }
    })
  })
  .use(rehypeFormat)
  .use(rehypeStringify)

try {
  const file = await processor.process(markup)
  file.path = path.join(DEST_PATH, 'index.html')
  await fs.promises.mkdir(file.dirname, { recursive: true })
  await write(file)
} catch (error) {
  throw error
}

regarding last code block: will it create public/ folder if it doesn't exist in this code:

file.path = path.join(DEST_PATH, 'index.html')
await fs.promises.mkdir(file.dirname, {recursive: true})
await write(file)

because that's why i had mkdirSync above the file.path.

if (!fs.existsSync(DEST_PATH)) {
  fs.mkdirSync(DEST_PATH)
}
file.path = path.join(DEST_PATH, 'index')
file.extname = '.html'

wooorm Oct 31, 2022
Maintainer

will it create public/ folder if it doesn't exist

recursive: true does that. It creates all “in-between” folders if they don’t exist yet.

Code looks good btw!:+1:

deadcoder0904 Nov 3, 2022
Author

ohh, i forgot the file isn't written until the 3rd line. thanks for the help :)

deadcoder0904 · 2022-10-29T04:35:38Z

deadcoder0904
Oct 29, 2022
Author

used unist-util-visit to add id to headers. this blog was useful in understanding mdast & hast structure.

import { visit } from 'unist-util-visit'

const processor = await unified()
	.use(remarkParse, { fragment: true })
	.use(remarkRehype)
	.use(rehypeDocument, {
		title,
		link,
	})
	.use(() => (tree) => {
		visit(tree, 'element', (node) => {
			const headers = ['h1', 'h2', 'h3', 'h4', 'h5', 'h6']
			if (headers.includes(node.tagName)) {
				const title = toString(node.children)
				const slug = slugger.slug(title)
				node.properties.id = slug
			}
		})
	})

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

syntax-tree

add a `id` attribute while visiting a tree using `unist-util-visit`? #88

{{title}}

Replies: 2 comments 8 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

syntax-tree

add a id attribute while visiting a tree using unist-util-visit? #88

deadcoder0904 Oct 28, 2022

Replies: 2 comments · 8 replies

wooorm Oct 28, 2022 Maintainer

deadcoder0904 Oct 29, 2022 Author

wooorm Oct 29, 2022 Maintainer

deadcoder0904 Oct 30, 2022 Author

wooorm Oct 31, 2022 Maintainer

deadcoder0904 Nov 3, 2022 Author

deadcoder0904 Oct 29, 2022 Author

add a `id` attribute while visiting a tree using `unist-util-visit`? #88

deadcoder0904
Oct 28, 2022

Replies: 2 comments 8 replies

wooorm
Oct 28, 2022
Maintainer

deadcoder0904 Oct 29, 2022
Author

wooorm Oct 29, 2022
Maintainer

deadcoder0904 Oct 30, 2022
Author

wooorm Oct 31, 2022
Maintainer

deadcoder0904 Nov 3, 2022
Author

deadcoder0904
Oct 29, 2022
Author