Background

Markdown flow

Before I discuss exactly what the purpose of this endeavor was, here's some context. I use markdown to write my blogs, a simple rich-text language format that doesn't pack in too much support for that type of customization. Markdown is nice because it lets you define in plain-text things like images, headings, tables, and more, but at the end of the day it's just a syntax for writing rich text; it's the job of the renderer to actually display it in its final form.

A simple markdown script may look something like this, with various headings and graphics. Here's more information on the syntax if you're interested.

1# Heading
2## Subheading
3### Subsubheading
4![imgName](imgSrc)
5[link](src)
6
7|column1 | column2|
8|--------| -------|

Part of what makes Markdown so popular is the fact that once you've finished writing markdown, there's many different frameworks in many different programming languages that allow for rendering the file. On the web, this usually means converting markdown text into HTML, so as to display the various elements visually. These frameworks abstract-ify the process, and make it as easy as inserting a few lines of code to get the markdown to render properly. Unfortunately for me, for my blogs I wasn't quite content with the default configuration of my renderer.

Markdown Images

My ultimate problem with my renderer was related to how it displayed images. In regular markdown, to include an image you use ![alt](src) syntax. Then, when rendered, markdown places an image in between the text above and below the image declaration. When I write blogs, however, I often want to include many different images of the same thing, or same content. A grid of graphics to showcase every perspective of an object or a sequence of photos to display a process. 'Vanilla' markdown on its own does not provide any other level of customization, but after some research I did determine that there are indeed some external solutions, outlined below, of which none quite met my needs.

Flavored Markdown

Some flavors of markdown, such as Github flavored markdown (GFL), allow for some limited customization with particular inline html syntax. For example, with GFL you can write <img align="left" src="img.jpg"> to define a left floating image, taking advantage of the special align prop. Other customizations are allowed inline, like setting the dimensions of images. To specify a specific pixel width of an image in markdown, you can use ![alt](src|width=123) syntax.

MDX Markdown

MDX markdown is a special type of markdown file that is denoted with the file extension .mdx. With MDX markdown, you can include React components inline, much like you can with JSX. JSX is a syntax extension to javascript that allows for inline HTML, and MDX is a syntax extension for JSX that allows for inline markdown. Without getting too far into the details, with MDX markdown you could theoretically implement some sort of custom image component, and use tags instead of markdown image syntax to place the customized images.

The Problem

This issue with these methods is that in all these various methods of customizing markdown inline HTML in some form or another is involved. This takes away from the beauty of vanilla markdown, since to customize how images render I have to type out tags and deal with properties to some extent or another. Sure, it can be minimized with proper planning and implementation, but only to an extent. What I like is simplicity, and so I decided to implement my own solution to get images to render my way. Perhaps the task was unnecessarily overkill, but gaining a better grasp of the rendering process was really rewarding, and extremely interesting.

The goal

For the final product, I decided to apply two different alterations to vanilla markdown rendering. Firstly, I wanted to be able to include "blocks" of images, so that I could cluster images together; secondly, I wanted regular images to be customizable with inline properties (more on this soon).

Image blocks

My decision, before even beginning to research how to go about it, was to have it so that markdown like this...

1# This is a header
2
3![This is an image](src1.png)
4![This is an image](src2.png)
5![This is an image](src3.png)
6
7This is a paragraph within the markdown.
8![This is a regular image](src4.png)

Image carousel

Image properties

Additionally, I decided that it would be best to allow for all image customization to occur in the definition of the images themselves, and, to ensure consistency with GFM, I chose to take advantage of the absolute-value-equals-sign syntax. All images could by default float to the right, but I'd want to be able to change their floating direction to the left by writing out something along the lines of ![alt](src|float=left). For now I chose to include the ability to customize the float and width of images, but it was important to me to design the system to allow for future extension.

Implementation

My website's frontend is currently created with React, a framework that allows for component-based web UI development. To implement a feature like markdown rendering into a react website, the common first step would be to do a quick Google to see if any components exist that do what I need, since many extensions to React are component-libraries. As it turns out, there indeed existed a component for the job: react-markdown.

React markdown

React markdown is a library that is essentially a wrapper on the unified ecosystem, a javascript package that implements abstract syntax trees (ASTs). unified allows for many different extensions, including a very popular one called remark to parse a markdown string into a unified AST, and another popular one called rehype for HTML parsing. To understand how react markdown works, it's important to understand how to render markdown to raw HTML with unified first. For this, let's take a look at a super simple unified script that I wrote during the tinkering phase of this project, which is along the lines of the boilerplate that ReactMarkdown conceals.

1...
2const buffer = fs.readFileSync("./example.md");
3
4retext()
5    .use(remarkParse)
6    .use(remarkRehype)
7    .use(rehypeStringify)
8    .process(buffer, (err, file) => {
9        const rendered = prettify(file.value);
10        console.log(rendered);
11        fs.writeFileSync("./example.html", rendered)
12    });

Here we begin with plaintext and load it into retext, a part of the unified collective that is meant for raw text. Then we build up a chain of plugins to funnel our markdown through. First we use remarkParse to convert the markdown string into a mdast, a markdown AST. Then we use remarkRehype to mutate that tree into a hast, a HTML AST, and finally we use rehypeStringify to deconstruct the tree into an HTML string, which we finally dump into a file.

Each of the things that we ".use()" is a javascript function that takes the tree and, optionally, file metadata as input, does stuff with it (like mutating the tree), and then spits back out the tree to be used by the next extension in the chain. The render is then stored in a VFile object, from which the output HTML string can be extracted.

React markdown makes this process super-simple, since it handles this process for you, and then also parses the output HTML string into React components. It can be used by simply passing the markdown in as the child element, as so...

1<ReactMarkdown>
2# Hello, *world*!
3This is an example of how to use react-markdown.
4</ReactMarkdown>

It only uses the bare minimum ".use()"s to convert the markdown into HTML, but it does allow you to pass remarkPlugins and rehypePlugins to further customize the mdast and hast ASTs. They provide a nice diagram of this process.

1                                                           react-markdown
2         +----------------------------------------------------------------------------------------------------------------+
3         |                                                                                                                |
4         |  +----------+        +----------------+        +---------------+       +----------------+       +------------+ |
5         |  |          |        |                |        |               |       |                |       |            | |
6markdown-+->+  remark  +-mdast->+ remark plugins +-mdast->+ remark-rehype +-hast->+ rehype plugins +-hast->+ components +-+->react elements
7         |  |          |        |                |        |               |       |                |       |            | |
8         |  +----------+        +----------------+        +---------------+       +----------------+       +------------+ |
9         |                                                                                                                |
10         +----------------------------------------------------------------------------------------------------------------+

By understanding that the library is simply a wrapper atop a much more complex process, it becomes critical to understand the actual render process itself. For example, at first the optional prop remarkRehypeOptions makes no sense, but as will soon become apparent, being able to set options for the remarkRehype stage in the render process (which will be discussed soon) is critical to implementing my custom image blocks.

Another important part of react markdown that I noticed in my research was concerned was the ability to set component overrides. You see, once react markdown has reached the final stage of rendering and has an HTML string, it then proceeds to convert that HTML into React components. It does this by using sensical defaults, such as the <img> component for images, <p> for paragraphs, and the like. However, the library allows you to set overrides on components, by defining your own components and then setting it in the options of the ReactMarkdown component. Here's the example their documentation provides...

1<ReactMarkdown
2  components={{
3    // Map `h1` (`# heading`) to use `h2`s.
4    h1: 'h2',
5    // Rewrite `em`s (`*like so*`) to `i` with a red foreground color.
6    em: ({node, ...props}) => <i style={{color: 'red'}} {...props} />
7  }}
8/>

The aha moment

It seemed that to incorporate clustered images, the simplest solution would be to have the groups of images in my markdown turn into some sort of fake HTML component with props including all the images data, so that I could then define an override for that component, to render it the way I wanted. In other words, if a cluster of images in markdown could transform into something of the form <imgBlock alts="alt1;alt2;alt3" srcs="src1;src2;src3"/> I could then define a React component of the form ImgBlock (alts, srcs) { ... } to handle the rendering. The hard part would be to locate and modify the rendering process to have the output string include those custom <ImgBlock>s

Render process

Now, before adding any custom plugins or alterations, let's take a look at the example I was working with, and discuss how the rendering flow actually works.

For the initial tests, the input looked like this...

1# An example document with a multiimage
2
3Test1
4
5![Image #1](https://src-to-img-1.com)
6![Image #2](https://src-to-img-2.com)
7![Image #3](https://src-to-img-2.com)
8![Image #4](https://src-to-img-2.com)
9
10Test2
11![Normal Image](https://src-to-normal-img.com|float=left|width=50)
12
13![Normal Image But Isolated](https://src-to-normal-img.com|width=35|float=right)

And the output like this...

1<h1>An example document with a multiimage</h1>
2<p>Test1</p>
3<p>
4    <img src="https://src-to-img-1.com" alt="Image #1" />
5    <img src="https://src-to-img-2.com" alt="Image #2" />
6    <img src="https://src-to-img-2.com" alt="Image #3" />
7    <img src="https://src-to-img-2.com" alt="Image #4" />
8</p>
9<p>
10    Test2 <img src="https://src-to-normal-img.com%7Cfloat=left%7Cwidth=50" alt="Normal Image" />
11</p>
12<p>
13    <img
14        src="https://src-to-normal-img.com%7Cwidth=35%7Cfloat=right"
15        alt="Normal Image But Isolated"
16    />
17</p>

Creating a plugin

Creating a plugin would involve simply adding a function to the chain. Part of what makes unified so great is that it provides tons of utilities that allow you to easily locate parents, traverse specific parts of the tree, search for nodes, and more. For my purpose, I'd just need unified-visit to walk the tree, which handily provides the ability to walk all nodes that pass a specific filter only. Looking back at the example, it appeared to be the case that all clustered images were determined to be the children of a paragraph, so I'd only care about paragraph nodes. To start, I wrote the plugin to just print them out, to see what I was working with.

1{
2  type: 'paragraph',
3  children: [ { type: 'text', value: 'Test1', position: [Object] } ],
4  position: {
5    start: { line: 3, column: 1, offset: 43 },
6    end: { line: 3, column: 6, offset: 48 }
7  }
8}
9{
10  type: 'paragraph',
11  children: [
12    {
13      type: 'image',
14      title: null,
15      url: 'https://src-to-img-1.com',
16      alt: 'Image #1',
17      position: [Object]
18    },
19    { type: 'text', value: '\r\n', position: [Object] },
20    {
21      type: 'image',
22      title: null,
23      url: 'https://src-to-img-2.com',
24      alt: 'Image #2',
25      position: [Object]
26    },
27    { type: 'text', value: '\r\n', position: [Object] },
28    {
29      type: 'image',
30      title: null,
31      url: 'https://src-to-img-2.com',
32      alt: 'Image #3',
33      position: [Object]
34    },
35    { type: 'text', value: '\r\n', position: [Object] },
36    {
37      type: 'image',
38      title: null,
39      url: 'https://src-to-img-2.com',
40      alt: 'Image #4',
41      position: [Object]
42    }
43  ],
44  position: {
45    start: { line: 5, column: 1, offset: 52 },
46    end: { line: 8, column: 38, offset: 206 }
47  }
48}
49{
50  type: 'paragraph',
51  children: [
52    { type: 'text', value: 'Test2\r\n', position: [Object] },       
53    {
54      type: 'image',
55      title: null,
56      url: 'https://src-to-normal-img.com|float=left|width=50',     
57      alt: 'Normal Image',        
58      position: [Object]
59    }
60  ],
61  position: {
62    start: { line: 10, column: 1, offset: 210 },
63    end: { line: 11, column: 67, offset: 283 }
64  }
65}
66{
67  type: 'paragraph',
68  children: [
69    {
70      type: 'image',
71      title: null,
72      url: 'https://src-to-normal-img.com|width=35|float=right',    
73      alt: 'Normal Image But Isolated',
74      position: [Object]
75    }
76  ],
77  position: {
78    start: { line: 13, column: 1, offset: 287 },
79    end: { line: 13, column: 81, offset: 367 }
80  }
81}

Looking at the output, I identified the image block to be the paragraph node on line 10, which I immediately started scouring for patterns that I may be able to check against. As can be seen, images on subsequent lines are pretty easily identifiable: the children of the parent paragraph element simply contains alternating image and text tags with text having a value of \r\n. Here's the plugin I wrote to swap out the alternating images and texts with image block nodes...

1import { visit } from "unist-util-visit";
2
3export default function remarkImageBlock() {
4    return (tree, file) => {
5        visit(tree, "paragraph", (node, index, parent) => {
6            if (node.children) {
7                const children = node.children.filter(
8                    child => !(child.type === "text" && child.value === "\r\n")
9                )
10                if (children && children.every(child => child.type === "image") && children.length > 1) {
11                    parent.children[index] = {
12                        type: "imgBlock",
13                        alts: children.map(child => child.alt),
14                        srcs: children.map(
15                            child =>
16                                child.url.includes("|") ? child.url.split("|")[0] : child.url
17                        ),
18                        titles: children.map(child => child.title),
19                        properties: children.map(
20                            child => child.url.includes("|") ? child.url.split("|").slice(1) : ""
21                        ),
22                        position: {
23                            start: children[0].position.start,
24                            end: children.slice(-1)[0].position.end
25                        },
26                        children: []
27                    }
28                }
29            }
30        });
31        return tree;
32    };
33}

While iterating over all the paragraph nodes, we get rid of all the text child nodes that have a value of \r\n, and then check to see if there's more than one child image left. If there is, we've located an image block, and swap the paragraph out for a custom imgBlock node by mutating the parent of the paragraph's children at paragraph's index.

Now, I adjusted the unified flow to include the new plugin.

1retext()
2    .use(remarkParse)
3    .use(remarkImageBlock)
4    .use(remarkRehype)
5    .use(rehypeStringify)
6    .process(buffer, (err, file) => {
7        fs.writeFileSync("./example.html", file.value)
8    });

Before changing the mdast (markdown AST) to a hast (HTML AST), I modify the mdast with my plugin to create imgBlock nodes where needed, swapping out old paragraph nodes. But upon printing the output, it didn't quite work.

Where did the imgBlocks go?

Instead of having the fake HTML <imgBlock>s that I expected, I got an empty div.

1<h1>An example document with a multiimage</h1>
2<p>Test1</p>
3<div></div>
4<p>
5    Test2 <img src="https://src-to-normal-img.com%7Cfloat=left%7Cwidth=50" alt="Normal Image" />
6</p>
7<p>
8    <img
9        src="https://src-to-normal-img.com%7Cwidth=35%7Cfloat=right"
10        alt="Normal Image But Isolated"
11    />
12</p>

But, when printing out the tree at the stage before remarkRehype, it looked like this...

1{
2  type: 'root',
3  children: [
4    {
5      type: 'heading',
6      depth: 1,
7      children: [Array],
8      position: [Object]
9    },
10    { type: 'paragraph', children:
11 [Array], position: [Object] },   
12    {
13      type: 'imgBlock',
14      alts: [Array],
15      srcs: [Array],
16      titles: [Array],
17      properties: [Array],        
18      position: [Object],
19      children: []
20    },
21    { type: 'paragraph', children:
22 [Array], position: [Object] },   
23    { type: 'paragraph', children:
24 [Array], position: [Object] }    
25  ],
26  position: {
27    start: { line: 1, column: 1, offset: 0 },
28    end: { line: 13, column: 81, offset: 367 }
29  }
30}

The culprit

It turned out that the imgBlocks were getting lost during the remarkRehype stage, since remarkRehype didn't know how to handle the foreign imgBlock nodes. When remarkRehype converts the mdast to an hdast it uses special predefined handlers, which can all be found here, and take the form of something along the lines of this...

1import {normalizeUri} from 'micromark-util-sanitize-uri'
2
3export function image(state, node) {
4  const properties = {src: normalizeUri(node.url)}
5
6  if (node.alt !== null && node.alt !== undefined) {
7    properties.alt = node.alt
8  }
9
10  if (node.title !== null && node.title !== undefined) {
11    properties.title = node.title
12  }
13
14  const result = {type: 'element', tagName: 'img', properties, children: []}
15  state.patch(node, result)
16  return state.applyData(node, result)
17}

Each has slight modifications based to port the markdown to HTML, but the general format is the same. In order to create an <imgBlock> HTML component, I'd need to define my own handler, and pass it into the remarkRehype plugin. Creating the handler was simple, since I could just reuse a similar schema to the above <img> handler definition.

At this stage, since I already had access to the alts, srcs, and titles of the images, I thought I mine as well scrape out properties from the src. That is, an image that was spelled out in markdown like ![alt](src.png|width=50|float=left) would have its src be src.png|width=50|float=left. This would be a good point to strip off those extra image configs and pass them down the chain as a separate properties prop.

1export function imgBlockHandler(state, node, parent) {
2    return {
3        type: "element",
4        tagName: "imgBlock",
5        properties: {
6            alts: node.alts.join(";"),
7            srcs: node.srcs.map(normalizeUri).join(";"),
8            titles: node.titles.join(";"),
9            properties: node.properties.join(";")
10        },
11        children: []
12    }
13}
14
15export function imgHandler(state, node, parent) {
16    const properties = {}
17    for (const property of node.url.matchAll(/((\w+)=(\w+))/g)) {
18        properties[property[2]] = property[3]
19    }
20    const url = /\w*|\w=*/.test(node.url) ? node.url.match(/([^\s^\|]*)\|/)[1] : node.url
21
22    return {
23        type: "element",
24        tagName: "img",
25        properties: {
26            alt: node.alt,
27            src: url,
28            title: node.title,
29            ...properties,  // like float: left, width: 50, etc.
30        },
31        children: []
32    }
33}

I decided to split the alts, srcs, and titles with semicolons, since I wanted to allow for an arbitrary number of images in a block. Since the markdown would become a html string before becoming a react components, any javascript privatives like Arrays would get lost, so this seemed to be the best solution.

I modified theunified script to be as follows, including these custom handlers in the mdast to hdast conversion process...

1retext()
2    .use(remarkParse)
3    .use(remarkImageBlock)
4    .use(remarkRehype, {
5        handlers: { imgBlock: imgBlockHandler, image: imgHandler },
6    })
7    .use(rehypeStringify)
8    .process(buffer, (err, file) => {
9        fs.writeFileSync("./example.html", file.value)
10    });

And now, with the custom handlers in place, I executed the unified script, and vola!

1<h1>An example document with a multiimage</h1>
2<p>Test1</p>
3<imgBlock
4    alts="Image #1;Image #2;Image #3;Image #4"
5    srcs="https://src-to-img-1.com;https://src-to-img-2.com;https://src-to-img-2.com;https://src-to-img-2.com"
6    titles=";;;"
7    properties=";;;"
8></imgBlock>
9<p>
10    Test2 <img alt="Normal Image" src="https://src-to-normal-img.com" float="left" width="50" />
11</p>
12<p>
13    <img
14        alt="Normal Image But Isolated"
15        src="https://src-to-normal-img.com"
16        width="35"
17        float="right"
18    />
19</p>

Success!

React component override

Now all that there was left to do was to define a react component to override the fake <imgBlock>s with, since browsers on their own don't know how to deal with them. I wrote out some react components for the purpose, and then slotted them into the components override prop of ReactMarkdown. With everythinig properly in place, the markdown rendering component now looked like this...

1<ReactMarkdown
2            children={markdown}
3            className="markdown"
4            remarkPlugins={[remarkImageBlock]}
5            remarkRehypeOptions={{
6                handlers: { imgBlock: imgBlockHandler, image: imgHandler },
7            }}
8            components={{
9                img: ({ node, ...props }) => (
10                    <MdImage
11                        alt={props.alt}
12                        title={props.title}
13                        src={props.src}
14                        width={props.width}
15                        float={props.float}
16                        label={props.alt}
17                        resourceMap={resourceMap}
18                    />
19                ),
20                imgBlock: ({ node, ...props }) => (
21                    <ImageBlock
22                        alts={props.alts}
23                        titles={props.titles}
24                        srcs={props.srcs}
25                        resourceMap={resourceMap}
26                    />
27                ),
28            }}
29        />

And it worked! This is the current renderer that I use for my website, with custom overrides for images and image blocks. I've also since added a code block rendering override, and may add another override for videos and/or gifs in the future.

Implications

As now can be seen, the process I used is extremely powerful. People have coded plugins to display inline LaTeX, display codeblocks (like the ones in my blog posts!), and so, so much more. What's unique and tricky about my process is how I take advantage of custom component overrides to add brand new components to markdown.

I can see infinite different ways that this same process could be utilized: adding brand new elements to inline markdown without requiring any inline HTML, modifying how certain paterns of elements display on screen, adding video support to markdown, and much more.