Guide contents

Lesson 4 of 12

Markdown to HTML Converter

We build mdhtml, which converts Markdown into an HTML page. It supports the subset people use most:

Markdown HTML
# Title to ###### Title <h1 id="title"> to <h6>, with anchor ids
paragraphs separated by blank lines <p>
- item, * item, 1. item <ul> / <ol>
> quoted text <blockquote>
a fenced code block with an optional language <pre><code class="language-rust">
--- <hr>
`code`, **bold**, *italic*, [text](url) <code>, <strong>, <em>, <a>

The design has two layers, the same as real Markdown engines: a block parser that looks at whole lines and decides what kind of block each belongs to, and an inline renderer that handles formatting inside a block's text.

Text
mdhtml/
└── src/
    ├── main.rs     # read a file (or a demo), write the page to stdout
    ├── inline.rs   # step 1: escaping and inline formatting
    ├── blocks.rs   # step 2: lines to a Vec<Block>
    └── render.rs   # step 3: Vec<Block> to HTML, anchors, table of contents

Step 1: Inline formatting

Text must be escaped before it goes into HTML, or a < in the input would start a tag (and user-supplied Markdown could inject scripts). Inline formatting is handled by a scanner that walks the text: at each position it checks for `, **, *, or [, and if the matching closing marker exists it emits the tag and jumps past it; otherwise it emits the character escaped. Bold, italics, and link labels can contain other formatting, so the scanner calls itself on their contents.

Rustcargo test
pub fn escape(text: &str) -> String {
    let mut out = String::with_capacity(text.len());
    for c in text.chars() {
        match c {
            '&' => out.push_str("&amp;"),
            '<' => out.push_str("&lt;"),
            '>' => out.push_str("&gt;"),
            '"' => out.push_str("&quot;"),
            _ => out.push(c),
        }
    }
    out
}

pub fn render_inline(text: &str) -> String {
    let mut out = String::new();
    let mut rest = text;
    while let Some(c) = rest.chars().next() {
        if let Some(after) = rest.strip_prefix('`') {
            if let Some(end) = after.find('`') {
                // Code spans are literal: no formatting inside, only escaping.
                out.push_str(&format!("<code>{}</code>", escape(&after[..end])));
                rest = &after[end + 1..];
                continue;
            }
        } else if let Some(after) = rest.strip_prefix("**") {
            if let Some(end) = after.find("**") {
                out.push_str(&format!("<strong>{}</strong>", render_inline(&after[..end])));
                rest = &after[end + 2..];
                continue;
            }
        } else if let Some(after) = rest.strip_prefix('*') {
            if let Some(end) = after.find('*') {
                out.push_str(&format!("<em>{}</em>", render_inline(&after[..end])));
                rest = &after[end + 1..];
                continue;
            }
        } else if let Some(after) = rest.strip_prefix('[') {
            if let Some((label, tail)) = after.split_once("](") {
                if let (false, Some((url, tail))) = (label.contains(']'), tail.split_once(')')) {
                    out.push_str(&format!("<a href=\"{}\">{}</a>", escape(url), render_inline(label)));
                    rest = tail;
                    continue;
                }
            }
        }
        // Not the start of any formatting: copy one character, escaped.
        out.push_str(&escape(&rest[..c.len_utf8()]));
        rest = &rest[c.len_utf8()..];
    }
    out
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn escapes_html() {
        assert_eq!(render_inline("a < b && c"), "a &lt; b &amp;&amp; c");
    }

    #[test]
    fn code_is_not_formatted() {
        assert_eq!(render_inline("use `**kwargs` here"), "use <code>**kwargs</code> here");
    }

    #[test]
    fn nested_formatting() {
        assert_eq!(render_inline("**bold and *italic***"), "<strong>bold and *italic</strong>*");
        assert_eq!(render_inline("[**docs**](https://doc.rust-lang.org)"), "<a href=\"https://doc.rust-lang.org\"><strong>docs</strong></a>");
    }

    #[test]
    fn unmatched_markers_are_plain_text() {
        assert_eq!(render_inline("2 * 3 = 6"), "2 * 3 = 6");
        assert_eq!(render_inline("[x] and [y](z)"), "[x] and <a href=\"z\">y</a>");
    }

    #[test]
    fn urls_are_escaped() {
        assert_eq!(render_inline("[x](\"><script>)"), "<a href=\"&quot;&gt;&lt;script&gt;\">x</a>");
    }
}
Output
running 5 tests
test tests::code_is_not_formatted ... ok
test tests::escapes_html ... ok
test tests::nested_formatting ... ok
test tests::unmatched_markers_are_plain_text ... ok
test tests::urls_are_escaped ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

**bold and *italic*** shows the limit of a simple scanner: the first ** after the opening one closes the bold, so the italic marker is left unmatched. Full CommonMark has pages of rules for such cases; for a converter of your own documents, this trade-off is fine.

Step 2: Parsing blocks

The block parser reads lines and produces a Vec<Block>. Most blocks are recognized by their first line. Paragraphs, lists, and quotes continue over several lines, and a paragraph ends at a blank line or at a line that starts a different block.

Rust
#[derive(Debug, PartialEq)]
enum Block {
    Heading { level: usize, text: String },
    Paragraph(String),
    List { ordered: bool, items: Vec<String> },
    Quote(String),
    Code { lang: Option<String>, code: String },
    Rule,
}

fn heading(line: &str) -> Option<(usize, &str)> {
    let level = line.chars().take_while(|&c| c == '#').count();
    let text = line[level..].strip_prefix(' ')?;
    (1..=6).contains(&level).then_some((level, text.trim()))
}

/// `- item`, `* item` (unordered) or `12. item` (ordered).
fn list_item(line: &str) -> Option<(bool, &str)> {
    if let Some(text) = line.strip_prefix("- ").or_else(|| line.strip_prefix("* ")) {
        return Some((false, text));
    }
    let digits = line.chars().take_while(char::is_ascii_digit).count();
    let text = line[digits..].strip_prefix(". ")?;
    (digits > 0).then_some((true, text))
}

fn starts_block(line: &str) -> bool {
    line.starts_with("```") || line.starts_with('>') || line == "---" || heading(line).is_some() || list_item(line).is_some()
}

fn parse_blocks(input: &str) -> Vec<Block> {
    let mut blocks = Vec::new();
    let mut lines = input.lines().map(str::trim_end).peekable();

    while let Some(line) = lines.next() {
        if line.trim().is_empty() {
            continue;
        }
        if let Some(lang) = line.strip_prefix("```") {
            let mut code = String::new();
            for line in lines.by_ref() {
                if line == "```" {
                    break;
                }
                code.push_str(line);
                code.push('\n');
            }
            let lang = (!lang.is_empty()).then(|| lang.to_string());
            blocks.push(Block::Code { lang, code });
        } else if let Some((level, text)) = heading(line) {
            blocks.push(Block::Heading { level, text: text.to_string() });
        } else if line == "---" {
            blocks.push(Block::Rule);
        } else if let Some((ordered, first)) = list_item(line) {
            let mut items = vec![first.to_string()];
            while let Some((same_kind, item)) = lines.peek().and_then(|next| list_item(next)) {
                if same_kind != ordered {
                    break;
                }
                items.push(item.to_string());
                lines.next();
            }
            blocks.push(Block::List { ordered, items });
        } else if line.starts_with('>') {
            let mut parts = vec![line.trim_start_matches('>').trim()];
            while let Some(next) = lines.next_if(|next| next.starts_with('>')) {
                parts.push(next.trim_start_matches('>').trim());
            }
            blocks.push(Block::Quote(parts.join(" ")));
        } else {
            let mut parts = vec![line.trim()];
            while let Some(next) = lines.next_if(|next| !next.trim().is_empty() && !starts_block(next)) {
                parts.push(next.trim());
            }
            blocks.push(Block::Paragraph(parts.join(" ")));
        }
    }
    blocks
}

fn main() {
    let doc = "\
# Release notes
Version 2.0 is out.
It is **faster**.

<h2 id="changes">Changes</h2>
- new parser
- fewer allocations
1. upgrade
2. run the tests
> Breaking: the `v1` API
> was removed.
```rust
fn main() {}
```
---
####### not a heading";

    for block in parse_blocks(doc) {
        println!("{block:?}");
    }
}
Output
Heading { level: 1, text: "Release notes" }
Paragraph("Version 2.0 is out. It is **faster**.")
Heading { level: 2, text: "Changes" }
List { ordered: false, items: ["new parser", "fewer allocations"] }
List { ordered: true, items: ["upgrade", "run the tests"] }
Quote("Breaking: the `v1` API was removed.")
Code { lang: Some("rust"), code: "fn main() {}\n" }
Rule
Paragraph("####### not a heading")

Peekable::next_if takes the next line only if it passes a test, which is exactly "continue the block while the next line belongs to it". The inline text is kept raw here; formatting happens when rendering.

Step 3: Rendering, anchors, and a table of contents

Rendering is a match over the blocks. Headings get an id so readers can link to them. The id is a slug: lowercase, spaces turned into hyphens, punctuation dropped. Two headings can produce the same slug, so a HashMap counts them and appends -1, -2, and so on, as GitHub does.

Rustcargo test
use std::collections::HashMap;

#[derive(Default)]
pub struct Slugger {
    seen: HashMap<String, usize>,
}

impl Slugger {
    pub fn slug(&mut self, heading: &str) -> String {
        let base: String = heading
            .to_lowercase()
            .chars()
            .filter_map(|c| match c {
                c if c.is_alphanumeric() => Some(c),
                ' ' | '-' | '_' => Some('-'),
                _ => None,
            })
            .collect();
        let count = self.seen.entry(base.clone()).or_insert(0);
        let slug = if *count == 0 { base } else { format!("{base}-{count}") };
        *count += 1;
        slug
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn slugs_are_lowercase_and_hyphenated() {
        let mut slugger = Slugger::default();
        assert_eq!(slugger.slug("Getting Started"), "getting-started");
        assert_eq!(slugger.slug("What's new in 2.0?"), "whats-new-in-20");
        assert_eq!(slugger.slug("Größe & Maße"), "größe--maße");
    }

    #[test]
    fn duplicates_get_a_suffix() {
        let mut slugger = Slugger::default();
        let slugs: Vec<String> = ["Example", "Example", "Example"].iter().map(|h| slugger.slug(h)).collect();
        assert_eq!(slugs, ["example", "example-1", "example-2"]);
    }
}
Output
running 2 tests
test tests::duplicates_get_a_suffix ... ok
test tests::slugs_are_lowercase_and_hyphenated ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Step 4: The complete converter

The final program renders blocks into a String with write!, collects headings for a table of contents, and wraps everything in a page. main reads the file named on the command line, or a demo document when there is none.

Rust
use std::collections::HashMap;
use std::fmt::Write as _;
use std::process::ExitCode;

// ---------- inline.rs ----------

fn escape(text: &str) -> String {
    let mut out = String::with_capacity(text.len());
    for c in text.chars() {
        match c {
            '&' => out.push_str("&amp;"),
            '<' => out.push_str("&lt;"),
            '>' => out.push_str("&gt;"),
            '"' => out.push_str("&quot;"),
            _ => out.push(c),
        }
    }
    out
}

fn render_inline(text: &str) -> String {
    let mut out = String::new();
    let mut rest = text;
    while let Some(c) = rest.chars().next() {
        if let Some(after) = rest.strip_prefix('`') {
            if let Some(end) = after.find('`') {
                out.push_str(&format!("<code>{}</code>", escape(&after[..end])));
                rest = &after[end + 1..];
                continue;
            }
        } else if let Some(after) = rest.strip_prefix("**") {
            if let Some(end) = after.find("**") {
                out.push_str(&format!("<strong>{}</strong>", render_inline(&after[..end])));
                rest = &after[end + 2..];
                continue;
            }
        } else if let Some(after) = rest.strip_prefix('*') {
            if let Some(end) = after.find('*') {
                out.push_str(&format!("<em>{}</em>", render_inline(&after[..end])));
                rest = &after[end + 1..];
                continue;
            }
        } else if let Some(after) = rest.strip_prefix('[') {
            if let Some((label, tail)) = after.split_once("](") {
                if let (false, Some((url, tail))) = (label.contains(']'), tail.split_once(')')) {
                    out.push_str(&format!("<a href=\"{}\">{}</a>", escape(url), render_inline(label)));
                    rest = tail;
                    continue;
                }
            }
        }
        out.push_str(&escape(&rest[..c.len_utf8()]));
        rest = &rest[c.len_utf8()..];
    }
    out
}

// ---------- blocks.rs ----------

enum Block {
    Heading { level: usize, text: String },
    Paragraph(String),
    List { ordered: bool, items: Vec<String> },
    Quote(String),
    Code { lang: Option<String>, code: String },
    Rule,
}

fn heading(line: &str) -> Option<(usize, &str)> {
    let level = line.chars().take_while(|&c| c == '#').count();
    let text = line[level..].strip_prefix(' ')?;
    (1..=6).contains(&level).then_some((level, text.trim()))
}

fn list_item(line: &str) -> Option<(bool, &str)> {
    if let Some(text) = line.strip_prefix("- ").or_else(|| line.strip_prefix("* ")) {
        return Some((false, text));
    }
    let digits = line.chars().take_while(char::is_ascii_digit).count();
    let text = line[digits..].strip_prefix(". ")?;
    (digits > 0).then_some((true, text))
}

fn starts_block(line: &str) -> bool {
    line.starts_with("```") || line.starts_with('>') || line == "---" || heading(line).is_some() || list_item(line).is_some()
}

fn parse_blocks(input: &str) -> Vec<Block> {
    let mut blocks = Vec::new();
    let mut lines = input.lines().map(str::trim_end).peekable();
    while let Some(line) = lines.next() {
        if line.trim().is_empty() {
            continue;
        }
        if let Some(lang) = line.strip_prefix("```") {
            let mut code = String::new();
            for line in lines.by_ref() {
                if line == "```" {
                    break;
                }
                code.push_str(line);
                code.push('\n');
            }
            let lang = (!lang.is_empty()).then(|| lang.to_string());
            blocks.push(Block::Code { lang, code });
        } else if let Some((level, text)) = heading(line) {
            blocks.push(Block::Heading { level, text: text.to_string() });
        } else if line == "---" {
            blocks.push(Block::Rule);
        } else if let Some((ordered, first)) = list_item(line) {
            let mut items = vec![first.to_string()];
            while let Some((same_kind, item)) = lines.peek().and_then(|next| list_item(next)) {
                if same_kind != ordered {
                    break;
                }
                items.push(item.to_string());
                lines.next();
            }
            blocks.push(Block::List { ordered, items });
        } else if line.starts_with('>') {
            let mut parts = vec![line.trim_start_matches('>').trim()];
            while let Some(next) = lines.next_if(|next| next.starts_with('>')) {
                parts.push(next.trim_start_matches('>').trim());
            }
            blocks.push(Block::Quote(parts.join(" ")));
        } else {
            let mut parts = vec![line.trim()];
            while let Some(next) = lines.next_if(|next| !next.trim().is_empty() && !starts_block(next)) {
                parts.push(next.trim());
            }
            blocks.push(Block::Paragraph(parts.join(" ")));
        }
    }
    blocks
}

// ---------- render.rs ----------

#[derive(Default)]
struct Slugger {
    seen: HashMap<String, usize>,
}

impl Slugger {
    fn slug(&mut self, heading: &str) -> String {
        let base: String = heading
            .to_lowercase()
            .chars()
            .filter_map(|c| match c {
                c if c.is_alphanumeric() => Some(c),
                ' ' | '-' | '_' => Some('-'),
                _ => None,
            })
            .collect();
        let count = self.seen.entry(base.clone()).or_insert(0);
        let slug = if *count == 0 { base } else { format!("{base}-{count}") };
        *count += 1;
        slug
    }
}

struct Page {
    title: String,
    toc: Vec<(usize, String, String)>, // (level, id, rendered text)
    body: String,
}

fn render(blocks: &[Block]) -> Page {
    let mut slugger = Slugger::default();
    let mut page = Page { title: String::from("Untitled"), toc: Vec::new(), body: String::new() };
    let out = &mut page.body;

    for block in blocks {
        // Writing to a String can't fail, so the fmt::Result is ignored with `let _`.
        let _ = match block {
            Block::Heading { level, text } => {
                let id = slugger.slug(text);
                let html = render_inline(text);
                if *level == 1 && page.title == "Untitled" {
                    page.title = text.clone();
                } else if *level <= 3 {
                    page.toc.push((*level, id.clone(), html.clone()));
                }
                writeln!(out, "<h{level} id=\"{id}\">{html}</h{level}>")
            }
            Block::Paragraph(text) => writeln!(out, "<p>{}</p>", render_inline(text)),
            Block::List { ordered, items } => {
                let tag = if *ordered { "ol" } else { "ul" };
                let items: String = items.iter().map(|item| format!("  <li>{}</li>\n", render_inline(item))).collect();
                writeln!(out, "<{tag}>\n{items}</{tag}>")
            }
            Block::Quote(text) => writeln!(out, "<blockquote><p>{}</p></blockquote>", render_inline(text)),
            Block::Code { lang, code } => {
                let class = lang.as_ref().map(|l| format!(" class=\"language-{}\"", escape(l))).unwrap_or_default();
                writeln!(out, "<pre><code{class}>{}</code></pre>", escape(code))
            }
            Block::Rule => writeln!(out, "<hr>"),
        };
    }
    page
}

fn to_html(page: &Page) -> String {
    let mut html = String::new();
    let _ = writeln!(html, "<!doctype html>\n<html>\n<head><meta charset=\"utf-8\"><title>{}</title></head>\n<body>", escape(&page.title));
    let _ = writeln!(html, "<h1>{}</h1>", render_inline(&page.title));
    if !page.toc.is_empty() {
        let _ = writeln!(html, "<nav><ul>");
        for (level, id, text) in &page.toc {
            let indent = "  ".repeat(level - 1);
            let _ = writeln!(html, "{indent}<li><a href=\"#{id}\">{text}</a></li>");
        }
        let _ = writeln!(html, "</ul></nav>");
    }
    // The body without the first <h1>, which is already the page heading.
    let body = page.body.split_once("</h1>\n").map_or(page.body.as_str(), |(_, rest)| rest);
    let _ = write!(html, "{body}</body>\n</html>");
    html
}

// ---------- main.rs ----------

const DEMO: &str = "\
# Shipping Guide
Orders ship within **2 business days**. See [tracking](https://example.com/track?id=1&v=2).

<h2 id="rates">Rates</h2>
1. Standard: *free* over $50
2. Express: `+$15`

<h2 id="packaging-returns">Packaging &amp; returns</h2>
> Keep the original box.
> Returns need an <RMA> number.

```rust
if weight > 20 && !oversized { surcharge(); }
```
";

fn main() -> ExitCode {
    let markdown = match std::env::args().nth(1) {
        Some(path) => match std::fs::read_to_string(&path) {
            Ok(text) => text,
            Err(error) => {
                eprintln!("mdhtml: {path}: {error}");
                return ExitCode::FAILURE;
            }
        },
        None => DEMO.to_string(),
    };
    println!("{}", to_html(&render(&parse_blocks(&markdown))));
    ExitCode::SUCCESS
}
Output
<!doctype html>
<html>
<head><meta charset="utf-8"><title>Shipping Guide</title></head>
<body>
<h1>Shipping Guide</h1>
<nav><ul>
  <li><a href="#rates">Rates</a></li>
  <li><a href="#packaging--returns">Packaging &amp; returns</a></li>
</ul></nav>
<p>Orders ship within <strong>2 business days</strong>. See <a href="https://example.com/track?id=1&amp;v=2">tracking</a>.</p>
<h2 id="rates">Rates</h2>
<ol>
  <li>Standard: <em>free</em> over $50</li>
  <li>Express: <code>+$15</code></li>
</ol>
<h2 id="packaging--returns">Packaging &amp; returns</h2>
<blockquote><p>Keep the original box. Returns need an &lt;RMA&gt; number.</p></blockquote>
<pre><code class="language-rust">if weight &gt; 20 &amp;&amp; !oversized { surcharge(); }
</code></pre>
</body>
</html>

Every <, &, and " from the input was escaped: in the text, in the link URL, and inside the code block.

Ideas for extending it

  • Nested lists: track each item's indentation and turn Block::List items into Vec<Block>.
  • Tables: parse | a | b | rows into a Block::Table { header, rows }.
  • Syntax highlighting of code blocks with the syntect crate.
  • A --watch mode that regenerates the HTML when the file changes, using the notify crate.
  • For production use, the pulldown-cmark crate implements full CommonMark as an iterator of events, a design worth reading after building this one.

Summary

  • Escaping every piece of user text before it reaches HTML is the first rule of generating HTML.
  • A two-layer design, blocks from lines and inline formatting inside blocks, keeps each parser simple.
  • str::strip_prefix, find, and split_once handle most scanning without indexes; a recursive call handles nesting.
  • Peekable::next_if expresses "consume the next line only if it continues this block".
  • write! into a String builds output efficiently; a HashMap makes heading slugs unique.

Next: Expense Tracker CLI.