Lesson 4 of 12
Markdown to HTML Converter
We build mdhtml, which converts Markdown into an HTML page. It supports the subset people use most:
| Markdown | HTML |
|---|---|
# Title to ###### Title |
<h1 id="title"> to <h6>, with anchor ids |
| paragraphs separated by blank lines | <p> |
- item, * item, 1. item |
<ul> / <ol> |
> quoted text |
<blockquote> |
| a fenced code block with an optional language | <pre><code class="language-rust"> |
--- |
<hr> |
`code`, **bold**, *italic*, [text](url) |
<code>, <strong>, <em>, <a> |
The design has two layers, the same as real Markdown engines: a block parser that looks at whole lines and decides what kind of block each belongs to, and an inline renderer that handles formatting inside a block's text.
mdhtml/
└── src/
├── main.rs # read a file (or a demo), write the page to stdout
├── inline.rs # step 1: escaping and inline formatting
├── blocks.rs # step 2: lines to a Vec<Block>
└── render.rs # step 3: Vec<Block> to HTML, anchors, table of contentsStep 1: Inline formatting
Text must be escaped before it goes into HTML, or a < in the input would start a tag (and user-supplied Markdown could inject scripts). Inline formatting is handled by a scanner that walks the text: at each position it checks for `, **, *, or [, and if the matching closing marker exists it emits the tag and jumps past it; otherwise it emits the character escaped. Bold, italics, and link labels can contain other formatting, so the scanner calls itself on their contents.
pub fn escape(text: &str) -> String {
let mut out = String::with_capacity(text.len());
for c in text.chars() {
match c {
'&' => out.push_str("&"),
'<' => out.push_str("<"),
'>' => out.push_str(">"),
'"' => out.push_str("""),
_ => out.push(c),
}
}
out
}
pub fn render_inline(text: &str) -> String {
let mut out = String::new();
let mut rest = text;
while let Some(c) = rest.chars().next() {
if let Some(after) = rest.strip_prefix('`') {
if let Some(end) = after.find('`') {
// Code spans are literal: no formatting inside, only escaping.
out.push_str(&format!("<code>{}</code>", escape(&after[..end])));
rest = &after[end + 1..];
continue;
}
} else if let Some(after) = rest.strip_prefix("**") {
if let Some(end) = after.find("**") {
out.push_str(&format!("<strong>{}</strong>", render_inline(&after[..end])));
rest = &after[end + 2..];
continue;
}
} else if let Some(after) = rest.strip_prefix('*') {
if let Some(end) = after.find('*') {
out.push_str(&format!("<em>{}</em>", render_inline(&after[..end])));
rest = &after[end + 1..];
continue;
}
} else if let Some(after) = rest.strip_prefix('[') {
if let Some((label, tail)) = after.split_once("](") {
if let (false, Some((url, tail))) = (label.contains(']'), tail.split_once(')')) {
out.push_str(&format!("<a href=\"{}\">{}</a>", escape(url), render_inline(label)));
rest = tail;
continue;
}
}
}
// Not the start of any formatting: copy one character, escaped.
out.push_str(&escape(&rest[..c.len_utf8()]));
rest = &rest[c.len_utf8()..];
}
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn escapes_html() {
assert_eq!(render_inline("a < b && c"), "a < b && c");
}
#[test]
fn code_is_not_formatted() {
assert_eq!(render_inline("use `**kwargs` here"), "use <code>**kwargs</code> here");
}
#[test]
fn nested_formatting() {
assert_eq!(render_inline("**bold and *italic***"), "<strong>bold and *italic</strong>*");
assert_eq!(render_inline("[**docs**](https://doc.rust-lang.org)"), "<a href=\"https://doc.rust-lang.org\"><strong>docs</strong></a>");
}
#[test]
fn unmatched_markers_are_plain_text() {
assert_eq!(render_inline("2 * 3 = 6"), "2 * 3 = 6");
assert_eq!(render_inline("[x] and [y](z)"), "[x] and <a href=\"z\">y</a>");
}
#[test]
fn urls_are_escaped() {
assert_eq!(render_inline("[x](\"><script>)"), "<a href=\""><script>\">x</a>");
}
}running 5 tests
test tests::code_is_not_formatted ... ok
test tests::escapes_html ... ok
test tests::nested_formatting ... ok
test tests::unmatched_markers_are_plain_text ... ok
test tests::urls_are_escaped ... ok
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s**bold and *italic*** shows the limit of a simple scanner: the first ** after the opening one closes the bold, so the italic marker is left unmatched. Full CommonMark has pages of rules for such cases; for a converter of your own documents, this trade-off is fine.
Step 2: Parsing blocks
The block parser reads lines and produces a Vec<Block>. Most blocks are recognized by their first line. Paragraphs, lists, and quotes continue over several lines, and a paragraph ends at a blank line or at a line that starts a different block.
#[derive(Debug, PartialEq)]
enum Block {
Heading { level: usize, text: String },
Paragraph(String),
List { ordered: bool, items: Vec<String> },
Quote(String),
Code { lang: Option<String>, code: String },
Rule,
}
fn heading(line: &str) -> Option<(usize, &str)> {
let level = line.chars().take_while(|&c| c == '#').count();
let text = line[level..].strip_prefix(' ')?;
(1..=6).contains(&level).then_some((level, text.trim()))
}
/// `- item`, `* item` (unordered) or `12. item` (ordered).
fn list_item(line: &str) -> Option<(bool, &str)> {
if let Some(text) = line.strip_prefix("- ").or_else(|| line.strip_prefix("* ")) {
return Some((false, text));
}
let digits = line.chars().take_while(char::is_ascii_digit).count();
let text = line[digits..].strip_prefix(". ")?;
(digits > 0).then_some((true, text))
}
fn starts_block(line: &str) -> bool {
line.starts_with("```") || line.starts_with('>') || line == "---" || heading(line).is_some() || list_item(line).is_some()
}
fn parse_blocks(input: &str) -> Vec<Block> {
let mut blocks = Vec::new();
let mut lines = input.lines().map(str::trim_end).peekable();
while let Some(line) = lines.next() {
if line.trim().is_empty() {
continue;
}
if let Some(lang) = line.strip_prefix("```") {
let mut code = String::new();
for line in lines.by_ref() {
if line == "```" {
break;
}
code.push_str(line);
code.push('\n');
}
let lang = (!lang.is_empty()).then(|| lang.to_string());
blocks.push(Block::Code { lang, code });
} else if let Some((level, text)) = heading(line) {
blocks.push(Block::Heading { level, text: text.to_string() });
} else if line == "---" {
blocks.push(Block::Rule);
} else if let Some((ordered, first)) = list_item(line) {
let mut items = vec![first.to_string()];
while let Some((same_kind, item)) = lines.peek().and_then(|next| list_item(next)) {
if same_kind != ordered {
break;
}
items.push(item.to_string());
lines.next();
}
blocks.push(Block::List { ordered, items });
} else if line.starts_with('>') {
let mut parts = vec![line.trim_start_matches('>').trim()];
while let Some(next) = lines.next_if(|next| next.starts_with('>')) {
parts.push(next.trim_start_matches('>').trim());
}
blocks.push(Block::Quote(parts.join(" ")));
} else {
let mut parts = vec![line.trim()];
while let Some(next) = lines.next_if(|next| !next.trim().is_empty() && !starts_block(next)) {
parts.push(next.trim());
}
blocks.push(Block::Paragraph(parts.join(" ")));
}
}
blocks
}
fn main() {
let doc = "\
# Release notes
Version 2.0 is out.
It is **faster**.
<h2 id="changes">Changes</h2>
- new parser
- fewer allocations
1. upgrade
2. run the tests
> Breaking: the `v1` API
> was removed.
```rust
fn main() {}
```
---
####### not a heading";
for block in parse_blocks(doc) {
println!("{block:?}");
}
}Heading { level: 1, text: "Release notes" }
Paragraph("Version 2.0 is out. It is **faster**.")
Heading { level: 2, text: "Changes" }
List { ordered: false, items: ["new parser", "fewer allocations"] }
List { ordered: true, items: ["upgrade", "run the tests"] }
Quote("Breaking: the `v1` API was removed.")
Code { lang: Some("rust"), code: "fn main() {}\n" }
Rule
Paragraph("####### not a heading")Peekable::next_if takes the next line only if it passes a test, which is exactly "continue the block while the next line belongs to it". The inline text is kept raw here; formatting happens when rendering.
Step 3: Rendering, anchors, and a table of contents
Rendering is a match over the blocks. Headings get an id so readers can link to them. The id is a slug: lowercase, spaces turned into hyphens, punctuation dropped. Two headings can produce the same slug, so a HashMap counts them and appends -1, -2, and so on, as GitHub does.
use std::collections::HashMap;
#[derive(Default)]
pub struct Slugger {
seen: HashMap<String, usize>,
}
impl Slugger {
pub fn slug(&mut self, heading: &str) -> String {
let base: String = heading
.to_lowercase()
.chars()
.filter_map(|c| match c {
c if c.is_alphanumeric() => Some(c),
' ' | '-' | '_' => Some('-'),
_ => None,
})
.collect();
let count = self.seen.entry(base.clone()).or_insert(0);
let slug = if *count == 0 { base } else { format!("{base}-{count}") };
*count += 1;
slug
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn slugs_are_lowercase_and_hyphenated() {
let mut slugger = Slugger::default();
assert_eq!(slugger.slug("Getting Started"), "getting-started");
assert_eq!(slugger.slug("What's new in 2.0?"), "whats-new-in-20");
assert_eq!(slugger.slug("Größe & Maße"), "größe--maße");
}
#[test]
fn duplicates_get_a_suffix() {
let mut slugger = Slugger::default();
let slugs: Vec<String> = ["Example", "Example", "Example"].iter().map(|h| slugger.slug(h)).collect();
assert_eq!(slugs, ["example", "example-1", "example-2"]);
}
}running 2 tests
test tests::duplicates_get_a_suffix ... ok
test tests::slugs_are_lowercase_and_hyphenated ... ok
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00sStep 4: The complete converter
The final program renders blocks into a String with write!, collects headings for a table of contents, and wraps everything in a page. main reads the file named on the command line, or a demo document when there is none.
use std::collections::HashMap;
use std::fmt::Write as _;
use std::process::ExitCode;
// ---------- inline.rs ----------
fn escape(text: &str) -> String {
let mut out = String::with_capacity(text.len());
for c in text.chars() {
match c {
'&' => out.push_str("&"),
'<' => out.push_str("<"),
'>' => out.push_str(">"),
'"' => out.push_str("""),
_ => out.push(c),
}
}
out
}
fn render_inline(text: &str) -> String {
let mut out = String::new();
let mut rest = text;
while let Some(c) = rest.chars().next() {
if let Some(after) = rest.strip_prefix('`') {
if let Some(end) = after.find('`') {
out.push_str(&format!("<code>{}</code>", escape(&after[..end])));
rest = &after[end + 1..];
continue;
}
} else if let Some(after) = rest.strip_prefix("**") {
if let Some(end) = after.find("**") {
out.push_str(&format!("<strong>{}</strong>", render_inline(&after[..end])));
rest = &after[end + 2..];
continue;
}
} else if let Some(after) = rest.strip_prefix('*') {
if let Some(end) = after.find('*') {
out.push_str(&format!("<em>{}</em>", render_inline(&after[..end])));
rest = &after[end + 1..];
continue;
}
} else if let Some(after) = rest.strip_prefix('[') {
if let Some((label, tail)) = after.split_once("](") {
if let (false, Some((url, tail))) = (label.contains(']'), tail.split_once(')')) {
out.push_str(&format!("<a href=\"{}\">{}</a>", escape(url), render_inline(label)));
rest = tail;
continue;
}
}
}
out.push_str(&escape(&rest[..c.len_utf8()]));
rest = &rest[c.len_utf8()..];
}
out
}
// ---------- blocks.rs ----------
enum Block {
Heading { level: usize, text: String },
Paragraph(String),
List { ordered: bool, items: Vec<String> },
Quote(String),
Code { lang: Option<String>, code: String },
Rule,
}
fn heading(line: &str) -> Option<(usize, &str)> {
let level = line.chars().take_while(|&c| c == '#').count();
let text = line[level..].strip_prefix(' ')?;
(1..=6).contains(&level).then_some((level, text.trim()))
}
fn list_item(line: &str) -> Option<(bool, &str)> {
if let Some(text) = line.strip_prefix("- ").or_else(|| line.strip_prefix("* ")) {
return Some((false, text));
}
let digits = line.chars().take_while(char::is_ascii_digit).count();
let text = line[digits..].strip_prefix(". ")?;
(digits > 0).then_some((true, text))
}
fn starts_block(line: &str) -> bool {
line.starts_with("```") || line.starts_with('>') || line == "---" || heading(line).is_some() || list_item(line).is_some()
}
fn parse_blocks(input: &str) -> Vec<Block> {
let mut blocks = Vec::new();
let mut lines = input.lines().map(str::trim_end).peekable();
while let Some(line) = lines.next() {
if line.trim().is_empty() {
continue;
}
if let Some(lang) = line.strip_prefix("```") {
let mut code = String::new();
for line in lines.by_ref() {
if line == "```" {
break;
}
code.push_str(line);
code.push('\n');
}
let lang = (!lang.is_empty()).then(|| lang.to_string());
blocks.push(Block::Code { lang, code });
} else if let Some((level, text)) = heading(line) {
blocks.push(Block::Heading { level, text: text.to_string() });
} else if line == "---" {
blocks.push(Block::Rule);
} else if let Some((ordered, first)) = list_item(line) {
let mut items = vec![first.to_string()];
while let Some((same_kind, item)) = lines.peek().and_then(|next| list_item(next)) {
if same_kind != ordered {
break;
}
items.push(item.to_string());
lines.next();
}
blocks.push(Block::List { ordered, items });
} else if line.starts_with('>') {
let mut parts = vec![line.trim_start_matches('>').trim()];
while let Some(next) = lines.next_if(|next| next.starts_with('>')) {
parts.push(next.trim_start_matches('>').trim());
}
blocks.push(Block::Quote(parts.join(" ")));
} else {
let mut parts = vec![line.trim()];
while let Some(next) = lines.next_if(|next| !next.trim().is_empty() && !starts_block(next)) {
parts.push(next.trim());
}
blocks.push(Block::Paragraph(parts.join(" ")));
}
}
blocks
}
// ---------- render.rs ----------
#[derive(Default)]
struct Slugger {
seen: HashMap<String, usize>,
}
impl Slugger {
fn slug(&mut self, heading: &str) -> String {
let base: String = heading
.to_lowercase()
.chars()
.filter_map(|c| match c {
c if c.is_alphanumeric() => Some(c),
' ' | '-' | '_' => Some('-'),
_ => None,
})
.collect();
let count = self.seen.entry(base.clone()).or_insert(0);
let slug = if *count == 0 { base } else { format!("{base}-{count}") };
*count += 1;
slug
}
}
struct Page {
title: String,
toc: Vec<(usize, String, String)>, // (level, id, rendered text)
body: String,
}
fn render(blocks: &[Block]) -> Page {
let mut slugger = Slugger::default();
let mut page = Page { title: String::from("Untitled"), toc: Vec::new(), body: String::new() };
let out = &mut page.body;
for block in blocks {
// Writing to a String can't fail, so the fmt::Result is ignored with `let _`.
let _ = match block {
Block::Heading { level, text } => {
let id = slugger.slug(text);
let html = render_inline(text);
if *level == 1 && page.title == "Untitled" {
page.title = text.clone();
} else if *level <= 3 {
page.toc.push((*level, id.clone(), html.clone()));
}
writeln!(out, "<h{level} id=\"{id}\">{html}</h{level}>")
}
Block::Paragraph(text) => writeln!(out, "<p>{}</p>", render_inline(text)),
Block::List { ordered, items } => {
let tag = if *ordered { "ol" } else { "ul" };
let items: String = items.iter().map(|item| format!(" <li>{}</li>\n", render_inline(item))).collect();
writeln!(out, "<{tag}>\n{items}</{tag}>")
}
Block::Quote(text) => writeln!(out, "<blockquote><p>{}</p></blockquote>", render_inline(text)),
Block::Code { lang, code } => {
let class = lang.as_ref().map(|l| format!(" class=\"language-{}\"", escape(l))).unwrap_or_default();
writeln!(out, "<pre><code{class}>{}</code></pre>", escape(code))
}
Block::Rule => writeln!(out, "<hr>"),
};
}
page
}
fn to_html(page: &Page) -> String {
let mut html = String::new();
let _ = writeln!(html, "<!doctype html>\n<html>\n<head><meta charset=\"utf-8\"><title>{}</title></head>\n<body>", escape(&page.title));
let _ = writeln!(html, "<h1>{}</h1>", render_inline(&page.title));
if !page.toc.is_empty() {
let _ = writeln!(html, "<nav><ul>");
for (level, id, text) in &page.toc {
let indent = " ".repeat(level - 1);
let _ = writeln!(html, "{indent}<li><a href=\"#{id}\">{text}</a></li>");
}
let _ = writeln!(html, "</ul></nav>");
}
// The body without the first <h1>, which is already the page heading.
let body = page.body.split_once("</h1>\n").map_or(page.body.as_str(), |(_, rest)| rest);
let _ = write!(html, "{body}</body>\n</html>");
html
}
// ---------- main.rs ----------
const DEMO: &str = "\
# Shipping Guide
Orders ship within **2 business days**. See [tracking](https://example.com/track?id=1&v=2).
<h2 id="rates">Rates</h2>
1. Standard: *free* over $50
2. Express: `+$15`
<h2 id="packaging-returns">Packaging & returns</h2>
> Keep the original box.
> Returns need an <RMA> number.
```rust
if weight > 20 && !oversized { surcharge(); }
```
";
fn main() -> ExitCode {
let markdown = match std::env::args().nth(1) {
Some(path) => match std::fs::read_to_string(&path) {
Ok(text) => text,
Err(error) => {
eprintln!("mdhtml: {path}: {error}");
return ExitCode::FAILURE;
}
},
None => DEMO.to_string(),
};
println!("{}", to_html(&render(&parse_blocks(&markdown))));
ExitCode::SUCCESS
}<!doctype html>
<html>
<head><meta charset="utf-8"><title>Shipping Guide</title></head>
<body>
<h1>Shipping Guide</h1>
<nav><ul>
<li><a href="#rates">Rates</a></li>
<li><a href="#packaging--returns">Packaging & returns</a></li>
</ul></nav>
<p>Orders ship within <strong>2 business days</strong>. See <a href="https://example.com/track?id=1&v=2">tracking</a>.</p>
<h2 id="rates">Rates</h2>
<ol>
<li>Standard: <em>free</em> over $50</li>
<li>Express: <code>+$15</code></li>
</ol>
<h2 id="packaging--returns">Packaging & returns</h2>
<blockquote><p>Keep the original box. Returns need an <RMA> number.</p></blockquote>
<pre><code class="language-rust">if weight > 20 && !oversized { surcharge(); }
</code></pre>
</body>
</html>Every <, &, and " from the input was escaped: in the text, in the link URL, and inside the code block.
Ideas for extending it
- Nested lists: track each item's indentation and turn
Block::Listitems intoVec<Block>. - Tables: parse
| a | b |rows into aBlock::Table { header, rows }. - Syntax highlighting of code blocks with the
syntectcrate. - A
--watchmode that regenerates the HTML when the file changes, using thenotifycrate. - For production use, the
pulldown-cmarkcrate implements full CommonMark as an iterator of events, a design worth reading after building this one.
Summary
- Escaping every piece of user text before it reaches HTML is the first rule of generating HTML.
- A two-layer design, blocks from lines and inline formatting inside blocks, keeps each parser simple.
str::strip_prefix,find, andsplit_oncehandle most scanning without indexes; a recursive call handles nesting.Peekable::next_ifexpresses "consume the next line only if it continues this block".write!into aStringbuilds output efficiently; aHashMapmakes heading slugs unique.
Next: Expense Tracker CLI.
