html-to-markdown

⚙️ Convert HTML to Markdown. Even works with entire websites and can be extended through rules.

2,461

128

2,461

View on GitHub

Top Related Projects

markdown

1,439

markdown parser and HTML renderer for Go

blackfriday

5,479

Blackfriday: a markdown processor for Go

goldmark

3,750

:trophy: A markdown parser written in Go. Easy to extend, standard(CommonMark) compliant, well structured.

lute

1,298

🎼 一款结构化的 Markdown 引擎，支持 Go 和 JavaScript。A structured Markdown engine that supports Go and JavaScript.

cmark

1,666

CommonMark parsing and rendering library and program in C

showdown

14,258

A bidirectional Markdown to HTML to Markdown converter written in Javascript

Quick Overview

JohannesKaufmann/html-to-markdown is a Go library that converts HTML to Markdown. It aims to provide a flexible and customizable solution for transforming HTML content into clean, readable Markdown format. The library supports various HTML elements and can be extended with custom rules.

Pros

Highly customizable with the ability to add custom rules
Supports a wide range of HTML elements and attributes
Actively maintained with regular updates
Comprehensive documentation and examples

Cons

Limited to Go programming language
May require additional configuration for complex HTML structures
Performance might be affected when dealing with large HTML documents
Some edge cases in HTML-to-Markdown conversion may not be perfectly handled

Code Examples

Basic HTML to Markdown conversion:

import (
    "fmt"
    md "github.com/JohannesKaufmann/html-to-markdown"
)

converter := md.NewConverter("", true, nil)
markdown, err := converter.ConvertString("<h1>Hello, World!</h1><p>This is a paragraph.</p>")
if err != nil {
    fmt.Println("Error:", err)
    return
}
fmt.Println(markdown)

Using custom rules:

import (
    md "github.com/JohannesKaufmann/html-to-markdown"
    "github.com/JohannesKaufmann/html-to-markdown/plugin"
)

converter := md.NewConverter("", true, nil)
converter.Use(plugin.GitHubFlavored())
converter.AddRules(md.Rule{
    Filter: []string{"div"},
    Replacement: func(content string, selec *md.Selection, options *md.Options) *string {
        class := selec.Attr("class")
        if class == "custom-div" {
            result := fmt.Sprintf(":::custom\n%s\n:::", content)
            return &result
        }
        return nil
    },
})

Converting from an HTML file:

import (
    "fmt"
    "os"
    md "github.com/JohannesKaufmann/html-to-markdown"
)

converter := md.NewConverter("", true, nil)
file, err := os.Open("input.html")
if err != nil {
    fmt.Println("Error:", err)
    return
}
defer file.Close()

markdown, err := converter.Convert(file)
if err != nil {
    fmt.Println("Error:", err)
    return
}
fmt.Println(markdown)

Getting Started

To use the html-to-markdown library in your Go project, follow these steps:

Install the library:

go get github.com/JohannesKaufmann/html-to-markdown

Import the library in your Go code:

import md "github.com/JohannesKaufmann/html-to-markdown"

Create a converter and use it to convert HTML to Markdown:

converter := md.NewConverter("", true, nil)
markdown, err := converter.ConvertString("<h1>Hello, World!</h1>")
if err != nil {
    // Handle error
}
fmt.Println(markdown)

Competitor Comparisons

markdown

1,439

markdown parser and HTML renderer for Go

Pros of markdown

More comprehensive Markdown parsing and rendering capabilities
Supports a wider range of Markdown flavors and extensions
Better performance for large documents

Cons of markdown

Lacks direct HTML-to-Markdown conversion functionality
More complex API, potentially steeper learning curve
May require additional setup for specific use cases

Code Comparison

html-to-markdown:

converter := md.NewConverter("", true, nil)
markdown, err := converter.ConvertString(htmlContent)

markdown:

parser := parser.New()
doc := parser.Parse([]byte(markdownContent))
html := markdown.Render(doc)

Summary

markdown is a more robust and feature-rich Markdown library, offering extensive parsing and rendering capabilities for various Markdown flavors. It excels in performance and flexibility but lacks direct HTML-to-Markdown conversion. html-to-markdown, on the other hand, specializes in converting HTML to Markdown, providing a simpler API for this specific task. The choice between the two depends on the project's requirements: markdown for comprehensive Markdown processing, or html-to-markdown for straightforward HTML-to-Markdown conversion.

blackfriday

5,479

Blackfriday: a markdown processor for Go

Pros of Blackfriday

More comprehensive Markdown support, including extensions like tables and fenced code blocks
Higher performance and efficiency for large-scale processing
Well-established and widely used in the Go ecosystem

Cons of Blackfriday

Primarily focused on Markdown-to-HTML conversion, not HTML-to-Markdown
Less flexible for customizing HTML-to-Markdown conversion rules
Not actively maintained (last commit in 2019)

Code Comparison

html-to-markdown:

converter := md.NewConverter("", true, nil)
markdown, err := converter.ConvertString(html)

Blackfriday:

output := blackfriday.Run(input)

While Blackfriday is primarily used for Markdown-to-HTML conversion, html-to-markdown is specifically designed for HTML-to-Markdown conversion. Blackfriday's API is simpler, but html-to-markdown offers more flexibility for customizing the conversion process.

html-to-markdown is more suitable for projects requiring HTML-to-Markdown conversion, while Blackfriday excels in Markdown processing and HTML generation. The choice between the two depends on the specific requirements of your project and the direction of conversion needed.

goldmark

3,750

:trophy: A markdown parser written in Go. Easy to extend, standard(CommonMark) compliant, well structured.

Pros of goldmark

Faster parsing and rendering performance
More extensive Markdown syntax support (CommonMark, GitHub Flavored Markdown)
Built-in HTML rendering capabilities

Cons of goldmark

Primarily focused on Markdown-to-HTML conversion, not HTML-to-Markdown
Steeper learning curve for customization and extension
Less specialized for HTML-to-Markdown conversion tasks

Code Comparison

html-to-markdown:

converter := md.NewConverter("", true, nil)
markdown, err := converter.ConvertString(html)

goldmark:

var buf bytes.Buffer
if err := goldmark.Convert([]byte(markdown), &buf); err != nil {
    panic(err)
}
html := buf.String()

Note: The code snippets demonstrate the primary use case for each library. html-to-markdown converts HTML to Markdown, while goldmark converts Markdown to HTML.

Summary

goldmark is a powerful Markdown parser and renderer with excellent performance and extensive syntax support. It's ideal for Markdown-to-HTML conversion tasks. html-to-markdown, on the other hand, specializes in converting HTML to Markdown, making it more suitable for tasks that require extracting Markdown from HTML content. Choose the library that best fits your specific use case and requirements.

lute

1,298

🎼 一款结构化的 Markdown 引擎，支持 Go 和 JavaScript。A structured Markdown engine that supports Go and JavaScript.

Pros of lute

Supports both HTML-to-Markdown and Markdown-to-HTML conversion
Implements additional Markdown syntax extensions (e.g., task lists, footnotes)
Written in Go, which may offer better performance for certain use cases

Cons of lute

Less focused on HTML-to-Markdown conversion specifically
May have a steeper learning curve due to broader feature set
Documentation is primarily in Chinese, which could be challenging for non-Chinese speakers

Code Comparison

html-to-markdown (Go):

converter := md.NewConverter("", true, nil)
markdown, err := converter.ConvertString(html)

lute (Go):

engine := lute.New()
markdown := engine.HTML2Markdown(html)

Both libraries offer straightforward conversion methods, but lute's API appears slightly simpler for basic HTML-to-Markdown conversion. However, html-to-markdown provides more configuration options in its conversion function.

Summary

lute offers a more comprehensive Markdown processing solution with bidirectional conversion and extended syntax support. It may be preferable for projects requiring a full-featured Markdown toolkit. html-to-markdown, on the other hand, is more focused on HTML-to-Markdown conversion and may be easier to integrate for this specific use case, especially for English-speaking developers.

cmark

1,666

CommonMark parsing and rendering library and program in C

Pros of cmark

Written in C, offering high performance and efficiency
Implements the full CommonMark spec, ensuring strict compliance
Provides bindings for multiple programming languages

Cons of cmark

Focused solely on Markdown parsing and rendering, not HTML-to-Markdown conversion
May require more setup and integration effort for specific use cases
Less flexibility for customizing Markdown output

Code Comparison

html-to-markdown:

md := md.NewConverter("", true, nil)
markdown := md.Convert(html)

cmark:

char *markdown = cmark_markdown_to_html(input, strlen(input), CMARK_OPT_DEFAULT);

Key Differences

html-to-markdown is specifically designed for HTML-to-Markdown conversion
cmark is a general-purpose Markdown parser and renderer
html-to-markdown offers more flexibility in handling HTML input
cmark provides better performance for Markdown processing tasks

Use Cases

Choose html-to-markdown for projects requiring HTML-to-Markdown conversion
Opt for cmark when working with pure Markdown or need high-performance parsing

Community and Maintenance

html-to-markdown: Active development, good documentation, and community support
cmark: Well-established project with a strong focus on the CommonMark spec

showdown

14,258

A bidirectional Markdown to HTML to Markdown converter written in Javascript

Pros of showdown

Bidirectional conversion: Can convert both HTML to Markdown and Markdown to HTML
Extensive customization options through extensions and configuration
Well-established project with a large community and long history

Cons of showdown

Primarily focused on Markdown to HTML conversion, with HTML to Markdown as a secondary feature
May produce less clean Markdown output when converting from HTML
Heavier library size due to its comprehensive feature set

Code Comparison

showdown:

var converter = new showdown.Converter();
var html = converter.makeHtml('# Hello, Markdown!');
var md = converter.makeMarkdown('<h1>Hello, HTML!</h1>');

html-to-markdown:

converter := md.NewConverter("", true, nil)
markdown, err := converter.ConvertString("<h1>Hello, HTML!</h1>")

html-to-markdown is more focused on HTML to Markdown conversion, offering a simpler API for this specific task. It's written in Go, which may be advantageous for certain projects. showdown provides a more comprehensive solution for both conversion directions but may be overkill if only HTML to Markdown conversion is needed.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

html-to-markdown

A robust html-to-markdown converter that transforms HTML (even entire websites) into clean, readable Markdown. It supports complex formatting, customizable options, and plugins for full control over the conversion process.

Use the fully extendable Golang library or a quick CLI command. Alternatively, try the Online Demo or REST API to see it in action!

Here are some cool features:

Bold & Italic: Supports bold and italicâeven within single words.
List: Handles ordered and unordered lists with full nesting support.
Blockquote: Blockquotes can include other elements, with seamless support for nested quotes.
Inline Code & Code Block: Correctly handles backticks and multi-line code blocks, preserving code structure.
Link & Image: Properly formats multi-line links, adding escapes for blank lines where needed.
Smart Escaping: Escapes special characters only when necessary, to avoid accidental Markdown rendering. ðï¸ ESCAPING.md
Remove/Keep HTML: Choose to strip or retain specific HTML tags for ultimate control over output.
Plugins: Easily extend with plugins. Or create custom ones to enhance functionality.

Usage

ð» Golang library | ð¦ CLI | â¶ï¸ Hosted Demo | ð Hosted REST API

[!TIP] Looking for an all in one cloud solution? We're sponsored by ð¥ Firecrawl, where you can scrape any website and turn it into AI friendly markdown with one API call.

Golang Library

Installation

go get -u github.com/JohannesKaufmann/html-to-markdown/v2

Or if you want a specific commit add the suffix /v2@commithash

[!NOTE]
This is the documentation for the v2 library. For the old version switch to the "v1" branch.

Usage

package main

import (
	"fmt"
	"log"

	htmltomarkdown "github.com/JohannesKaufmann/html-to-markdown/v2"
)

func main() {
	input := `<strong>Bold Text</strong>`

	markdown, err := htmltomarkdown.ConvertString(input)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(markdown)
	// Output: **Bold Text**
}

ð§âð» Example code, basics

Use WithDomain to convert relative links to absolute links:

package main

import (
	"fmt"
	"log"

	htmltomarkdown "github.com/JohannesKaufmann/html-to-markdown/v2"
	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
)

func main() {
	input := `<img src="/assets/image.png" />`

	markdown, err := htmltomarkdown.ConvertString(
		input,
		converter.WithDomain("https://example.com"),
	)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(markdown)
	// Output: ![](https://example.com/assets/image.png)
}

The function htmltomarkdown.ConvertString() is a small wrapper around converter.NewConverter() and the base and commonmark plugins. If you want more control, use the following:

package main

import (
	"fmt"
	"log"

	"github.com/JohannesKaufmann/html-to-markdown/v2/converter"
	"github.com/JohannesKaufmann/html-to-markdown/v2/plugin/base"
	"github.com/JohannesKaufmann/html-to-markdown/v2/plugin/commonmark"
)

func main() {
	input := `<strong>Bold Text</strong>`

	conv := converter.NewConverter(
		converter.WithPlugins(
			base.NewBasePlugin(),
			commonmark.NewCommonmarkPlugin(
				commonmark.WithStrongDelimiter("__"),
				// ...additional configurations for the plugin
			),
		),
	)

	markdown, err := conv.ConvertString(input)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(markdown)
	// Output: __Bold Text__
}

ð§âð» Example code, options

[!NOTE]
If you use NewConverter directly make sure to also register the commonmark and base plugin.

Plugins

Published Plugins

These are the plugins located in the plugin folder:

Name	Description
Base	Implements basic shared functionality (e.g. removing nodes)
Commonmark	Implements Markdown according to the Commonmark Spec

GitHubFlavored	planned
TaskListItems	planned
Strikethrough	Converts `<strike>`, `<s>`, and `<del>` to the `~~` syntax.
Table	planned

VimeoEmbed	planned
YoutubeEmbed	planned

ConfluenceCodeBlock	planned
ConfluenceAttachments	planned

[!NOTE]
Not all the plugins from v1 are already ported to v2. These will soon be implemented...

These are the plugins in other repositories:

Name	Description
[Plugin Name](Your Link)	A short description

Writing Plugins

You want to write custom logic?

Write your logic and register it.
- ð§âð» Example code, register
Optional: Package your logic into a plugin and publish it.
- ðï¸ WRITING_PLUGINS.md

CLI - Using it on the command line

Using the Golang library provides the most customization, while the CLI is the simplest way to get started.

Installation

Homebrew Tap

brew install JohannesKaufmann/tap/html2markdown

Debian

A deb package is available. See the Setup Instructions.

Note: Support for other Linux distributions is tracked in #119

Pre-compiled Binaries

Download pre-compiled binaries for Linux, macOS or Windows from the releases page. Extract the archive and copy the executable to a location in your system PATH (e.g. /usr/local/bin).

Installation via Go

If you have Go installed, you can install the CLI directly using:

go install github.com/JohannesKaufmann/html-to-markdown/v2/cli/html2markdown@latest

This will download the source code and compile it into an executable in your Go binary directory (typically $GOPATH/bin).

Build from Source

Binaries are automatically built via GoReleaser and attached to each release.

To build locally (requires Go):

go build ./cli/html2markdown

Version

html2markdown --version

[!NOTE]
Make sure that --version prints 2.X.X as there is a different CLI for V2 of the converter.

Usage

$ echo "<strong>important</strong>" | html2markdown

**important**

$ curl --no-progress-meter http://example.com | html2markdown

# Example Domain

This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.

[More information...](https://www.iana.org/domains/example)

Use --help to learn about the configurations, for example:

--domain="https://example.com" to convert relative links to absolute links.
--exclude-selector=".ad" to exclude the html elements with class="ad" from the conversion.
--include-selector="article" to only include the <article> html elements in the conversion.

(The cli does not support every option yet. Over time more customization will be added)

FAQ

Extending with Plugins

Need your own logic? Write your own code and then register it.
- Don't like the defaults that the library uses? You can use PriorityEarly to run you logic earlier than others.
- ð§âð» Example code, register
If you believe that you logic could also benefit others, you can package it up into a plugin.
- ðï¸ WRITING_PLUGINS.md

Bugs

You found a bug?

Open an issue with the HTML snippet that does not produce the expected results. Please, please, plase submit the HTML snippet that caused the problem. Otherwise it is very difficult to reproduce and fix...

Security

This library produces markdown that is readable and can be changed by humans.

Once you convert this markdown back to HTML (e.g. using goldmark or blackfriday) you need to be careful of malicious content.

This library does NOT sanitize untrusted content. Use an HTML sanitizer such as bluemonday before displaying the HTML in the browser.

ðï¸ SECURITY.md if you find a security vulnerability

Goroutines

You can use the Converter from (multiple) goroutines. Internally a mutex is used & there is a test to verify that behaviour.

Escaping & Backslash

Some characters have a special meaning in markdown (e.g. "*" for emphasis). The backslash \ character is used to "escape" those characters. That is perfectly safe and won't be displayed in the final render.

ðï¸ ESCAPING.md

Contributing

You want to contribute? Thats great to hear! There are many ways to help:

Helping to answer questions, triaging issues, writing documentation, writing code, ...

If you want to make a code change: Please first discuss the change you wish to make, by opening an issue. I'm also happy to guide you to where a change is most likely needed. There are also extensive tests (see below) so you can freely experiment ð§âð¬

Note: The outside API should not change because of backwards compatibility...

Testing

You don't have to be afraid of breaking the converter, since there are many "Golden File" tests:

Add your problematic HTML snippet to one of the .in.html files in the testdata folders. Then run go test -update and have a look at which .out.md files changed in GIT.

You can now change the internal logic and inspect what impact your change has by running go test -update again.

Note: Before submitting your change as a PR, make sure that you run those tests and check the files into GIT...

License

Unless otherwise specified, the project is licensed under the terms of the MIT license.

ðï¸ LICENSE

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot