nom

Rust parser combinator framework

9,340

800

9,340

252

View on GitHub

Top Related Projects

rust-peg

1,443

Parsing Expression Grammar (PEG) parser generator for Rust

combine

1,290

A parser combinator library for Rust

Quick Overview

Nom is a parser combinator library written in Rust. It allows developers to build complex parsers from smaller, simpler ones, making it easier to handle various data formats and protocols. Nom is designed to be fast, reliable, and flexible, suitable for both binary and text formats.

Pros

High performance due to zero-copy parsing and compile-time optimizations
Extensive set of built-in parsers and combinators for common parsing tasks
Strong error handling and reporting capabilities
Supports both binary and text formats

Cons

Steep learning curve for developers new to parser combinators
Can be verbose for simple parsing tasks compared to regex-based solutions
Documentation can be overwhelming for beginners
Compile times can be long for complex parsers

Code Examples

Parsing a simple integer:

use nom::{
  IResult,
  character::complete::digit1,
  combinator::map_res,
};

fn parse_u32(input: &str) -> IResult<&str, u32> {
    map_res(digit1, |s: &str| s.parse::<u32>())(input)
}

assert_eq!(parse_u32("123"), Ok(("", 123)));

Parsing a pair of values separated by a comma:

use nom::{
  IResult,
  sequence::separated_pair,
  character::complete::{alpha1, char},
};

fn parse_pair(input: &str) -> IResult<&str, (&str, &str)> {
    separated_pair(alpha1, char(','), alpha1)(input)
}

assert_eq!(parse_pair("hello,world"), Ok(("", ("hello", "world"))));

Parsing a simple JSON-like structure:

use nom::{
  IResult,
  sequence::delimited,
  character::complete::{char, alpha1},
  bytes::complete::is_not,
};

fn parse_json_like(input: &str) -> IResult<&str, &str> {
    delimited(
        char('{'),
        delimited(
            char('"'),
            is_not("\""),
            char('"')
        ),
        char('}')
    )(input)
}

assert_eq!(parse_json_like("{\"hello\"}"), Ok(("", "hello")));

Getting Started

To use nom in your Rust project, add the following to your Cargo.toml:

[dependencies]
nom = "7.1.3"

Then, in your Rust file:

use nom::{
    IResult,
    bytes::complete::tag,
    character::complete::alpha1,
    sequence::tuple,
};

fn parser(input: &str) -> IResult<&str, (&str, &str, &str)> {
    tuple((alpha1, tag(" = "), alpha1))(input)
}

fn main() {
    let result = parser("key = value");
    println!("{:?}", result);
}

This example demonstrates a simple parser that matches a key-value pair separated by " = ". Run this code to see how nom parses the input string.

Competitor Comparisons

nom

9,335

Rust parser combinator framework

Pros of nom

Well-established and mature parser combinator library
Extensive documentation and community support
Flexible and powerful, suitable for various parsing tasks

Cons of nom

Steeper learning curve for beginners
Can be verbose for simple parsing tasks
Performance overhead for certain use cases

Code Comparison

nom:

use nom::{
  IResult,
  bytes::complete::tag,
  sequence::tuple
};

fn parser(input: &str) -> IResult<&str, (&str, &str)> {
  tuple((tag("Hello"), tag(" world")))(input)
}

nom>:

use nom_supreme::parser_ext::ParserExt;
use nom_supreme::tag::complete::tag;

fn parser(input: &str) -> IResult<&str, (&str, &str)> {
  (tag("Hello").and(tag(" world"))).parse(input)
}

The nom> example demonstrates a more concise and readable syntax for combining parsers, which is one of its main advantages over the original nom library. However, both libraries offer similar functionality and can be used to achieve the same parsing goals.

lalrpop

3,004

LR(1) parser generator for Rust

Pros of lalrpop

Generates fast, efficient parsers for context-free grammars
Provides better error reporting and recovery mechanisms
Supports left-recursive grammars directly

Cons of lalrpop

Steeper learning curve due to its grammar-based approach
Less flexible for parsing complex or ambiguous structures
Requires a separate build step to generate the parser code

Code Comparison

lalrpop example:

grammar;

pub Expr: i32 = {
    <l:Expr> "+" <r:Factor> => l + r,
    <l:Expr> "-" <r:Factor> => l - r,
    Factor,
};

Factor: i32 = {
    <n:r"[0-9]+"> => n.parse().unwrap(),
    "(" <Expr> ")",
};

nom example:

use nom::{
    IResult,
    character::complete::digit1 as digit,
    combinator::map_res,
    sequence::delimited,
    branch::alt,
    multi::fold_many0,
};

fn expr(input: &str) -> IResult<&str, i32> {
    let (input, init) = factor(input)?;
    fold_many0(
        alt((
            |input| {
                let (input, _) = tag("+")(input)?;
                let (input, factor) = factor(input)?;
                Ok((input, factor))
            },
            |input| {
                let (input, _) = tag("-")(input)?;
                let (input, factor) = factor(input)?;
                Ok((input, -factor))
            },
        )),
        init,
        |acc, factor| acc + factor,
    )(input)
}

fn factor(input: &str) -> IResult<&str, i32> {
    alt((
        map_res(digit, |s: &str| s.parse::<i32>()),
        delimited(tag("("), expr, tag(")")),
    ))(input)
}

rust-peg

1,443

Parsing Expression Grammar (PEG) parser generator for Rust

Pros of rust-peg

Simpler syntax for defining grammars using PEG notation
Generates more efficient code for certain types of grammars
Better error reporting and recovery mechanisms

Cons of rust-peg

Less flexible than nom for complex parsing scenarios
Smaller community and fewer resources compared to nom
Limited support for streaming input

Code Comparison

rust-peg example:

peg::parser!{
  pub grammar list_parser() for str {
    rule number() -> u32
      = n:$(['0'..='9']+) {? n.parse().or(Err("invalid number")) }

    pub rule list() -> Vec<u32>
      = "[" v:(number() ** ",") "]" { v }
  }
}

nom example:

use nom::{
  IResult,
  sequence::delimited,
  character::complete::{char, digit1},
  multi::separated_list0,
};

fn number(input: &str) -> IResult<&str, u32> {
  let (input, num_str) = digit1(input)?;
  Ok((input, num_str.parse().unwrap()))
}

fn list(input: &str) -> IResult<&str, Vec<u32>> {
  delimited(
    char('['),
    separated_list0(char(','), number),
    char(']')
  )(input)
}

Both examples demonstrate parsing a list of numbers enclosed in square brackets. rust-peg uses a more declarative syntax, while nom employs a combinator-based approach.

combine

1,290

A parser combinator library for Rust

Pros of combine

More flexible and composable parser combinators
Better support for error recovery and partial parsing
Easier to integrate with custom input types

Cons of combine

Steeper learning curve due to more complex API
Slightly slower performance in some scenarios
Less extensive documentation compared to nom

Code comparison

nom:

named!(parse_pair<&str, (i32, i32)>,
    do_parse!(
        x: digit >> tag!(",") >> y: digit >>
        ((x.parse().unwrap(), y.parse().unwrap()))
    )
);

combine:

let parse_pair = (
    i32::from_str,
    char(','),
    i32::from_str
).map(|(x, _, y)| (x, y));

Both nom and combine are popular parser combinator libraries for Rust. nom is known for its performance and extensive documentation, while combine offers more flexibility and composability. nom uses macros extensively, which can be easier for beginners but may lead to less readable code. combine's approach is more idiomatic Rust, using traits and functions.

nom is generally faster for simple parsers, but combine can be more efficient for complex grammars due to its better error handling and recovery. combine also provides better support for partial parsing and working with custom input types.

Ultimately, the choice between nom and combine depends on the specific requirements of your project and personal preference in API design.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

nom, eating data byte by byte

nom is a parser combinators library written in Rust. Its goal is to provide tools to build safe parsers without compromising the speed or memory consumption. To that end, it uses extensively Rust's strong typing and memory safety to produce fast and correct parsers, and provides functions, macros and traits to abstract most of the error prone plumbing.

nom logo in CC0 license, by Ange Albertini

nom will happily take a byte out of your files :)

Example
Documentation
Why use nom?
Parser combinators
Technical features
Rust version requirements
Installation
Related projects
Parsers written with nom
Contributors

Example

Hexadecimal color parser:

use nom::{
  bytes::complete::{tag, take_while_m_n},
  combinator::map_res,
  sequence::Tuple,
  IResult,
  Parser,
};

#[derive(Debug, PartialEq)]
pub struct Color {
  pub red: u8,
  pub green: u8,
  pub blue: u8,
}

fn from_hex(input: &str) -> Result<u8, std::num::ParseIntError> {
  u8::from_str_radix(input, 16)
}

fn is_hex_digit(c: char) -> bool {
  c.is_digit(16)
}

fn hex_primary(input: &str) -> IResult<&str, u8> {
  map_res(
    take_while_m_n(2, 2, is_hex_digit),
    from_hex
  ).parse(input)
}

fn hex_color(input: &str) -> IResult<&str, Color> {
  let (input, _) = tag("#")(input)?;
  let (input, (red, green, blue)) = (hex_primary, hex_primary, hex_primary).parse(input)?;
  Ok((input, Color { red, green, blue }))
}

fn main() {
  println!("{:?}", hex_color("#2F14DF"))
}

#[test]
fn parse_color() {
  assert_eq!(
    hex_color("#2F14DF"),
    Ok((
      "",
      Color {
        red: 47,
        green: 20,
        blue: 223,
      }
    ))
  );
}

Documentation

If you need any help developing your parsers, please ping geal on IRC (Libera, Geeknode, OFTC), go to #nom-parsers on Libera IRC, or on the Gitter chat room.

Why use nom

If you want to write:

Binary format parsers

nom was designed to properly parse binary formats from the beginning. Compared to the usual handwritten C parsers, nom parsers are just as fast, free from buffer overflow vulnerabilities, and handle common patterns for you:

TLV
Bit level parsing
Hexadecimal viewer in the debugging macros for easy data analysis
Streaming parsers for network formats and huge files

Example projects:

Text format parsers

While nom was made for binary format at first, it soon grew to work just as well with text formats. From line based formats like CSV, to more complex, nested formats such as JSON, nom can manage it, and provides you with useful tools:

Fast case insensitive comparison
Recognizers for escaped strings
Regular expressions can be embedded in nom parsers to represent complex character patterns succinctly
Special care has been given to managing non ASCII characters properly

Example projects:

Programming language parsers

While programming language parsers are usually written manually for more flexibility and performance, nom can be (and has been successfully) used as a prototyping parser for a language.

nom will get you started quickly with powerful custom error types, that you can leverage with nom_locate to pinpoint the exact line and column of the error. No need for separate tokenizing, lexing and parsing phases: nom can automatically handle whitespace parsing, and construct an AST in place.

Example projects:

Streaming formats

While a lot of formats (and the code handling them) assume that they can fit the complete data in memory, there are formats for which we only get a part of the data at once, like network formats, or huge files. nom has been designed for a correct behaviour with partial data: If there is not enough data to decide, nom will tell you it needs more instead of silently returning a wrong result. Whether your data comes entirely or in chunks, the result should be the same.

It allows you to build powerful, deterministic state machines for your protocols.

Example projects:

Parser combinators

Parser combinators are an approach to parsers that is very different from software like lex and yacc. Instead of writing the grammar in a separate file and generating the corresponding code, you use very small functions with very specific purpose, like "take 5 bytes", or "recognize the word 'HTTP'", and assemble them in meaningful patterns like "recognize 'HTTP', then a space, then a version". The resulting code is small, and looks like the grammar you would have written with other parser approaches.

This has a few advantages:

The parsers are small and easy to write
The parsers components are easy to reuse (if they're general enough, please add them to nom!)
The parsers components are easy to test separately (unit tests and property-based tests)
The parser combination code looks close to the grammar you would have written
You can build partial parsers, specific to the data you need at the moment, and ignore the rest

Technical features

nom parsers are for:

byte-oriented: The basic type is &[u8] and parsers will work as much as possible on byte array slices (but are not limited to them)
bit-oriented: nom can address a byte slice as a bit stream
string-oriented: The same kind of combinators can apply on UTF-8 strings as well
zero-copy: If a parser returns a subset of its input data, it will return a slice of that input, without copying
streaming: nom can work on partial data and detect when it needs more data to produce a correct result
descriptive errors: The parsers can aggregate a list of error codes with pointers to the incriminated input slice. Those error lists can be pattern matched to provide useful messages.
custom error types: You can provide a specific type to improve errors returned by parsers
safe parsing: nom leverages Rust's safe memory handling and powerful types, and parsers are routinely fuzzed and tested with real world data. So far, the only flaws found by fuzzing were in code written outside of nom
speed: Benchmarks have shown that nom parsers often outperform many parser combinators library like Parsec and attoparsec, some regular expression engines and even handwritten C parsers

Some benchmarks are available on GitHub.

Rust version requirements (MSRV)

The 7.0 series of nom supports Rustc version 1.56 or greater.

The current policy is that this will only be updated in the next major nom release.

Installation

nom is available on crates.io and can be included in your Cargo enabled project like this:

[dependencies]
nom = "7"

There are a few compilation features:

alloc: (activated by default) if disabled, nom can work in no_std builds without memory allocators. If enabled, combinators that allocate (like many0) will be available
std: (activated by default, activates alloc too) if disabled, nom can work in no_std builds

You can configure those features like this:

[dependencies.nom]
version = "7"
default-features = false
features = ["alloc"]

Related projects

Parsers written with nom

Here is a (non exhaustive) list of known projects using nom:

Text file formats: Ceph Crush, Cronenberg, Email, XFS Runtime Stats, CSV, FASTA, FASTQ, INI, ISO 8601 dates, libconfig-like configuration file format, Web archive, PDB, proto files, Fountain screenplay markup, vimwiki & vimwiki_macros, Kconfig language, Askama templates
Programming languages: PHP, Basic Calculator, GLSL, Lua, Python, SQL, Elm, SystemVerilog, Turtle, CSML, Wasm, Pseudocode, Filter for MeiliSearch, PotterScript
Interface definition formats: Thrift
Audio, video and image formats: GIF, MagicaVoxel .vox, MIDI, SWF, WAVE, Matroska (MKV), Exif/Metadata parser for JPEG/HEIF/HEIC/MOV/MP4
Document formats: TAR, GZ, GDSII
Cryptographic formats: X.509
Network protocol formats: Bencode, D-Bus, DHCP, HTTP, URI, IMAP (alt), IRC, Pcap-NG, Pcap, Pcap + PcapNG, IKEv2, NTP, SNMP, Kerberos v5, DER, TLS, IPFIX / Netflow v10, GTP, SIP, SMTP, Prometheus
Language specifications: BNF
Misc formats: Game Boy ROM, ANT FIT, Version Numbers, Telcordia/Bellcore SR-4731 SOR OTDR files, MySQL binary log, URI, Furigana, Wordle Result, NBT

Want to create a new parser using nom? A list of not yet implemented formats is available here.

Want to add your parser here? Create a pull request for it!

Contributors

nom is the fruit of the work of many contributors over the years, many thanks for your help!

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot