combine

A parser combinator library for Rust

1,290

View on GitHub

Top Related Projects

nom

9,335

Rust parser combinator framework

Quick Overview

The combine crate is a Rust library that provides a set of combinators for working with iterators and streams. It offers a wide range of functionality, including parallel processing, error handling, and more, making it a powerful tool for working with data in a Rust-based application.

Pros

Comprehensive Functionality: The combine crate provides a rich set of combinators and utilities for working with iterators and streams, covering a wide range of use cases.
Parallelism: The library supports parallel processing of data, allowing for efficient utilization of multi-core systems.
Error Handling: The combine crate includes robust error handling mechanisms, making it easier to manage and propagate errors in complex data processing pipelines.
Flexibility: The library is designed to be highly composable, allowing developers to build custom data processing workflows by combining various combinators.

Cons

Steep Learning Curve: The extensive set of combinators and the functional programming-style of the library may present a steeper learning curve for developers new to Rust or functional programming.
Performance Overhead: The flexibility and abstraction provided by the combine crate may introduce some performance overhead compared to more low-level, manual iterator manipulation.
Dependency Management: The combine crate has a relatively large number of dependencies, which may increase the complexity of managing project dependencies.
Documentation: While the library is well-documented, some users may find the documentation could be more beginner-friendly or provide more practical examples.

Code Examples

Here are a few examples of how to use the combine crate:

Parallel Processing:

use combine::stream::easy;
use combine::parser::char::digit;
use combine::{many1, parser};

fn main() {
    let input = "123 456 789";
    let result: Vec<i32> = parser(many1(digit()))
        .easy_parse(input)
        .unwrap()
        .0
        .into_par_iter()
        .map(|c| c.parse().unwrap())
        .collect();

    println!("{:?}", result); // Output: [123, 456, 789]
}

Error Handling:

use combine::error::ParseError;
use combine::parser::char::{digit, string};
use combine::{choice, many1, parser, Parser};

fn parse_number() -> impl Parser<&str, Output = i32, Error = ParseError<&str>> {
    many1(digit()).and_then(|digits: String| digits.parse())
}

fn parse_input() -> impl Parser<&str, Output = (i32, i32), Error = ParseError<&str>> {
    (parse_number(), string(" "), parse_number())
        .map(|(a, _, b)| (a, b))
}

fn main() {
    let input = "123 456";
    let result = parser(parse_input()).easy_parse(input);
    match result {
        Ok((output, _)) => println!("{:?}", output), // Output: (123, 456)
        Err(err) => println!("Error: {:?}", err),
    }
}

Composing Combinators:

use combine::parser::char::{digit, space};
use combine::{many, parser, sep_by, Parser};

fn parse_numbers() -> impl Parser<&str, Output = Vec<i32>, Error = combine::error::ParseError<&str>> {
    sep_by(
        many(digit().map(|c: char| c.to_digit(10).unwrap() as i32)),
        space(),
    )
}

fn main() {
    let input = "123 456 789";
    let result = parser(parse_numbers()).easy_parse(input);
    match result {
        Ok((output, _)) => println!("{:?}", output), // Output: [123, 456, 789]
        Err(err) => println!("Error: {:?}", err),
    }
}

Getting Started

To use the combine crate in your Rust project, add the following to

Competitor Comparisons

nom

9,335

Rust parser combinator framework

Pros of nom

nom is a parser combinator library, which allows for the creation of complex parsers in a modular and composable way.
nom has a large and active community, with a wide range of available parsers and utilities.
nom is known for its performance, with benchmarks showing it to be faster than many other parser combinator libraries.

Cons of nom

nom has a steeper learning curve than Combine, as it uses a more low-level and explicit approach to parser construction.
nom may require more boilerplate code to set up and configure, compared to the more high-level and abstracted Combine.

Code Comparison

Combine:

let parser = choice!(
    tag("true").map(|_| true),
    tag("false").map(|_| false),
);

nom:

fn parse_bool(input: &str) -> IResult<&str, bool> {
    alt!(input,
        tag("true") => { |_| true } |
        tag("false") => { |_| false }
    )
}

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

combine

An implementation of parser combinators for Rust, inspired by the Haskell library Parsec. As in Parsec the parsers are LL(1) by default but they can opt-in to arbitrary lookahead using the attempt combinator.

Example

extern crate combine;
use combine::{many1, Parser, sep_by};
use combine::parser::char::{letter, space};

// Construct a parser that parses *many* (and at least *1) *letter*s
let word = many1(letter());

// Construct a parser that parses many *word*s where each word is *separated by* a (white)*space*
let mut parser = sep_by(word, space())
    // Combine can collect into any type implementing `Default + Extend` so we need to assist rustc
    // by telling it that `sep_by` should collect into a `Vec` and `many1` should collect to a `String`
    .map(|mut words: Vec<String>| words.pop());
let result = parser.parse("Pick up that word!");
// `parse` returns `Result` where `Ok` contains a tuple of the parsers output and any remaining input.
assert_eq!(result, Ok((Some("word".to_string()), "!")));

Larger examples can be found in the examples, tests and benches folders.

Tutorial

A tutorial as well as explanations on what goes on inside combine can be found in the wiki.

Translation

Japanese

Links

Documentation and examples

crates.io

Features

Parse arbitrary streams - Combine can parse anything from &[u8] and &str to iterators and Read instances. If none of the builtin streams fit your use case you can even implement a couple traits your self to create your own custom stream!
zero-copy parsing - When parsing in memory data, combine can parse without copying. See the range module for parsers specialized for zero-copy parsing.
partial parsing - Combine parsers can be stopped at any point during parsing and later be resumed without losing any progress. This makes it possible to start parsing partial data coming from an io device such as a socket without worrying about if enough data is present to complete the parse. If more data is needed the parser will stop and may be resumed at the same point once more data is available. See the async example for an example and this post for an introduction.

About

A parser combinator is, broadly speaking, a function which takes several parsers as arguments and returns a new parser, created by combining those parsers. For instance, the many parser takes one parser, p, as input and returns a new parser which applies p zero or more times. Thanks to the modularity that parser combinators gives it is possible to define parsers for a wide range of tasks without needing to implement the low level plumbing while still having the full power of Rust when you need it.

The library adheres to semantic versioning.

If you end up trying it I welcome any feedback from your experience with it. I am usually reachable within a day by opening an issue, sending an email or posting a message on Gitter.

FAQ

Why does my errors contain inscrutable positions?

Since combine aims to crate parsers with little to no overhead, streams over &str and &[T] do not carry any extra position information, but instead, they only rely on comparing the pointer of the buffer to check which Stream is further ahead than another Stream. To retrieve a better position, either call translate_position on the PointerOffset which represents the position or wrap your stream with State.

How does it compare to nom?

https://github.com/Marwes/combine/issues/73 contains discussion and links to comparisons to nom.

Parsers written in combine

Formats and protocols

GraphQL https://github.com/graphql-rust/graphql-parser (Uses a custom tokenizer as input)
DiffX https://github.com/brennie/diffx-rs
Redis https://github.com/mitsuhiko/redis-rs/pull/141 (Uses partial parsing)
Toml https://github.com/ordian/toml_edit
Maker Interchange Format https://github.com/aidanhs/frametool (Uses combine as a lexer)
Javascript https://github.com/freemasen/ress
JPEG Metadata https://github.com/vadixidav/exifsd

Miscellaneous

Template language https://github.com/tailhook/trimmer
Code exercises https://github.com/dgel/adventOfCode2017
Programming language
- https://github.com/MaikKlein/spire-lang
- https://github.com/vadixidav/typeflow/tree/master/lang
Query parser (+ more) https://github.com/mozilla/mentat
Query parser https://github.com/tantivy-search/tantivy

Extra

There is an additional crate which has parsers to lex and parse programming languages in combine-language.

Contributing

The easiest way to contribute is to just open an issue about any problems you encounter using combine but if you are interested in adding something to the library here is a list of some of the easier things to work on to get started.

Add additional parsers If you have a suggestion for another parser just open an issue or a PR with an implementation.
Add additional examples More examples for using combine will always be useful!
Add and improve the docs Not the fanciest of work but one cannot overstate the importance of good documentation.

Top Related Projects

nom

9,335

Rust parser combinator framework

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot