Parse a single token of any type and return it.

By default, parsers will not backtrack if they fail after consuming at least one token. Passing a parser to backtrackable will change this behaviour and allows us to jump back to the state of the parser before it consumed any input and try another one.

This is most useful when you want to quickly try a few different parsers using one_of.

🚨 Backtracking parsers can drastically reduce performance, so you should avoid them where possible. A common reason folks reach for backtracking is when they want to try multiple branches that start with the same token or same sequence of tokens.

To avoid backtracking in these cases, you can create an intermediate parser that consumes the common tokens and then use one_of to try the different branches.

Do the first parser, then apply f to its result, which returns another parser and is subsequently run. If the first parser fails, f is not called. Because this parser is so common, we normally import it unqualified.

This is very useful for running parsers in sequence with Gleam’s use syntax.

import chomp.{do, return}

fn point() {
  use _ <- do(chomp.token(LParen))
  use x <- do(int_parser())
  use _ <- do(chomp.token(Comma))
  use y <- do(int_parser())
  use _ <- do(chomp.token(RParen))

  return(Point(x, y))
}

(See the main page for a complete example)

A combination of do and in.

Parse successfully only if at the end of the token stream.

Create a parser that consumes no tokens and always fails with the given error.

A parser that returns the current token position.

Run a parser in a certain context. This allows you to include useful information—context—in error messages. For example, instead of a message such as “I expected a value” you could say “I expected a value inside of this list in the function foo”. The latter is far more user-friendly!

Context also holds on to the position of the parser where it was entered. So if you had an InList context entered right after a [ token, any errors encountered inside the list can reference where the list started.

I found a problem in this list:

| the list starts here
v
[a, b, ]
       ~
       |
I wanted a value

Run the given parser and then inspect it’s state.

Returns True if there are no tokens left to consume.

Defer the creation of a parser until it is needed. This is often most useful when creating a parser that is recursive and is not a function.

Parse zero or more of the given parser.

Parse one or more of the given parser.

💡 If this parser succeeds, the list produced is guaranteed to be non-empty. Feel free to let assert the result!

Run parser, applying f to its result.

Run a parser and if it fails and did not consume any tokens, apply the given function to the error and return the new error. This function provides a way to add more information to error messages. For example, if you just parsed some indentation in an indentation-sensitive language and now want to parse a statement, you could map the statement parser’s error to add extra info:

fn parse_statement() {
  // parse a statement
  |> chomp.or_error("I wanted a statement")
}

fn parse_something() {
  use _ <- do(parse_indentation())
  use statement <- do(
    parse_statement() |> chomp.map_error(because_of_indent)
  )
  // ...
}

fn because_of_indent(error) {
  case error {
    chomp.Custom(error) ->
      chomp.Custom("Since there was an indent, " <> error)
    _ -> error
  }
}

Now if parsing the statement in parse_something fails, the error message will be Since there was an indent, I wanted a statement. Sweet!

Note that, like or_error, EndOfInput errors are not changed to minimize confusing error messages.

Try each parser in order until one succeeds. If none succeed, the last parser’s error is returned. It is recommended to use a custom error with or_error for better error messages.

Try the given parser, but if it fails return None instead of failing.

Try the given parser, but if it fails return the given default value instead of failing.

Run a parser and if it fails and did not consume any tokens, return the given error. This function is extremely useful for providing custom error messages to parsers such as one_of, take_map, and take_if.

Note that EndOfInput errors are not replaced to minimize confusing error messages.

import chomp

fn value() {
  chomp.one_of([
    string(),
    number(),
    variable(),
    list(),
    group(),
  ])
  |> chomp.or_error("I expected a value (string, number, variable, or list)")
}

Run parser, replacing its result with b.

Run a parser and if it fails, return the given error. This parser is similar to or_error, but it’s not quite the same—this one does not care whether the parser consumed any tokens when it failed. That means that if you have a parser like this:

chomp.one_of([function(), constant()])
|> chomp.replace_error("I expected a function or constant")

You will always get the error message “I expected a function or constant” even if any of the one_of parsers consumed tokens—a behavior that probably isn’t what you want.

For instance, if this was parsed:

function 23() {}
//       ^^ this is the error

We would get the error message “I expected a function or constant” because the function parser’s error was swallowed by replace_error. Contrast that to or_error, where we would get something much better such as “I expected an identifier”.

Of course there are cases where replace_error can be helpful; just be aware that you’ll often want to reach for or_error first.

The simplest kind of parser. return consumes no tokens and always produces the given value. Sometimes called succeed instead.

This function might seem useless at first, but it is very useful when used in combination with do or then.

import chomp.{do, return}

fn unit8_parser() {
  use int <- do(int_parser())

  case int >= 0, int <= 255 {
    True, True ->
      return(int)

    False, _ ->
      throw("Expected an int >= 0")

    _, False ->
      throw("Expected an int <= 255")
 }
}

💡 return and succeed are names for the same thing. We suggesting using return unqualified when using do and Gleam’s use syntax, and chomp.succeed in a pipeline with chomp.then.

Parsers don’t do anything until they’re run! The run function takes a Parser and a list of Tokens and runs it; returning either the parsed value or a tuple of the Error with the position where the parser failed and the final context stack.

Parse a sequence separated by the separator parser.

💡 succeed and return are names for the same thing. We suggest using succeed in a pipeline with chomp.then, and return unqalified when using do with Gleam’s use syntax.

Take and return a token if it satisfies the predicate.

Take the next token and attempt to transform it with the given function. This is useful when creating reusable primtive parsers for your own tokens such as take_identifier or take_number.

Another name for do.

The opposite of return, this parser always fails with the given error. Sometimes called fail instead.

Parse a token of a particular type, returning its position.

use start_pos <- do(chomp.token(LParen))
// ...
use end_pos <- do(chomp.token(RParen))

Parse until the given token is encountered, returning a list of the results.

Parse until the end of the token stream, returning a list of the results.

💡 This parser produces better error messages than using both many and end in order, so you’ll often want to use this function instead. A common use-case is when you’re parsing top-level statements or expressions in a programming language until the end of the file.

use statements <- do(chomp.until_end(statement()))
return(statements)

// is generally better than

use statements <- do(chomp.many(statement()))
use _ <- do(chomp.end())
return(statements)