nibble

Types

DeadEnd

A dead end represents a the point where a parser that had committed down a path failed. It contains the position of the failure, the Error describing the failure, and the context stack for any parsers that had run.

pub type DeadEnd(tok, ctx) {
  DeadEnd(
    pos: Span,
    problem: Error(tok),
    context: List(#(Span, ctx)),
  )
}

Constructors

DeadEnd(
  pos: Span,
  problem: Error(tok),
  context: List(#(Span, ctx)),
)

Error

</>

pub type Error(tok) {
  BadParser(String)
  Custom(String)
  EndOfInput
  Expected(String, got: tok)
  Unexpected(tok)
}

Constructors

```
BadParser(String)
```
```
Custom(String)
```
```
EndOfInput
```
```
Expected(String, got: tok)
```
```
Unexpected(tok)
```

Loop

</>

pub type Loop(a, state) {
  Continue(state)
  Break(a)
}

Constructors

```
Continue(state)
```
```
Break(a)
```

Parser

opaque </>

The Parser type has three parameters, let’s take a look at each of them:

Parser(a, tok, ctx)
// (1) ^
// (2)    ^^^
// (3)         ^^^

a is the type of value that the parser knows how to produce. If you were writing a parser for a programming language, this might be your expression type.
tok is the type of tokens that the parser knows how to consume. You can take a look at the Token type for a bit more info, but note that it’s not necessary for the token stream to come from nibble’s lexer.
ctx is used to make error reporting nicer. You can place a parser into a custom context. When the parser runs the context gets pushed into a stack. If the parser fails you can see the context stack in the error message, which can make error reporting and debugging much easier!

pub opaque type Parser(a, tok, ctx)

Functions

any

</>

pub fn any() -> Parser(a, a, b)

Returns the next token in the input stream. Fails if there are no more tokens.

backtrackable

</>

pub fn backtrackable(parser: Parser(a, b, c)) -> Parser(a, b, c)

By default, parsers will not backtrack if they fail after consuming at least one token. Passing a parser to backtrackable will change this behaviour and allows us to jump back to the state of the parser before it consumed any input and try another one.

This is most useful when you want to quickly try a few different parsers using one_of.

🚨 Backtracing parsers can drastically reduce performance, so you should avoid them where possible. A common reason folks reach for backtracking is when they want to try multiple branches that start with the same token or same sequence of tokens.

To avoid backtracking in these cases, you can create an intermediate parser that consumes the common tokens and then use one_of to try the different branches.

do

</>

pub fn do(
  parser: Parser(a, b, c),
  f: fn(a) -> Parser(d, b, c),
) -> Parser(d, b, c)

do_in

</>

pub fn do_in(
  context: a,
  parser: Parser(b, c, a),
  f: fn(b) -> Parser(d, c, a),
) -> Parser(d, c, a)

eof

</>

pub fn eof() -> Parser(Nil, a, b)

Succeeds if the input stream is empty, fails otherwise. This is useful to verify that you’ve consumed all the tokens in the input stream.

fail

</>

pub fn fail(message: String) -> Parser(a, b, c)

Create a parser that consumes no tokens and always fails with the given error message.

guard

</>

pub fn guard(cond: Bool, expecting: String) -> Parser(Nil, a, b)

Fails if the given condition is false, otherwise returns Nil.

in

</>

pub fn in(parser: Parser(a, b, c), context: c) -> Parser(a, b, c)

inspect

</>

pub fn inspect(
  parser: Parser(a, b, c),
  message: String,
) -> Parser(a, b, c)

Run the given parser and then inspect it’s state.

lazy

</>

pub fn lazy(parser: fn() -> Parser(a, b, c)) -> Parser(a, b, c)

Defer the creation of a parser until it is needed. This is often most useful when creating a parser that is recursive and is not a function.

loop

</>

pub fn loop(
  init: a,
  step: fn(a) -> Parser(Loop(b, a), c, d),
) -> Parser(b, c, d)

many

</>

pub fn many(parser: Parser(a, b, c)) -> Parser(List(a), b, c)

Returns consecutive applications of the given parser. If you are parsing values with a separator, use sequence instead.

💡 This parser can succeed without consuming any input. You can end up with an infinite loop if you’re not careful. Use many1 if you want to guarantee you take at least one token.

many1

</>

pub fn many1(parser: Parser(a, b, c)) -> Parser(List(a), b, c)

This is the same as many1, but is guaranteed to return at least one value.

map

</>

pub fn map(
  parser: Parser(a, b, c),
  f: fn(a) -> d,
) -> Parser(d, b, c)

one_of

</>

pub fn one_of(parsers: List(Parser(a, b, c))) -> Parser(a, b, c)

Try the given parsers in order until one succeeds. If all fail, the parser fails.

optional

</>

pub fn optional(
  parser: Parser(a, b, c),
) -> Parser(Option(a), b, c)

Try the given parser, but if it fails return None instead of failing.

or

</>

pub fn or(parser: Parser(a, b, c), default: a) -> Parser(a, b, c)

Try the given parser, but if it fails return the given default value instead of failing.

replace

</>

pub fn replace(
  parser: Parser(a, b, c),
  with b: d,
) -> Parser(d, b, c)

return

</>

pub fn return(value: a) -> Parser(a, b, c)

The simplest kind of parser. return consumes no tokens and always produces the given value. Sometimes called succeed instead.

This function might seem useless at first, but it is very useful when used in combination with do or then.

import nibble.{do, return}

fn unit8_parser() {
  use int <- do(int_parser())

  case int >= 0, int <= 255 {
    True, True ->
      return(int)

    False, _ ->
      throw("Expected an int >= 0")

    _, False ->
      throw("Expected an int <= 255")
 }
}

💡 return and succeed are names for the same thing. We suggesting using return unqualified when using do and Gleam’s use syntax, and nibble.succeed in a pipeline with nibble.then.

run

</>

pub fn run(
  src: List(Token(a)),
  parser: Parser(b, a, c),
) -> Result(b, List(DeadEnd(a, c)))

Parsers don’t do anything until they’re run! The run function takes a Parser and a list of Tokens and runs it; returning either the parsed value or a list of DeadEnds where the parser failed.

sequence

</>

pub fn sequence(
  parser: Parser(a, b, c),
  separator sep: Parser(d, b, c),
) -> Parser(List(a), b, c)

Consumes a sequence of tokens using the given parser, separated by the given separator parser. Returns a list of the parsed values, ignoring the results of the separator parser.

span

</>

pub fn span() -> Parser(Span, a, b)

A parser that returns the current token position.

succeed

</>

pub fn succeed(value: a) -> Parser(a, b, c)

The simplest kind of parser. succeed consumes no tokens and always produces the given value. Sometimes called return instead.

This function might seem useless at first, but it is very useful when used in combination with do or then.

import nibble

fn unit8_parser() {
  int_parser()
  |> nibble.then(fn(int) {
    case int >= 0, int <= 255 {
      True, True -> succeed(int)
      False, _ -> fail("Expected an int >= 0")
      _, False -> fail("Expected an int <= 255")
    }
  })
}

💡 succeed and return are names for the same thing. We suggest using succeed in a pipeline with nibble.then, and return unqalified when using do with Gleam’s use syntax.

take_at_least

</>

pub fn take_at_least(
  parser: Parser(a, b, c),
  count: Int,
) -> Parser(List(a), b, c)

Apply the parser a minimum of count times, returning a list of the results.

take_exactly

</>

pub fn take_exactly(
  parser: Parser(a, b, c),
  count: Int,
) -> Parser(List(a), b, c)

Take count consecutive tokens from the stream using the given parser.

take_if

</>

pub fn take_if(
  expecting: String,
  predicate: fn(a) -> Bool,
) -> Parser(a, a, b)

Takes the next token off the stream if it satisfies the given predicate.

take_map

</>

pub fn take_map(
  expecting: String,
  f: fn(a) -> Option(b),
) -> Parser(b, a, c)

Take the next token and attempt to transform it with the given function. This is useful when creating reusable primitive parsers for your own tokens such as take_identifier or take_number.

take_map_while

</>

pub fn take_map_while(
  f: fn(a) -> Option(b),
) -> Parser(List(b), a, c)

Applies a function to consecutive tokens while the given function returns Some.

💡 This parser can succeed without consuming any input (if the predicate immediately succeeds). You can end up with an infinite loop if you’re not careful. Use take_map_while1 if you want to guarantee you take at least one token.

take_map_while1

</>

pub fn take_map_while1(
  expecting: String,
  f: fn(a) -> Option(b),
) -> Parser(List(b), a, c)

Applies a function to consecutive tokens while the given function returns Some.

💡 If this parser succeeds, the list produced is guaranteed to be non-empty. Feel free to let assert the result!

take_until

</>

pub fn take_until(
  predicate: fn(a) -> Bool,
) -> Parser(List(a), a, b)

Take token from the stream until the given predicate is satisfied.

💡 This parser can succeed without consuming any input (if the predicate immediately succeeds). You can end up with an infinite loop if you’re not careful. Use take_until1 if you want to guarantee you take at least one token.

take_until1

</>

pub fn take_until1(
  expecting: String,
  predicate: fn(a) -> Bool,
) -> Parser(List(a), a, b)

Take token from the stream until the given predicate is satisfied.

💡 If this parser succeeds, the list produced is guaranteed to be non-empty. Feel free to let assert the result!

take_up_to

</>

pub fn take_up_to(
  parser: Parser(a, b, c),
  count: Int,
) -> Parser(List(a), b, c)

Apply the parser up to count times, returning a list of the results.

💡 This parser can succeed without consuming any input (if the parser fails immediately) and return an empty list. You can end up with an infinite loop if you’re not careful.

take_while

</>

pub fn take_while(
  predicate: fn(a) -> Bool,
) -> Parser(List(a), a, b)

Take tokens from the stream while the given predicate is satisfied.

💡 This parser can succeed without consuming any input (if the predicate immediately fails). You can end up with an infinite loop if you’re not careful. Use take_while1 if you want to guarantee you take at least one token.

take_while1

</>

pub fn take_while1(
  expecting: String,
  predicate: fn(a) -> Bool,
) -> Parser(List(a), a, b)

Take tokens from the stream while the given predicate is satisfied.

💡 If this parser succeeds, the list produced is guaranteed to be non-empty. Feel free to let assert the result!

then

</>

pub fn then(
  parser: Parser(a, b, c),
  f: fn(a) -> Parser(d, b, c),
) -> Parser(d, b, c)

throw

</>

pub fn throw(message: String) -> Parser(a, b, c)

The opposite of return, this parser always fails with the given message. Sometimes called fail instead.

token

</>

pub fn token(tok: a) -> Parser(Nil, a, b)

Returns nil if the token tok is the next token in the input stream. Fails if the next token is not tok or if the input stream is empty.