Ergo.Combinators (Ergo v0.9.9)
Ergo.Combinators
is the key set of parsers used for combining together other parsers.
Parsers
- choice
- sequence
- many
- optional
- ignore
- transform
- lookeahead
- not_lookahead
Link to this section Summary
Functions
The string/1 parser takes a parser that returns an AST which is a string and converts the AST into an atom.
The choice/2
parser takes a list of parsers and a list (defaults to []) of
options. It tries to match each in turn, returning :ok if it does. Otherwise,
if none match, it returns :error.
The hoist/1
parser takes a parser expected to return an AST which is a
1-item list. The returned parser extracts the item from the list and
returns an AST of just that item.
The ignore/1 parser matches but ignores the AST of its child parser.
The lazy/1 parser is intended for use in cases where constructing parsers
creates a recursive call. By using lazy
the original parser call is
deferred until later, breaking the infinite recursion.
The lookahead
parser accepts a parser and matches it but does not update the context when it succeeds.
The many/2
parser takes a parser and attempts to match it on the input
multiple (including possibly zero) times. Accepts options min:
and max:
to modify how many times the included parser must match for many
to
match.
The not_lookahead
parser accepts a parser and attempts to match it. If the match fails the not_lookahead parser returns status: :ok but does not affect the context otherwise.
The optional/2 parser takes a parser and attempts to match it and succeeds whether or not it can.
The replace/3 combinator replaces the AST value of it's child with a constant.
The satisfy/3 parser takes a parser and a predicate function. If the parser is successful the AST is passed to the predicate function. If the predicate function returns true the parser returns the successful context, otherwise an error context is returned.
The sequence/2
parser takes a list of parsers which are applied in turn.
If any of the parsers fails the sequence itself fails.
The string/1 parser takes a parser that returns an AST which is a list of characters and converts the AST into a string.
The transform/2
parser runs a transforming function on the AST of its child parser.
Link to this section Functions
atom(parser)
The string/1 parser takes a parser that returns an AST which is a string and converts the AST into an atom.
Examples
iex> alias Ergo
iex> alias Ergo.Context
iex> import Ergo.{Terminals, Combinators}
iex> parser = many(wc()) |> string() |> atom()
iex> assert %Context{status: :ok, ast: :fourty_two} = Ergo.parse(parser, "fourty_two")
choice(parsers, opts \\ [])
The choice/2
parser takes a list of parsers and a list (defaults to []) of
options. It tries to match each in turn, returning :ok if it does. Otherwise,
if none match, it returns :error.
Valid options are:
label: "text"
a label output in debuggingast: fn
a function passed the AST value for processing if the parser succeedsctx: fn
a function passed theContext
for processing if the parser succeedserr: fn
a function passed theContext
for processing if the parser fails
Examples
iex> alias Ergo.Context
iex> import Ergo.{Terminals, Combinators}
iex> parser = choice([literal("Foo"), literal("Bar"), literal("Hello"), literal("World")], label: "Foo|Bar|Hello|World")
iex> context = Ergo.parse(parser, "Hello World")
iex> assert %Context{status: :ok, ast: "Hello", input: " World", index: 5, col: 6} = context
iex> alias Ergo.Context
iex> import Ergo.{Terminals, Combinators}
iex> parser = choice([literal("Hello"), literal("World")], ast: &String.upcase/1)
iex> context = Ergo.parse(parser, "Hello World")
iex> assert %Context{status: :ok, ast: "HELLO"} = context
iex> alias Ergo.Context
iex> import Ergo.{Terminals, Combinators}
iex> fun = fn %Context{ast: ast} = ctx -> %{ctx | ast: String.upcase(ast)} end
iex> parser = choice([literal("Hello"), literal("World")], ctx: fun)
iex> context = Ergo.parse(parser, "Hello World")
iex> assert %Context{status: :ok, ast: "HELLO"} = context
iex> alias Ergo.Context
iex> import Ergo.{Terminals, Combinators}
iex> parser = choice([literal("Foo"), literal("Bar")], label: "Foo|Bar")
iex> context = Ergo.parse(parser, "Hello World")
iex> %Context{status: {:error, [{:no_valid_choice, {1, 1}, "Foo|Bar did not find a valid choice"}]}, ast: nil, input: "Hello World"} = context
hoist(parser)
The hoist/1
parser takes a parser expected to return an AST which is a
1-item list. The returned parser extracts the item from the list and
returns an AST of just that item.
This often comes up with the sequence/2 parser and ignore, where all but one item in a sequence are ignored. Using hoist pulls that item up so that subsequent parsers don't need to deal with the list.
Examples
iex> alias Ergo
iex> alias Ergo.Context
iex> import Ergo.{Terminals, Combinators}
iex> parser = sequence([ignore(many(char(?a))), char(?b)]) |> hoist()
iex> assert %Context{status: :ok, ast: ?b} = Ergo.parse(parser, "aaaaaaaab")
ignore(parser, opts \\ [])
The ignore/1 parser matches but ignores the AST of its child parser.
Examples
iex> alias Ergo.Context
iex> import Ergo.{Terminals, Combinators}
iex> parser = sequence([literal("Hello"), ignore(ws()), literal("World")])
iex> context = Ergo.parse(parser, "Hello World")
iex> assert %Context{status: :ok, ast: ["Hello", "World"], index: 11, col: 12} = context
The lazy/1 parser is intended for use in cases where constructing parsers
creates a recursive call. By using lazy
the original parser call is
deferred until later, breaking the infinite recursion.
lookahead(parser, opts \\ [])
The lookahead
parser accepts a parser and matches it but does not update the context when it succeeds.
Example
iex> alias Ergo.Context
iex> import Ergo.{Combinators, Terminals}
iex> parser = lookahead(literal("Hello"))
iex> assert %Context{status: :ok, ast: nil, input: "Hello World", index: 0} = Ergo.parse(parser, "Hello World")
iex> alias Ergo.Context
iex> import Ergo.{Combinators, Terminals}
iex> parser = lookahead(literal("Helga"))
iex> assert %Context{status: {:error, [{:lookahead_fail, {1, 4}, _}, {:bad_literal, {1, 4}, _}, {:unexpected_char, {1, 4}, _}]}, index: 3, col: 4, input: "lo World"} = Ergo.parse(parser, "Hello World")
many(parser, opts \\ [])
The many/2
parser takes a parser and attempts to match it on the input
multiple (including possibly zero) times. Accepts options min:
and max:
to modify how many times the included parser must match for many
to
match.
Examples
This test will need to be rewritten in terms of Ergo.diganose
# iex> Logger.disable(self())
# iex> alias Ergo.Context
# iex> import Ergo.{Combinators, Terminals}
# iex> parser = many(wc(), label: "Chars")
# iex> context = Ergo.parse(parser, "Hello World")
# iex> assert %Context{status: :ok, ast: [?H, ?e, ?l, ?l, ?o], input: " World", index: 5, col: 6, char: ?o} = context
iex> alias Ergo.Context
iex> import Ergo.{Combinators, Terminals}
iex> parser = many(wc(), min: 6)
iex> context = Ergo.parse(parser, "Hello World")
iex> assert %Context{status: {:error, [{:many_less_than_min, {1, 6}, "5 < 6"}]}, ast: nil, input: " World", index: 5, col: 6} = context
iex> alias Ergo.{Context, Parser}
iex> import Ergo.{Combinators, Terminals}
iex> parser = many(wc(), max: 3)
iex> context = Ergo.parse(parser, "Hello World")
iex> assert %Context{status: :ok, ast: [?H, ?e, ?l], input: "lo World", index: 3, col: 4} = context
iex> alias Ergo.{Context, Parser}
iex> import Ergo.{Combinators, Terminals}
iex> parser = many(wc(), ast: &Enum.count/1)
iex> context = Ergo.parse(parser, "Hello World")
iex> assert %Context{status: :ok, ast: 5, input: " World", index: 5, col: 6} = context
not_lookahead(parser, opts \\ [])
The not_lookahead
parser accepts a parser and attempts to match it. If the match fails the not_lookahead parser returns status: :ok but does not affect the context otherwise.
If the match succeeds the not_lookahead
parser fails with {:error, :lookahead_fail}
Examples
iex> alias Ergo.Context iex> import Ergo.{Combinators, Terminals} iex> parser = not_lookahead(literal("Foo")) iex> assert %Context{status: :ok, input: "Hello World"} = Ergo.parse(parser, "Hello World")
iex> alias Ergo.{Context, Parser} iex> import Ergo.{Combinators, Terminals} iex> parser = not_lookahead(literal("Hello")) iex> assert %Context{status: {:error, [{:lookahead_fail, {1,6}, "Satisfied: literal<Hello>"}]}, input: " World"} = Ergo.parse(parser, "Hello World")
optional(parser, opts \\ [])
The optional/2 parser takes a parser and attempts to match it and succeeds whether or not it can.
Examples
iex> alias Ergo.Context
iex> import Ergo.{Terminals, Combinators}
iex> ctx = Ergo.parse(optional(literal("Hello")), "Hello World")
iex> assert %Context{status: :ok, ast: "Hello", input: " World", index: 5, col: 6} = ctx
In this example we deliberately ensure that the Context ast is not nil
iex> alias Ergo.{Context, Parser}
iex> import Ergo.{Terminals, Combinators}
iex> ctx = Context.new(" World", ast: [])
iex> parser = optional(literal("Hello"))
iex> new_context = Parser.invoke(ctx, parser)
iex> assert %Context{status: :ok, ast: nil, input: " World", index: 0, col: 1} = new_context
parse_many(parser, ctx, min, max, count)
replace(parser, replacement_value, opts \\ [])
The replace/3 combinator replaces the AST value of it's child with a constant.
Examples
iex> alias Ergo.Context
iex> alias Ergo
iex> import Ergo.{Combinators, Terminals}
iex> parser = ignore(literal("foo")) |> replace(:foo)
iex> assert %Context{status: {:error, _}} = Ergo.parse(parser, "flush")
satisfy(parser, pred_fn, opts \\ [])
The satisfy/3 parser takes a parser and a predicate function. If the parser is successful the AST is passed to the predicate function. If the predicate function returns true the parser returns the successful context, otherwise an error context is returned.
Example
iex> alias Ergo.Context
iex> import Ergo.{Terminals, Combinators, Numeric}
iex> parser = satisfy(any(), fn char -> char in (?0..?9) end, label: "digit char")
iex> assert %Context{status: :ok, ast: ?4} = Ergo.parse(parser, "4")
iex> assert %Context{status: {:error, [{:unsatisfied, {1, 1}, "Failed to satisfy: digit char"}]}} = Ergo.parse(parser, "!")
iex> parser = satisfy(number(), fn n -> Integer.mod(n, 2) == 0 end, label: "even number")
iex> assert %Context{status: :ok, ast: 42} = Ergo.parse(parser, "42")
iex> assert %Context{status: {:error, [{:unsatisfied, {1, 1}, "Failed to satisfy: even number"}]}} = Ergo.parse(parser, "27")
sequence(parsers, opts \\ [])
The sequence/2
parser takes a list of parsers which are applied in turn.
If any of the parsers fails the sequence itself fails.
The sequence/2
parser treats the commit/0
parser differently. When it
sees commit it increments the contexts commit count. When the parser
is complete, if it has increased the commit count it decrements it before
returning it. See also many
Examples
iex> alias Ergo.Context
iex> import Ergo.{Terminals, Combinators}
iex> parser = sequence([literal("Hello"), ws(), literal("World")])
iex> context = Ergo.parse(parser, "Hello World")
%Context{status: :ok, ast: ["Hello", ?\s, "World"], index: 11, line: 1, col: 12} = context
This test will need to be rewritten in terms of Ergo.diagnose
# iex> Logger.disable(self())
# iex> alias Ergo.Context
# iex> import Ergo.{Terminals, Combinators}
# iex> parser = sequence([literal("Hello"), ws(), literal("World")], label: "HelloWorld")
# iex> context = Ergo.parse(parser, "Hello World")
# iex> assert %Context{status: :ok, ast: ["Hello", ?\s, "World"], index: 11, line: 1, col: 12} = context
iex> alias Ergo.Context
iex> import Ergo.{Terminals, Combinators}
iex> parser = sequence([literal("Hello"), ws(), literal("World")], ast: fn ast -> Enum.join(ast, " ") end)
iex> context = Ergo.parse(parser, "Hello World")
iex> assert %Context{status: :ok, ast: "Hello 32 World", index: 11, line: 1, col: 12} = context
This test will need to be rewritten in terms of Ergo.diagnose
# iex> Logger.disable(self())
# iex> alias Ergo.Context
# iex> import Ergo.{Terminals, Combinators}
# iex> parser = sequence([literal("Hello"), ws(), literal("World")], label: "HelloWorld", ast: fn ast -> Enum.join(ast, " ") end)
# iex> context = Ergo.parse(parser, "Hello World")
# iex> assert %Context{status: :ok, ast: "Hello 32 World", index: 11, line: 1, col: 12} = context
iex> alias Ergo.Context
iex> import Ergo.{Combinators, Terminals}
iex> parser = sequence([literal("foo"), ws(), literal("bar")])
iex> assert %Context{status: {:error, [{:bad_literal, _, _}, {:unexpected_char, _, _}]}} = Ergo.parse(parser, "Hello World")
string(parser)
The string/1 parser takes a parser that returns an AST which is a list of characters and converts the AST into a string.
Examples
iex> alias Ergo
iex> alias Ergo.Context
iex> import Ergo.{Terminals, Combinators}
iex> parser = many(alpha()) |> string()
iex> assert %Context{status: :ok, ast: "FourtyTwo"} = Ergo.parse(parser, "FourtyTwo")
transform(parser, transformer_fn, opts \\ [])
The transform/2
parser runs a transforming function on the AST of its child parser.
Examples
# Sum the digits
iex> alias Ergo.Context
iex> import Ergo.{Combinators, Terminals}
iex> digit_to_int = fn d -> List.to_string([d]) |> String.to_integer() end
iex> t_fn = fn ast -> ast |> Enum.map(digit_to_int) |> Enum.sum() end
iex> parser = sequence([digit(), digit(), digit(), digit()]) |> transform(t_fn)
iex> context = Ergo.parse(parser, "1234")
iex> %Context{status: :ok, ast: 10, index: 4, line: 1, col: 5} = context