View Source Mneme (Mneme v0.2.2)
/ni:mi:/ - Snapshot testing for Elixir ExUnit
Early days
Mneme is in its infancy and has an intentionally minimal API consisting largely of a single macro. Please feel free to submit any feedback, bugs, or suggestions as issues on Github. Thanks!
Snapshot tests assert that some expression matches a reference value.
It's like an ExUnit assert
, except that the reference value is
managed for you by Mneme.
Mneme follows in the footsteps of existing snapshot testing libraries like Insta (Rust), expect-test (OCaml), and assert_value (Elixir). Instead of simple value or string comparison, however, Mneme leans heavily into pattern matching.
example
Example
Let's say you've written a test for a function that removes even numbers from a list:
test "drop_evens/1 should remove all even numbers from an enum" do
auto_assert drop_evens(1..10)
auto_assert drop_evens([])
auto_assert drop_evens([:a, :b, 2, :c])
end
The first time you run this test, you'll see interactive prompts for
each call to auto_assert
showing a diff and asking if you'd like to
accept the generated pattern. After accepting them, your test is
updated:
test "drop_evens/1 should remove all even numbers from an enum" do
auto_assert [1, 3, 5, 7, 9] <- drop_evens(1..10)
auto_assert [] <- drop_evens([])
auto_assert [:a, :b, :c] <- drop_evens([:a, :b, 2, :c])
end
The next time you run this test, you won't receive a prompt and these will act (almost) like any other assertion. If the result of the call ever changes, you'll be prompted again and can choose to update the test or reject it and let it fail.
With a few exceptions, auto_assert/1
acts very similarly to a normal
assert
. See the macro docs
for a list of differences.
quick-start
Quick start
Add
:mneme
do your deps inmix.exs
:defp deps do [ {:mneme, ">= 0.0.0", only: :test} ] end
Add
:mneme
to your:import_deps
in.formatter.exs
:[ import_deps: [:mneme], inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"] ]
Start Mneme right after you start ExUnit in
test/test_helper.exs
:ExUnit.start() Mneme.start()
Add
use Mneme
wherever youuse ExUnit.Case
:defmodule MyTest do use ExUnit.Case, async: true use Mneme test "arithmetic" do # use auto_assert instead of ExUnit's assert - run this test # and delight in all the typing you don't have to do auto_assert 2 + 2 end end
match-patterns
Match patterns
Mneme tries to generate match patterns that are equivalent to what a human (or at least a nice LLM) would write. Basic data types like strings, numbers, lists, tuples, etc. will be as you would expect.
Some values, however, do not have a literal representation that can be used in a pattern match. Pids are such an example. For those, guards are used:
auto_assert self()
# after running the test and accepting the change
auto_assert pid when is_pid(pid) <- self()
Additionally, local variables can be found and pinned as a part of the pattern. This keeps the number of hard-coded values down, reducing the likelihood that tests have to be updated in the future.
test "create_post/1 creates a new post with valid attrs", %{user: user} do
valid_attrs = %{title: "my_post", author: user}
auto_assert create_post(valid_attrs)
end
# after running the test
test "create_post/1 creates a new post with valid attrs", %{user: user} do
valid_attrs = %{title: "my_post", author: user}
auto_assert {:ok, %Post{title: "my_post", author: ^user}} <- create_post(valid_attrs)
end
In many cases, multiple valid patterns will be possible. Usually, the "simplest" pattern will be selected by default when you are prompted, but you can cycle through the options as well.
non-exhaustive-list-of-special-cases
Non-exhaustive list of special cases
Pinned variables are generated by default if a value is equal to a variable in scope.
Date and time values are written using their sigil representation.
Struct patterns only include fields that are different from the struct defaults.
Structs defined by Ecto schemas exclude primary keys, association foreign keys, and auto generated fields like
:inserted_at
and:updated_at
. This is because these fields are often randomly generated and would fail on subsequent tests.
formatting
Formatting
Mneme uses Rewrite
to update
source code, formatting that code before saving the file. Currently,
the Elixir formatter and FreedomFormatter
are supported. If you do
not use a formatter, the first auto-assertion will reformat the entire
file.
continuous-integration
Continuous Integration
In a CI environment, Mneme will not attempt to prompt and update any
assertions, but will instead fail any tests that would update. This
behavior is enabled by the CI
environment variable, which is set by
convention by many continuous integration providers.
export CI=true
editor-support
Editor support
Guides for optional editor integration can be found here:
acknowledgements
Acknowledgements
Special thanks to:
What if writing tests was a joyful experience?, from the Jane Street Tech Blog, for inspiring this library.
Sourceror, a library that makes complex code modifications simple.
Rewrite, which provides the diff functionality present in Mneme.
Owl, which makes it much easier to build a pretty CLI.
Insta, a snapshot testing tool for Rust, whose great documentation provided an excellent reference for snapshot testing.
assert_value, an existing Elixir project that provides similar functionality. Thank you for paving the way!
configuration
Configuration
Certain behavior can be configured globally using application config or locally in test modules either at the module, describe-block, or test level.
To configure Mneme globally, you can set :defaults
for the :mneme
application:
config :mneme,
defaults: [
diff: :semantic
]
These defaults can be overriden in test modules at various levels
either as options to use Mneme
or as module attributes.
defmodule MyTest do
use ExUnit.Case
# reject all changes to auto-assertions by default
use Mneme, action: :reject
test "this test will fail" do
auto_assert 1 + 1
end
describe "some describe block" do
# accept all changes to auto-assertions in this describe block
@mneme_describe action: :accept
test "this will update without prompting" do
auto_assert 2 + 2
end
# prompt for any changes in this test
@mneme action: :prompt
test "this will prompt before updating" do
auto_assert 3 + 3
end
end
end
Configuration that is "closer to the test" will override more general configuration:
@mneme > @mneme_describe > opts to use Mneme > :mneme app config
The exception to this is the CI
environment variable, which causes
all updates to be rejected. See the "Continuous Integration" section
for more info.
options
Options
:action
- The action to be taken when an auto-assertion updates. Actions are one of:prompt
,:accept
, or:reject
. IfCI=true
is set in environment variables, the action will always be:reject
. The default value is:prompt
.:default_pattern
- The default pattern to be selected if prompted to update an assertion. Can be one of:infer
,:first
, or:last
. The default value is:infer
.:diff
- Controls the diff engine used to display changes when an auto-assertion updates. If:semantic
, uses a custom diff engine to highlight only meaningful changes in the value. If:text
, uses the Myers Difference algorithm to highlight all changes in text. The default value is:semantic
.:target
- The target output for auto-assertions. If:mneme
, the expression will remain an auto-assertion. If:ex_unit
, the expression will be rewritten as an ExUnit assertion. The default value is:mneme
.
Link to this section Summary
Functions
Sets up Mneme configuration for this module and imports auto_assert/1
.
Generate or run an assertion.
Starts Mneme to run auto-assertions as they appear in your tests.
Link to this section Functions
Sets up Mneme configuration for this module and imports auto_assert/1
.
This macro accepts all options described in the "Configuration" section above.
example
Example
defmodule MyTest do
use ExUnit.Case
use Mneme # <- add this
test "..." do
auto_assert ...
end
end
Generate or run an assertion.
auto_assert
generates assertions when tests run, issuing a terminal
prompt before making any changes (unless configured otherwise).
auto_assert [1, 2] ++ [3, 4]
# after running the test and accepting the change
auto_assert [1, 2, 3, 4] <- [1, 2] ++ [3, 4]
If the match no longer succeeds, a warning and new prompt will be issued to update it to the new value.
auto_assert [1, 2, 3, 4] <- [1, 2] ++ [:a, :b]
# after running the test and accepting the change
auto_assert [1, 2, :a, :b] <- [1, 2] ++ [:a, :b]
Prompts are only issued if the pattern doesn't match the value, so that pattern can also be changed manually.
# this assertion succeeds, so no prompt is issued
auto_assert [1, 2, | _] <- [1, 2] ++ [:a, :b]
differences-from-exunit-assert
Differences from ExUnit assert
The auto_assert
macro is meant to match assert
as closely as
possible. In fact, it generates ExUnit assertions under the hood.
There are, however, a few small differences to note:
Pattern-matching assertions use the
<-
operator instead of the=
match operator. Value-comparison assertions still use==
(for instance, when the expression returnsnil
orfalse
).Guards can be added with a
when
clause, whileassert
would require a second assertion. For example:auto_assert pid when is_pid(pid) <- self() assert pid = self() assert is_pid(pid)
Bindings in an
auto_assert
are not available outside of that assertion. For example:auto_assert pid when is_pid(pid) <- self() pid # ERROR: pid is not bound
If you need to use the result of the assertion, it will evaluate to the expression's value.
pid = auto_assert pid when is_pid(pid) <- self() pid # pid is the result of self()
Starts Mneme to run auto-assertions as they appear in your tests.
This will almost always be added to your test/test_helper.exs
, just
below the call to ExUnit.start()
:
# test/test_helper.exs
ExUnit.start()
Mneme.start()
options
Options
:restart
(boolean) - Restarts Mneme if it has previously been started. This option enables certain IEx-based testing workflows that allow tests to be run without a startup penalty. Defaults tofalse
.