Testing Spark Verifiers

Verifier errors raised inside Spark's @after_verify hook are caught by the framework and emitted as stderr warnings instead of propagated as exceptions. This keeps compilation flowing when a DSL is invalid, but it means the natural ExUnit pattern — assert_raise/2 — does not work.

Spark.Test provides ExUnit helpers that turn the captured errors and warnings back into structured data your tests can pattern-match on.

Quick reference

import Spark.Test makes all six available.

Defining modules inside the helpers

This rule catches everyone. Inside the macro's anonymous-function body, a bare alias like BadPerson resolves against the surrounding test module's scope. Outside — where your assert patterns live — the same alias resolves at the top level. They become different atoms, and your patterns will never match.

Force top-level resolution with the explicit Elixir.X form:

errors =
  dsl_errors do
    defmodule Elixir.BadPerson do  # <-- note the Elixir. prefix
      ...
    end
  end

assert [{BadPerson, _}] = errors    # <-- bare alias resolves to Elixir.BadPerson, matches

Always use the Elixir.X form for any module you intend to assert on.

Testing a verifier that produces errors

Anything that returns {:error, %Spark.Error.DslError{}} from Spark.Dsl.Verifier.verify/1 — your own verifiers and Spark's built-ins (VerifyEntityUniqueness, VerifySectionSingletonEntities).

Match a specific error

assert_dsl_error/2 is the most common shape. It takes a pattern, runs the do-block, and returns the first matching error so you can do follow-up assertions on it:

defmodule MyApp.PersonValidatorTest do
  use ExUnit.Case, async: true

  import Spark.Test

  test "rejects an invalid email" do
    err =
      assert_dsl_error %Spark.Error.DslError{path: [:fields, :email]} do
        defmodule Elixir.BadPerson do
          use MyApp.PersonValidator

          fields do
            field :email, :string do
              check &String.contains?(&1, "@")
            end
          end
        end
      end

    assert err.message =~ "invalid email"
  end
end

The pattern is matched via match?/2, so any valid Elixir pattern works. On no match the test fails with a message that includes the pattern source and the inspected list of actual errors.

When the pattern doesn't matter, use the /1 form:

err =
  assert_dsl_error do
    defmodule Elixir.BadPerson do
      # ...
    end
  end

Assert clean compilation

For happy-path tests where no error should be produced:

test "valid configuration is accepted" do
  refute_dsl_errors do
    defmodule Elixir.GoodPerson do
      use MyApp.PersonValidator

      fields do
        field :email, :string
      end
    end
  end
end

Returns :ok on success. On failure the message lists the offending modules and their errors.

Work with the full collected list

Use dsl_errors/1 directly when you need to compare error counts, assert on multiple distinct errors at once, or otherwise inspect the full result. The shape is [{module, [%Spark.Error.DslError{}]}, ...] in definition order:

errors =
  dsl_errors do
    defmodule Elixir.HasTwoIssues do
      # ...
    end
  end

assert [{HasTwoIssues, errors_list}] = errors
assert length(errors_list) == 2

Testing a verifier that produces warnings

The same three helpers exist (dsl_warnings/1, assert_dsl_warning/2,/1, refute_dsl_warnings/1) with one shape difference:

Warnings are normalized to {message, location | nil} tuples, not Spark.Error.DslError structs. A bare-string warning {:warn, "msg"} becomes {"msg", nil}; a tuple-form warning {:warn, {"msg", anno}} preserves the location. Source annotations require debug_info to be enabled for tests — see Asserting on source annotations below.

test "deprecated option emits a warning" do
  {message, _location} =
    assert_dsl_warning {_, _} do
      defmodule Elixir.UsesDeprecatedOption do
        use MyApp.Validator

        fields do
          field :legacy_option, :string
        end
      end
    end

  assert message =~ "deprecated"
end

For literal-message matching put the string in the pattern:

{_, _} =
  assert_dsl_warning {"legacy_option is deprecated", _} do
    # ...
  end

refute_dsl_warnings/1 and dsl_warnings/1 work identically to their errors counterparts, just with the warning payload shape.

What the helpers don't capture

• Spark.Warning.warn/3 called directly — for example by Spark.Options for schema-level deprecation warnings, or by Spark.Dsl.Extension.__after_verify__/1 for __spark_metadata__ deprecation. These bypass the verifier-callback path. Use capture_io(:stderr, ...) to assert on them.
• Transformer warnings — {:warn, dsl_state, warnings} returns from Spark.Dsl.Transformer.transform/1 flow through a separate code path and are not collected.

Asserting on source annotations

Spark only captures :erl_anno.anno() when debug_info is enabled. To enable it for runtime modules compiled inside the helpers, add the following to your mix.exs:

test_elixirc_options: [debug_info: true]

Without it, location in {message, location} payloads is nil even when the verifier passes the anno correctly.

Async safety

The helpers are safe in async: true ExUnit tests. The collection mechanism uses per-process state (Process.put/2) and point-to-point send/2; nothing crosses process boundaries.

One caveat: any defmodule defined inside a process you spawn yourself won't be captured, because the spawned process has its own dictionary. Either keep defmodule calls in the same process as the helper, or set the collector flag explicitly inside the spawned process:

parent = self()

Task.async(fn ->
  Process.put({Spark.Dsl, :test_collector}, parent)
  defmodule Elixir.SomeModule do
    # ...
  end
end)
|> Task.await()

See also

• Spark.Test — the helper module
• Spark.Dsl.Verifier — the verifier callback contract
• Spark.Error.DslError — the error struct returned by the error helpers