Pipette (Pipette v0.5.1)

Copy Markdown View Source

Declarative Buildkite pipeline generation for monorepos, written in Elixir.

Define your CI pipeline using Pipette.DSL — a declarative syntax built on Spark. Pipette inspects changed files, applies branch policies and scope rules, then generates a Buildkite YAML pipeline containing only the groups that need to run.

Quick start

defmodule MyApp.Pipeline do
  use Pipette.DSL

  branch "main", scopes: :all, disable: [:targeting]

  scope :api_code, files: ["apps/api/**"]
  scope :web_code, files: ["apps/web/**"]

  ignore ["docs/**", "*.md"]

  group :api do
    label ":elixir: API"
    scope :api_code
    step :test, label: "Test", command: "mix test"
    step :lint, label: "Lint", command: "mix credo"
  end

  group :web do
    label ":globe_with_meridians: Web"
    scope :web_code
    step :test, label: "Test", command: "pnpm test"
    step :build, label: "Build", command: "pnpm build"
  end

  group :deploy do
    label ":rocket: Deploy"
    depends_on [:api, :web]
    only "main"
    step :push, label: "Push", command: "./deploy.sh"
  end

  trigger :notify do
    pipeline "notify-pipeline"
    depends_on :deploy
    only "main"
  end

  force_activate %{"FORCE_DEPLOY" => [:deploy]}
end

Running the pipeline

# In your .buildkite/pipeline.exs:
Pipette.run(MyApp.Pipeline)

# Dry run (returns YAML without uploading):
{:ok, yaml} = Pipette.generate(MyApp.Pipeline)

How it works

  1. Reads the compiled %Pipette.Pipeline{} from the DSL module (validation and key generation happen at compile time via Spark)
  2. Builds a %Pipette.Context{} from Buildkite environment variables
  3. Determines changed files via git diff
  4. Resolves force-activated groups from environment variables
  5. Runs the activation engine to determine which groups to include
  6. Resolves trigger steps based on active groups and branch filters
  7. Serializes active groups and triggers to Buildkite YAML
  8. Uploads the YAML via buildkite-agent pipeline upload (or returns it in dry-run mode)

Options

Both run/2 and generate/2 accept these options:

  • :env — environment variable map (defaults to System.get_env())
  • :dry_run — when true, returns YAML instead of uploading (defaults to DRY_RUN=1)
  • :changed_files — explicit list of changed files (skips git diff)
  • :extra_groups — 2-arity function (ctx, changed_files) -> [Group.t()] for dynamic groups

Testing

Use generate/2 with explicit :env and :changed_files to test activation logic:

{:ok, yaml} = Pipette.generate(MyApp.Pipeline,
  env: %{
    "BUILDKITE_BRANCH" => "feature/login",
    "BUILDKITE_PIPELINE_DEFAULT_BRANCH" => "main",
    "BUILDKITE_COMMIT" => "abc123",
    "BUILDKITE_MESSAGE" => "Add login"
  },
  changed_files: ["apps/api/lib/user.ex"]
)

assert yaml =~ "api"
refute yaml =~ "web"

Summary

Functions

generate(pipeline_module, opts \\ [])

Generate pipeline YAML without uploading. Convenience wrapper around run/2 with dry_run: true.

run(pipeline_module, opts \\ [])

Run the pipeline: validate, resolve activation, and upload to Buildkite.

Functions

generate(pipeline_module, opts \\ [])

@spec generate(
  module(),
  keyword()
) :: {:ok, String.t()} | :noop

Generate pipeline YAML without uploading. Convenience wrapper around run/2 with dry_run: true.

Returns {:ok, yaml} when groups are activated, or :noop when no groups match.

Examples

{:ok, yaml} = Pipette.generate(MyApp.Pipeline,
  env: %{"BUILDKITE_BRANCH" => "main", ...},
  changed_files: ["apps/api/lib/user.ex"]
)

:noop = Pipette.generate(MyApp.Pipeline,
  env: %{"BUILDKITE_BRANCH" => "feature/docs", ...},
  changed_files: ["README.md"]
)

run(pipeline_module, opts \\ [])

@spec run(
  module(),
  keyword()
) :: {:ok, String.t()} | :ok | :noop | {:error, String.t()}

Run the pipeline: validate, resolve activation, and upload to Buildkite.

Returns:

  • :ok — pipeline uploaded successfully
  • {:ok, yaml} — dry run mode, returns the YAML string
  • :noop — no groups activated (e.g. docs-only changes)
  • {:error, message} — upload failed

Options

  • :env — environment variable map (defaults to System.get_env())
  • :dry_run — return YAML instead of uploading (defaults to DRY_RUN=1 env var)
  • :changed_files — explicit list of changed files (skips git diff)
  • :extra_groups — fn ctx, changed_files -> [Group.t()] for dynamic groups

Examples

# Normal CI usage (reads env, runs git diff, uploads):
Pipette.run(MyApp.Pipeline)

# Dry run from command line:
# DRY_RUN=1 elixir .buildkite/pipeline.exs

# Programmatic dry run with explicit inputs:
{:ok, yaml} = Pipette.run(MyApp.Pipeline,
  dry_run: true,
  env: %{"BUILDKITE_BRANCH" => "main", ...},
  changed_files: ["apps/api/lib/user.ex"]
)