mix workspace.run (Workspace v0.2.1)

View Source

Run a mix task on one or more workspace projects.

You need to specify the task to be executed with the --task, -t option:

$ mix workspace.run -t format

This will run mix format on all projects of the workspace. You can also select the projects to run, or exclude specific projects:

# Run format only on foo and bar
$ mix workspace.run -t format -p foo -p bar

# Run format on all projects except foo
$ mix workspace.run -t format -e foo

Passing options to tasks

You can pass task specific options after the return separator (--). For example:

$ mix workspace.run -t format -p foo -p bar -- --check-formatted

Command-line Options

  • --allow-failure... (atom) - Allow the task for this specific project to fail. Can be set more than once. [values can be grouped with the , separator]

  • --dry-run (boolean) - If set it will not execute the command, useful for testing and debugging. [default: false]

  • --early-stop (boolean) - If set the execution will stop if the execution of any project failed [default: false]

  • --env-var... (string) - Optional environment variables to be set before command execution. They are expected to be in the form ENV_VAR_NAME=value. You can use this multiple times for setting multiple variables.

  • --order (string) - The execution order based on the workspace graph. Can be one of the following:

  • alphabetical - The projects are sorted alphabetically

  • postorder - Performs a depth-first search on the project graph and returns the projects in post-order. In this order, outer leaves (projects without dependencies) are returned first, followed by their parent projects, respecting the dependency relationships between them. Allowed values: ["alphabetical", "postorder"]. [default: alphabetical]

  • --partitions (integer) - Sets the number of partitions to split executions in. It must be a number greater than zero. If set to 1 it acts as a no-op. If more than one you must also set the the WORKSPACE_RUN_PARTITION environment variable with the partition to use in the current execution. See the "Run partitioning" section for more details

  • -t, --task (string) - Required. The task to execute

Workspace status options

Status is retrieved from the diff between the given --base and --head. Knowing the changed files we can limit the execution of workspace commands only to relevant projects.

  • -a, --affected (boolean) - Run only on affected projects [default: false]
  • --base (string) - The base git reference to compare the head to. Applied only when --affected or --modified are set.
  • --head (string) - A reference to the git head. Applied only if --base is set for getting the changed files [default: HEAD]
  • -m, --modified (boolean) - Run only on modified projects [default: false]
  • --only-roots (boolean) - If set, the task will be executed only on graph's root nodes. [default: false]

Filtering options

  • --dependency (string) - If set, only projects that have the given dependency will be considered.
  • --dependent (string) - If set, only projects that are dependencies of the given project are considered.
  • -e, --exclude... (string) - Ignore the given projects [values can be grouped with the , separator]
  • --exclude-tag... (string) - If set, any projects with any of the given tag(s) will be excluded. For scoped tags you should provide a colon separated string (examples: shared, scope:api, type:utils). For selecting a specific tag use --tag [values can be grouped with the , separator]
  • --path... (string) - A path under which projects will be considered. Paths should be relative with respect to the workspace root. All other projects can be ignored. Can be set multiple times.
  • -p, --project... (string) - The project name, can be defined multiple times. If not set all projects are considered [values can be grouped with the , separator]
  • --tag... (string) - If set, only projects with the given tag(s) will be considered. For scoped tags you should provide a colon separated string (examples: shared, scope:api, type:utils). For excluding a specific tag use --exclude-tag [values can be grouped with the , separator]

Display options

  • --show-status (boolean) - If set the status of each project will be included in the output graph [default: false]
  • --verbose (boolean) - If set enables verbose logging [default: false]

Export options

  • --export (string) - A json path to export the run results. If set a json file will be generated with the run results for each project. This may be useful for your CI/CD automation pipelines, in case you want to post-process the run results.

Global workspace options

  • --config-path (string) - The path to the workspace config to be used, relative to the workspace path [default: .workspace.exs]
  • --workspace-path (string) - If set it specifies the root workspace path, defaults to current directory

Filtering runs

Monorepos can have hundreds of projects so being able to run a task against all or a subset of them is a key feature of Workspace.

One of the key features of workspace.run is the flexibility regarding the projects on which the task will be executed.

The project graph

The core concept of Workspace is the project graph. This is a directed acyclic graphs depicting the internal project dependencies. This is used by all tasks internally in order to specify the status of the projects.

Assume you have the following workspace:

package_a
├── package_b ✚
│   └── package_g
├── package_c
│   ├── package_e
│   └── package_f
│       └── package_g
└── package_d
package_h ✚
└── package_d

It consists of various packages, and two of them (_b and _h) are modified. Using the graph we can find the dependencies between packages and limit the execution of a task only to the subset that makes sense.

Using the proper execution strategy wisely significantly improves the CI execution time.

Where to run a task?

This clearly depends on the type of the task. Let's see some examples:

  • mix format makes sense to be executed only on the modified packages.
  • mix test should be executed on the modified and the parents of them since a change on a package may affect a dependent one.
  • mix deps.get makes sense to be executed only on the root packages if you have adopted a common deps path, since internal dependencies will be inherited from dependent packages.
  • In the main branch it makes sense to run the full suite on the complete workspace.

In order to visualize the graph of the current workspace check the mix workspace.graph task.

Filtering examples

Let's see some examples:

# Run test on all workspace projects
$ mix workspace.run -t test

# Run test only on foo and bar
$ mix workspace.run -t test -p foo -p bar

# Run test on all projects excluding foo and bar
$ mix workspace.run -t test --exclude foo --exclude bar

# Run test on all affected projects
$ mix workspace.run -t test --affected

# Run test on all modified projects
$ mix workspace.run -t test --modified

# Run test only on top level projects
$ mix workspace.run -t test --only-roots

Execution order

By default the given task will run on all projects (matching any provided filter) alphabetically with respect to the project app name. You can also pass the --order postorder option which applies a topologic sort. This will perform a depth first search on the project graph and return the project in post-order, e.g. outer leaves are returned first respecting your workspace graph.

$ mix workspace.run -t compile --order postorder

The preorder option ensures that dependencies are executed first, which can reduce execution time by avoiding unnecessary task invocations on dependencies (only if the task requires dependencies being executed first, e.g. mix compile with a common build path).

Run partitioning

In big workspaces some CI steps may take a lot time. You can split the execution of a task in multiple partitions in order to speed up this process. This is done by setting the --partitions option and setting the WORKSPACE_RUN_PARTITION environment variable to control the current partition.

For example to split the execution of mix test on all workspace projects into 4 partitions, you would use the following:

$ WORKSPACE_RUN_PARTITION=1 mix workspace.run -t run --partitions 4
$ WORKSPACE_RUN_PARTITION=2 mix workspace.run -t run --partitions 4
$ WORKSPACE_RUN_PARTITION=3 mix workspace.run -t run --partitions 4
$ WORKSPACE_RUN_PARTITION=4 mix workspace.run -t run --partitions 4

The matching projects are sorted upfront and assigned to each partition in a round-robin fashion.

The --env-var option

Some tasks depend on environment variables. You can use the --env-var option to set an environment variable only during a task's execution. This is particularly useful in CI environments.

$ mix workspace.run -t compile --env-var MIX_ENV=compile

You can set multiple environment variables by setting the --env-var multiple times.

Early stopping

You can set the --early-stop flag in order to immediately terminate the command if the execution in any of the projects has failed.

Allowing failures on some projects

On big codebases not all projects are of the same quality standards and some linter tasks may fail on them. In such cases you may want to still run the linting tasks on these projects but ignore their output status from the overall run task's status. In such cases you can use the --allow-failure option:

# Run lint on all projects but ignore the output status of project foo
$ mix workspace.run -t lint --allow-failure foo

Dry running a task

You can set the --dry-run option in order to dry run the task, e.g. to check the sequence of mix tasks that will be invoked.

Exporting run results

You can set the --export option in order to save the run results per workspace project in a json file. This may be useful in case you want to post-process run results in your CI pipelines, for example you could use it to generate a summary of the execution and post it as a comment to the corresponding PR/MR.

# Exporting execution results to test.json
$ mix workspace.run -t test --export test.json