Rambo v0.3.4

Rambo (Rambo v0.3.4) View Source

Rambo is the easiest way to run external programs.

  • Run programs that require EOF to produce output
  • No more zombies, even if the VM crashes! No scripts required!
  • No additional installs or compilers (Linux, macOS & Windows only)
  • Stream logs back to your app
  • Chain commands together
  • Kill stalled commands
  • Set timeout or run indefinitely
  • Powered by asynchronous I/O, incredibly efficient!

Usage 

Rambo.run("echo")
{:ok, %Rambo{status: 0, out: "\n", err: ""}}

# send standard input
Rambo.run("cat", in: "rambo")

# pass arguments
Rambo.run("ls", ["-l", "-a"])

# chain commands
Rambo.run("ls") |> Rambo.run("sort") |> Rambo.run("head")

# set timeout
Rambo.run("find", "peace", timeout: 1981)

Logging

Logs to standard error are printed by default, so errors are visible before your command finishes. Change this with the :log option.

Rambo.run("ls", log: :stderr) # default
Rambo.run("ls", log: :stdout) # log stdout only
Rambo.run("ls", log: true)    # log both stdout and stderr
Rambo.run("ls", log: false)   # don’t log output

# or to any function
Rambo.run("echo", log: &IO.inspect/1)

Kill

Kill your command from another process, Rambo returns with any gathered results so far.

task = Task.async(fn ->
  Rambo.run("cat")
end)

Rambo.kill(task.pid)

Task.await(task)
{:killed, %Rambo{status: nil, out: "", err: ""}}

Why?

Erlang ports do not work with programs that expect EOF to produce output. The only way to close standard input is to close the port, which also closes standard output, preventing results from coming back to your app. This gotcha is marked Won’t Fix.

Design

When Rambo is asked to run a command, it starts a shim that spawns your command as a child. After writing to standard input, the file descriptor is closed while output is streamed back to your app.

+-----------------+       stdin
|          +------+------+ --> +---------+
|  Erlang  | Port | Shim |     | Command |
|          +------+------+ <-- +---------+
+-----------------+       stdout

If your app exits prematurely, the child is automatically killed to prevent orphans.

Caveats

You cannot call Rambo.run from a GenServer because Rambo uses receive, which interferes with GenServer’s receive loop. However, you can wrap the call in a Task.

task = Task.async(fn ->
  Rambo.run("mission")
end)

Task.await(task)

Comparisons

Rambo does not spawn any processes nor support bidirectional communication with your commands. It is intentionally kept simple to run transient jobs with minimal overhead, such as calling a Python or Node script to transform some data. For more complicated use cases, see below.

System.cmd

If you don’t need to pipe standard input or capture standard error, just use System.cmd.

Porcelain

Porcelain cannot send EOF to trigger output by default. The Goon driver must be installed separately to add this capability. Rambo ships with the required native binaries.

Goon is written in Go, a multithreaded runtime with a garbage collector. To be as lightweight as possible, Rambo’s shim is written in Rust using non-blocking, asynchronous I/O only. No garbage collection runtime, no latency spikes.

Most importantly, Porcelain currently leaks processes. Writing a new driver to replace Goon should fix it, but Porcelain appears to be abandoned so effort went into creating Rambo.

MuonTrap

MuonTrap is designed to run long-running external programs. You can attach the OS process to your supervision tree, and restart it if it crashes. Likewise if your Elixir process crashes, the OS process is terminated too.

You can also limit CPU and memory usage on Linux through cgroups.

erlexec

erlexec is great if you want fine grain control over external programs.

Each external OS process is mirrored as an Erlang process, so you get asynchronous and bidirectional communication. You can kill your OS processes with any signal or monitor them for termination, among many powerful features.

Choose erlexec if you want a kitchen sink solution.

ExCmd

ExCmd can stream data with backpressure, wrapped in a convenient Stream API. Requires separate install of Odu. By the same author as Exile.

Exile

Exile is also focused on streaming like ExCmd but implemented with NIFs so it does not require shims.

Installation

Add rambo to your list of dependencies in mix.exs:

def deps do
  [
    {:rambo, "~> 0.3"}
  ]
end

Linux, macOS and Windows binaries are bundled (x86-64 architecture only). For other environments, install the Rust compiler or Rambo won’t compile.

To remove unused binaries, set :purge to true in your configuration.

config :rambo,
  purge: true

License

Rambo is released under MIT license.

Link to this section Summary

Types

args()
result()
t()

Functions

kill(pid)

Stop by killing your command.

run(command)

Runs command.

run(command, args_or_opts)

Runs command with arguments or options.

run(command, args, opts)

Runs command with arguments and options.

Link to this section Types

Link to this type

args()

View Source

Specs

args() :: String.t() | [iodata()] | nil
Link to this type

result()

View Source

Specs

result() :: {:ok, t()} | {:error, t() | String.t()}
Link to this type

t()

View Source

Specs

t() :: %Rambo{err: String.t(), out: String.t(), status: integer()}

Link to this section Functions

Link to this function

kill(pid)

View Source

Specs

kill(pid()) :: {:killed, t()}

Stop by killing your command.

Pass the pid of the process that called run/1. That process will return with {:killed, %Rambo{}} with results accumulated thus far.

Example 

iex> task = Task.async(fn ->
...>   Rambo.run("cat")
...> end)
iex> Rambo.kill(task.pid)
iex> Task.await(task)
{:killed, %Rambo{status: nil}}
Link to this function

run(command)

View Source

Specs

run(command :: String.t() | result()) :: result()

Runs command.

Executes the command and returns {:ok, %Rambo{}} or {:error, reason}. reason is a string if the child process failed to start, or a %Rambo{} struct if the child process started successfully but exited with a non-zero status.

Multiple calls can be chained together with the |> pipe operator to simulate Unix pipes.

Rambo.run("ls") |> Rambo.run("sort") |> Rambo.run("head")

If any command did not exit with 0, the rest will not be executed and the last executed result is returned in an :error tuple.

See run/2 or run/3 to pass arguments or options.

Examples 

iex> Rambo.run("echo")
{:ok, %Rambo{out: "\n", status: 0, err: ""}}
Link to this function

run(command, args_or_opts)

View Source

Specs

run(command :: String.t() | result(), args_or_opts :: args() | Keyword.t()) ::
  result()

Runs command with arguments or options.

Arguments can be a string or list of strings. See run/3 for options.

Examples 

iex> Rambo.run("echo", "john")
{:ok, %Rambo{out: "john\n", status: 0}}

iex> Rambo.run("echo", ["-n", "john"])
{:ok, %Rambo{out: "john", status: 0}}

iex> Rambo.run("cat", in: "john")
{:ok, %Rambo{out: "john", status: 0}}
Link to this function

run(command, args, opts)

View Source

Specs

run(command :: String.t() | result(), args :: args(), opts :: Keyword.t()) ::
  result()

Runs command with arguments and options.

Options

  • :in - pipe iodata as standard input
  • :cd - the directory to run the command in
  • :env - map or list of tuples containing environment key-value as strings
  • :log - stream standard output or standard error to console or a function. May be :stdout, :stderr, true for both, false for neither, or a function with one arity. If a function is given, it will be passed {:stdout, output} or {:stderr, error} tuples. Defaults to :stderr.
  • :timeout - kills command after timeout in milliseconds. Defaults to no timeout.

Examples 

iex> Rambo.run("/bin/sh", ["-c", "echo $JOHN"], env: %{"JOHN" => "rambo"})
{:ok, %Rambo{out: "rambo\n", status: 0}}

iex> Rambo.run("echo", "rambo", log: &IO.inspect/1)
{:ok, %Rambo{out: "rambo\n", status: 0}}