GenServer which wraps spawned external command.

Use ExCmd.stream!/1 over using this. Use this only if you are familiar with life-cycle and need more control of the IO streams and OS process.

Comparison with Port

• it is demand driven. User explicitly has to read the command output, and the progress of the external command is controlled using OS pipes. ExCmd never load more output than we can consume, so we should never experience memory issues

• it can close stdin while consuming output

• tries to handle zombie process by attempting to cleanup external process. Note that there is no middleware involved with ex_cmd so it is still possible to endup with zombie process.

• selectively consume stdout and stderr

Internally ExCmd uses non-blocking asynchronous system calls to interact with the external process. It does not use port's message based communication, instead uses raw stdio and NIF. Uses asynchronous system calls for IO. Most of the system calls are non-blocking, so it should not block the beam schedulers. Make use of dirty-schedulers for IO

Introduction

ExCmd.Process is a process based wrapper around the external process. It is similar to port as an entity but the interface is different. All communication with the external process must happen via ExCmd.Process interface.

ExCmd process life-cycle tied to external process and owners. All system resources such are open file-descriptors, external process are cleaned up when the ExCmd.Process dies.

Owner

Each ExCmd.Process has an owner. And it will be the process which created it (via ExCmd.Process.start_link/2). Process owner can not be changed.

Owner process will be linked to the ExCmd.Process. So when the ex_cmd process is dies abnormally the owner will be killed too or visa-versa. Owner process should avoid trapping the exit signal, if you want avoid the caller getting killed, create a separate process as owner to run the command and monitor that process.

Only owner can get the exit status of the command, using ExCmd.Process.await_exit/2. All ex_cmd processes MUST be awaited. Exit status or reason is ALWAYS sent to the owner. It is similar to Task. If the owner exit without await_exit, the ex_cmd process will be killed, but if the owner continue without await_exit then the ex_cmd process will linger around till the process exit.

iex> alias ExCmd.Process
iex> {:ok, p} = Process.start_link(~w(echo hello))
iex> Process.read(p, 100)
{:ok, "hello\n"}
iex> Process.read(p, 100) # read till we get :eof
:eof
iex> Process.await_exit(p)
{:ok, 0}

Pipe & Pipe Owner

Standard IO pipes/channels/streams of the external process such as STDIN, STDOUT, STDERR are called as Pipes. User can either write or read data from pipes.

Each pipe has an owner process and only that process can write or read from the ex_cmd process. By default the process who created the ex_cmd process is the owner of all the pipes. Pipe owner can be changed using ExCmd.Process.change_pipe_owner/3.

Pipe owner is monitored and the pipes are closed automatically when the pipe owner exit. Pipe Owner can close the pipe early using ExCmd.Process.close_stdin/1 etc.

ExCmd.Process.await_exit/2 closes all of the caller owned pipes by default.

iex> {:ok, p} = Process.start_link(~w(cat))
iex> writer = Task.async(fn ->
...>   :ok = Process.change_pipe_owner(p, :stdin, self())
...>   Process.write(p, "Hello World")
...> end)
iex> Task.await(writer)
:ok
iex> Process.read(p, 100)
{:ok, "Hello World"}
iex> Process.await_exit(p)
{:ok, 0}

Pipe Operations

Only Pipe owner can read or write date to the owned pipe. All Pipe operations (read/write) blocks the caller as a mechanism to put back-pressure, and this also makes the API simpler. This is same as how command-line programs works on the shell, along with pipes in-between, Example: cat larg-file | grep "foo". Internally ExCmd uses asynchronous IO APIs to avoid blocking VM (by default NIF calls blocks the VM scheduler), so you can open several pipes and do concurrent IO operations without blocking VM.

stderr

by default is :stderr is connected to console, data written to stderr will appear on the console.

You can change the behavior by setting :stderr:

1. :console - stderr output is redirected to console
2. :redirect_to_stdout - stderr output is redirected to stdout
3. :disable - stderr output is redirected /dev/null suppressing all output. See below for more details. (Default)

Using redirect_to_stdout

stderr data will be redirected to stdout. When you read stdout you will see both stdout & stderr combined and you won't be able differentiate stdout and stderr separately. This is similar to :stderr_to_stdout option present in Ports.

Process Termination

When owner does (normally or abnormally) the ExCmd process always terminated irrespective of pipe status or process status. External process get a chance to terminate gracefully, if that fail it will be killed.

If owner calls await_exit then the owner owned pipes are closed and we wait for external process to terminate, if the process already terminated then call returns immediately with exit status. Else command will be attempted to stop gracefully following the exit sequence based on the timeout value (5s by default).

If owner calls await_exit with timeout as :infinity then ExCmd does not attempt to forcefully stop the external command and wait for command to exit on itself. The await_exit call can be blocked indefinitely waiting for external process to terminate.

If external process exit on its own, exit status is collected and ExCmd process will wait for owner to close pipes. Most commands exit with pipes are closed, so just ensuring to close pipes when works is done should be enough.

Example of process getting terminated by SIGTERM signal

# sleep command does not watch for stdin or stdout, so closing the
# pipe does not terminate the sleep command.
iex> {:ok, p} = Process.start_link(~w(sleep 100000000)) # sleep indefinitely
iex> Process.await_exit(p, 2000) # ensure `await_exit` finish within `2000ms`. By default it waits for 5s
{:error, :killed} # command exit due to SIGTERM

Examples

Run a command without any input or output

iex> {:ok, p} = Process.start_link(["sh", "-c", "exit 2"])
iex> Process.await_exit(p)
{:ok, 2}

Single process reading and writing to the command

# bc is a calculator, which reads from stdin and writes output to stdout
iex> {:ok, p} = Process.start_link(~w(cat))
iex> Process.write(p, "hello\n") # there must be new-line to indicate the end of the input line
:ok
iex> Process.read(p)
{:ok, "hello\n"}
iex> Process.write(p, "world\n")
:ok
iex> Process.read(p)
{:ok, "world\n"}
# We must close stdin to signal the command that we are done.
# since `await_exit` implicitly closes the pipes, in this case we don't have to
iex> Process.await_exit(p)
{:ok, 0}

Running a command which flush the output on stdin close. This is not supported by Erlang/Elixir ports.

# `base64` command reads all input and writes encoded output when stdin is closed.
iex> {:ok, p} = Process.start_link(~w(base64))
iex> Process.write(p, "abcdef")
:ok
iex> Process.close_stdin(p) # we can selectively close stdin and read all output
:ok
iex> Process.read(p)
{:ok, "YWJjZGVm\n"}
iex> Process.read(p) # typically it is better to read till we receive :eof when we are not sure how big the output data size is
:eof
iex> Process.await_exit(p)
{:ok, 0}

Read and write to pipes in separate processes

iex> {:ok, p} = Process.start_link(~w(cat))
iex> writer = Task.async(fn ->
...>   :ok = Process.change_pipe_owner(p, :stdin, self())
...>   Process.write(p, "Hello World")
...>   # no need to close the pipe explicitly here. Pipe will be closed automatically when process exit
...> end)
iex> reader = Task.async(fn ->
...>   :ok = Process.change_pipe_owner(p, :stdout, self())
...>   Process.read(p)
...> end)
iex> :timer.sleep(500) # wait for the reader and writer to change pipe owner, otherwise `await_exit` will close the pipes before we change pipe owner
iex> Process.await_exit(p, :infinity) # let the reader and writer take indefinite time to finish
{:ok, 0}
iex> Task.await(writer)
:ok
iex> Task.await(reader)
{:ok, "Hello World"}