librarian v0.2.0 SSH View Source
SSH streams and SSH and basic SCP functionality
The librarian SSH module provides SSH streams (see stream/3
) and
three protocols over the SSH stream:
run/3
, which runs a command on the remote SSH host.fetch/3
, which uses the SCP protocol to obtain a remote file,send/4
, which uses the SCP protocol to send a file to the remote host.
Note that not all SSH hosts (for example, embedded shells), implement an SCP command, so you may not necessarily be able to perform SCP over your SSH stream.
Using SSH
The principles of this library are simple. You will first want to create
an SSH connection using the connect/2
function. There you will provide
credentials (or let the system figure out the default credentials). The
returned conn
term can then be passed to the multiple utilities.
{:ok, conn} = SSH.connect("some.other.server")
SSH.run!(conn, "echo hello ssh") # ==> "hello ssh"
Using SCP
This library also provides send!/4
and fetch!/3
functions which let you
perform SCP operations via the SSH streams.
{:ok, conn} = SSH.connect("some.other.server")
SSH.send!(conn, binary_or_filestream, "path/to/destination.file")
{:ok, conn} = SSH.connect("some.other.server")
SSH.fetch!(conn, "path/to/source.file")
|> Enum.map(&do_something_with_chunks/1)
Important
If you are performing a streaming SCP send, you may only pass a filestream
or a stream-of-a-filestream into the send!/3
function. If you are
streaming your filestream through other stream operators make sure that the
total file size remains unchanged.
Bang vs non-bang functions
As a general rule, if you expect to run a single or series of tasks with transient (or no) supervision, for example in a worker task or elixir script you should use the bang function and let the task fail, designing your supervision accordingly. This will also potentially let you be lazy about system resources such as SSH connections.
If you expect your SSH task to run as a part of a long-running process (for example, checking in on a host and retrieving data), you should use the error tuple forms and also be careful about closing your ssh connections after use. Check the connection labels documentation for a strategy to organize your code around this neatly.
Mocking
There's a good chance you'll want to mock your SSH commands and responses.
The SSH.Api
behaviour module is provided for that purpose.
Logging
The SSH and related modules interface with Elixir (and Erlang's) logging
facility. The default metadata tagged on the message is ssh: true
; if
you would like to set it otherwise you can set the :librarian, :ssh_metadata
application environment variable.
Customization
If you would like to write your own SSH stream handlers that plug in to
the SSH stream and provide either rudimentary interactivity or early stream
token processing, you may want to consider implementing a module following
the SSH.ModuleApi
behaviour, and initiating your stream as desired.
Limitations
This library has largely been tested against Linux SSH clients. Not all SSH schemes are amenable to stream processing. In those cases you should implement an ssh client gen_server using erlang's ssh_client, though support for this in elixir is planned in the near-term.
Link to this section Summary
Types
channel reference for the SSH and SCP operations
connection reference for the SSH and SCP operations
erlang ip4 format, {byte, byte, byte, byte}
connect to a remote is specified using either a domain name or an ip address
unix-style return codes for ssh-executed functions
Functions
closes the ssh connection.
initiates an ssh connection with a remote server.
like connect/2
but raises with a ConnectionError instead of emitting an error tuple.
retrieves a binary file from the remote host.
like fetch/3
except raises instead of emitting an error tuple.
runs a command on the remote host. Typically returns {:ok, result, retval}
where
retval
is the unix return value from the range 0..255
.
like run/3
except raises on errors instead of returning an error tuple.
sends binary content to the remote host.
like send/4
, except raises on errors, instead of returning an error tuple.
creates an SSH stream struct as an ok tuple or error tuple.
like stream/2
, except raises on an error instead of an error tuple.
Link to this section Types
Specs
chan() :: :ssh.channel_id()
channel reference for the SSH and SCP operations
Specs
conn() :: :ssh.connection_ref()
connection reference for the SSH and SCP operations
Specs
Specs
Specs
Specs
ip4() :: :inet.ip4_address()
erlang ip4 format, {byte, byte, byte, byte}
Specs
connect to a remote is specified using either a domain name or an ip address
Specs
retval() :: 0..255
unix-style return codes for ssh-executed functions
Specs
Specs
run_result() :: {:ok, run_content(), retval()} | {:error, term()}
Specs
send_result() :: :ok | {:error, term()}
Link to this section Functions
Specs
closes the ssh connection.
Typically you will pass the connection reference to this function. If your connection is contained to its own transient task process, you may not need to call this function as the ssh client library will detect that the process has ended and clean up after you.
In some cases, you may want to be able to close a connection out-of-band. In this case, you may label your connection and use the label to perform the close operation. See labels
Specs
connect(remote(), keyword()) :: connect_result()
initiates an ssh connection with a remote server.
options:
:use_ssh_config
seeSSH.Config
, defaults tofalse
.:global_config_path
seeSSH.Config
.:user_config_path
seeSSH.Config
.:user
username to log in as.:port
port to use to ssh, defaults to 22.:label
see labels:link
iftrue
, links the connection with the calling process. Note the calling process will not die if the SSH connection is closed usingclose/1
.
and other SSH options. Some conversions between ssh options and SSH.connect options:
ssh commandline option | SSH library option |
---|---|
-o StrictHostKeyChecking=no | silently_accept_hosts: true |
-q | quiet_mode: true |
-o ConnectTimeout=time | connect_timeout: time_in_ms |
-i pemfile | identity: file |
also consult documentation on client options in the erlang docs
labels:
You can label your ssh connection to provide a side-channel for
correctly closing the connection pid. This is most useful in
the context of with/1
blocks. As an example, the following
code works:
def run_ssh_tasks do
with {:ok, conn} <- SSH.connect("some_host", label: :this_task),
{:ok, _result1, 0} <- SSH.run(conn, "some_command"),
{:ok, result2, 0} <- SSH.run(conn, "some other command") do
{:ok, result1}
end
after
SSH.close(:this_task)
end
Some important points:
- If you are wrangling multiple SSH sessions, please use unique connection labels.
- The ssh connection label is stored in the process dictionary, so the label will not be valid across process boundaries.
- If the ssh connection failed in the first place, the tagged close will return an error tuple. In the example, this will be silent.
Specs
like connect/2
but raises with a ConnectionError instead of emitting an error tuple.
Specs
fetch(conn(), Path.t(), keyword()) :: fetch_result()
retrieves a binary file from the remote host.
Under the hood, this uses the scp protocol to transfer files.
The SCP protocol is as follows:
- execute
scp
remotely in the undocumented-f <source>
mode - send a single zero byte to initiate the conversation
- wait for a control string
"C0<perms> <size> <filename>"
- send a single zero byte
- wait for the binary data + terminating zero
- send a single zero byte
The perms term should be in octal, and the filename should be rootless.
Example:
SSH.fetch(conn, "path/to/desired/file")
Specs
like fetch/3
except raises instead of emitting an error tuple.
Specs
run(conn(), String.t(), keyword()) :: run_result()
runs a command on the remote host. Typically returns {:ok, result, retval}
where
retval
is the unix return value from the range 0..255
.
the result
value is governed by the passed options, but defaults to a string. of
the run value.
Options
{iostream, redirect}
:iostream
may be either:stdout
or:stderr
.redirect
may be one of the following::stream
sends the data to the stream.:stdout
sends the data to thegroup_leader
stdout.:stderr
sends the data to the standard error io stream.:silent
deletes all of the data.:raw
sends the data to the stream tagged with source information as either{:stdout, data}
or{:stderr, data}
, as appropriate.{:file, path}
sends the data to a new or existing file at the provided path.fun/1
processes the data via the function, with the output flat-mapped into the stream. this means that the results offun/1
should be lists, with an empty list sending nothing into the stream.fun/2
is likefun/1
except the stream struct is passed as the second parameter. The output offun/2
should take the shape{flat_map_results, modified_stream}
. You may use the:data
field of the stream struct to store arbitrary data; and a value ofnil
indicates that it has been unused.
{:tty, true | <options>}
: register the connection as a tty connection. Note this changes the default behavior to send the output to group leader stdout instead of to the result, but this is overridable with the iostream redirect above. For options, see:ssh_client_connection.ptty_alloc/4
{:env, <env list>}
: a list of environment variables to be passed. NB: typically environment variables are filtered by the host environment.{:dir, path}
: changes directory topath
and then runs the command{:as, :binary}
(default): outputs result as a binary{:as, :iolist}
: outputs result as an iolist{:as, :tuple}
: result takes the shape of the tuple{stdout_binary, stderr_binary}
note that this mode will override any other redirection selected.
Example:
SSH.run(conn, "hostname") # ==> {:ok, "hostname_of_remote\n", 0}
SSH.run(conn, "some_program", stderr: :silent) # ==> similar to running "some_program 2>/dev/null"
SSH.run(conn, "some_program", stderr: :stream) # ==> similar to running "some_program 2>&1"
SSH.run(conn, "some_program", stdout: :silent, stderr: :stream) # ==> only capture standard error
Specs
run!(conn(), String.t(), keyword()) :: run_content() | no_return()
like run/3
except raises on errors instead of returning an error tuple.
Note that by default this raises in the case that the SSH connection fails AND in the case that the remote command returns non-zero.
Specs
send(conn(), iodata() | filestreams(), Path.t(), keyword()) :: send_result()
sends binary content to the remote host.
Under the hood, this uses the scp protocol to transfer files.
Protocol is as follows:
- execute
scp
remotely in the undocumented-t <destination>
mode - send a control string
"C0<perms> <size> <filename>"
- wait for single zero byte
- send the binary data + terminating zero
- wait for single zero byte
- send
EOF
The perms term should be in octal, and the filename should be rootless.
options:
:permissions
- sets unix-style permissions on the file. Defaults to0o644
Example:
SSH.send(conn, "foo", "path/to/desired/file")
Specs
like send/4
, except raises on errors, instead of returning an error tuple.
Specs
stream(conn(), String.t(), keyword()) :: {:ok, SSH.Stream.t()} | {:error, String.t()}
creates an SSH stream struct as an ok tuple or error tuple.
Options
{iostream, redirect}
:iostream
may be either:stdout
or:stderr
.redirect
may be one of the following::stream
sends the data to the stream.:stdout
sends the data to thegroup_leader
stdout.:stderr
sends the data to the standard error io stream.:silent
deletes all of the data.:raw
sends the data to the stream tagged with source information as either{:stdout, data}
or{:stderr, data}
, as appropriate.{:file, path}
sends the data to a new or existing file at the provided path.fun/1
processes the data via the function, with the output flat-mapped into the stream. this means that the results offun/1
should be lists, with an empty list sending nothing into the stream.fun/2
is likefun/1
except the stream struct is passed as the second parameter. The output offun/2
should take the shape{flat_map_results, modified_stream}
. You may use the:data
field of the stream struct to store arbitrary data; and a value ofnil
indicates that it has been unused.
{:stream_control_messages, boolean}
: should the stream control messages:eof
, or{:retval, integer}
be sent to the stream?module: {mod, init}
, The stream is operated using an module with behaviourSSH.ModuleApi
data_timeout: timeout
, how long to wait between packets till we send a timeout event.
Specs
stream!(conn(), String.t(), keyword()) :: SSH.Stream.t() | no_return()
like stream/2
, except raises on an error instead of an error tuple.