Snex.Interpreter (Snex v0.4.1)

Runs a Python interpreter in a separate OS process.

This module is responsible for facilitating in-and-out communication between Elixir and the spawned Python interpreter.

Usually you won't interact with this module directly. Instead, you would create a custom interpreter module with use Snex.Interpreter:

defmodule SnexTest.NumpyInterpreter do
  use Snex.Interpreter,
    pyproject_toml: """
    [project]
    name = "my-numpy-project"
    version = "0.0.0"
    requires-python = "==3.11.*"
    dependencies = ["numpy>=2"]
    """
  end

See the Snex module documentation for more detail.

Summary

Types

environment()

init_script()

option()

Options for start_link/1.

port_opts()

See :erlang.open_port/2 options for detailed documentation.

server()

Running instance of Snex.Interpreter.

wrap_exec()

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

os_pid(interpreter)

Returns the OS PID of the Python interpreter.

start_link(opts \\ [])

Starts a new Python interpreter.

stop(interpreter, reason \\ :normal, timeout \\ :infinity)

Stops the interpreter process with reason reason.

Types

environment()

@type environment() :: %{optional(String.t()) => String.t()}

init_script()

@type init_script() ::
  String.t()
  | Snex.Code.t()
  | {String.t() | Snex.Code.t(), %{optional(String.t()) => any()}}

option()

@type option() ::
  {:python, String.t()}
  | {:wrap_exec, wrap_exec()}
  | {:cd, Path.t()}
  | {:environment, environment()}
  | {:init_script, init_script()}
  | {:init_script_timeout, timeout()}
  | {:sync_start?, boolean()}
  | {:label, term()}
  | {:encoding_opts, Snex.Serde.encoding_opts()}
  | {:port_opts, port_opts()}
  | {:eager_polyfill?, boolean()}
  | GenServer.option()

Options for start_link/1.

port_opts()

@type port_opts() ::
  :use_stdio
  | {:parallelism, boolean()}
  | {:busy_limits_port, {non_neg_integer(), non_neg_integer()} | :disabled}
  | {:busy_limits_msgq, {non_neg_integer(), non_neg_integer()} | :disabled}

See :erlang.open_port/2 options for detailed documentation.

:use_stdio is false by default, and Snex is optimized to use non-stdio pipes. Setting this option is not generally recommended, but can be useful for running Python interpreter in a Docker container or over SSH, where stdio is the only immediately available transport. Important: this option will set sys.stdin and sys.stdout to None in the Python process.

:busy_limits_port is set to {4194304, 8388608} if not specified. The high watermark of this option is also used as the buffer limit on Python side.

server()

@type server() :: GenServer.server()

Running instance of Snex.Interpreter.

wrap_exec()

@type wrap_exec() :: mfa() | (String.t(), [String.t()] -> {String.t(), [String.t()]})

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

os_pid(interpreter)

@spec os_pid(server()) :: non_neg_integer()

Returns the OS PID of the Python interpreter.

start_link(opts \\ [])

@spec start_link([option()]) :: GenServer.on_start()

Starts a new Python interpreter.

The interpreter can be used by functions in the Snex module.

Options

:python (String.t/0) - The Python executable to use. This can be a full path or a command to find via System.find_executable/1.
:wrap_exec (wrap_exec/0) - A function to wrap the Python executable and arguments. It can be given as an MFA or a function that takes two arguments: the Python executable path and its arguments. If given as {M,F,A}, the arguments will be appended to the A argument list.
:cd (String.t/0) - The directory to change to before running the interpreter.
:environment (environment/0) - A map of environment variables to set when running the Python executable.
:init_script (init_script/0) - A string of Python code to run when the interpreter is started, or a tuple with {python_code, additional_vars}. additional_vars are additional variables that will be added to the root environment before running the script.
The environment left by the script will be the initial context for all Snex.make_env/3 calls using this interpreter. This includes the variables passed through additional_vars. E.g.:
```
{:ok, inp} = Snex.Interpreter.start_link(init_script: {"y = 2 * x", %{"x" => 3}})
# Any new `env` will already contain `x` and `y`
{:ok, env} = Snex.make_env(inp)
{:ok, {3, 6}} = Snex.pyeval(env, "return x, y")
```
Failing to run the script will cause the process initialization to fail.
:init_script_timeout (timeout/0) - The timeout for the init script. Can be a number of milliseconds or :infinity. Default: 60000.
:sync_start? (boolean/0) - If true, the interpreter will start and run the init script in the init/1 callback. Setting this to false is useful for long-running init scripts; the downside is that if something goes wrong, the interpreter process will start crashing after successfully starting as a part of the supervision tree. Default: true.
:label (term/0) - The label of the interpreter process. This label will be used to label the process through :proc_lib.set_label/1.
:encoding_opts (Snex.Serde.encoding_opts/0) - Options for encoding Elixir terms to a desired representation on the Python side. These settings will be used when encoding terms for this interpreter, but can be overridden by individual commands.
:port_opts (port_opts/0) - Advanced options for the port used to communicate with the Python interpreter.
:eager_polyfill? (boolean/0) - If true, Snex's asyncio.AbstractEventLoop will use an eager polyfill for the event loop. This causes tasks scheduled with loop.call_soon (including asyncio.Future.set_result and loop.create_task) to run immediately in the same loop tick, instead of waiting for epoll() inbetween. Reduces latency of Snex operations, but is potentially incompatible with (or obsoleted by) future Python versions. Default: true.
any other options will be passed to GenServer.start_link/3.

stop(interpreter, reason \\ :normal, timeout \\ :infinity)

@spec stop(server(), term(), timeout()) :: :ok

Stops the interpreter process with reason reason.

Pending callers will return {:error, %Snex.Error{code: :call_failed, reason: reason}}.