Python_Interface_Documentation (Snex v0.4.1)

Copy Markdown View Source

Documentation for Python snex module.

The snex module is pre-imported inside Snex.pyeval/4 environments. It can also be imported from your Python project if it's ran by Snex - or if you include deps/snex/py_src in your PYTHONPATH.

Important

This is not a functional Elixir module. The types and functions on this page describe Python types and functions.

Summary

Python Types

Atom represents an Elixir atom.

A proxy class for calling BEAM functions from Python.

DistinctAtom represents an Elixir atom that is not equal to a bare str.

Configuration for encoding Elixir terms into Python objects.

Term represents an Elixir term, opaque on Python side.

Python Functions

Call a function in BEAM, returning the result. Thread-safe.

Call a function in BEAM without waiting for a response. Thread-safe.

Run an IO loop for a reader/writer pair of asyncio streams. Thread-safe.

Send data to a BEAM process. Thread-safe.

Run Snex server loop as a context manager.

Run Snex server loop until canceled.

Set a custom encoding function for objects that are not supported by default.

snex.EncodingOpt

The representation of Elixir atoms on the Python side.

The representation of Elixir binaries on the Python side.

The representation of Elixir MapSets on the Python side.

Python Types

snex.Atom()

@type snex.Atom() :: str()

Atom represents an Elixir atom.

Will be converted to atom when decoded in Elixir.

snex.BeamModuleProxy()

@type snex.BeamModuleProxy() :: object()

A proxy class for calling BEAM functions from Python.

BeamModuleProxy provides a syntax sugar for snex.call and snex.cast. Its instances can be used to "address" BEAM functions using an Elixir-like syntax.

An instance of this class representing the Elixir module prefix is available exported as snex.Elixir, and is auto-imported as Elixir into Snex environments. See the examples below for usage.

Examples

>>> from snex import Elixir
>>> await Elixir.Enum.frequencies(["a", "b", "a", "a", "d", "b"])
{"a": 3, "b": 2, "d": 1}

# Similar to `snex.call`, you can pass `node` or `result_encoding_opts`
>>> encoding_opts = snex.EncodingOpts(binary_as=snex.EncodingOpt.BinaryAs.BYTES)
>>> await Elixir.String.reverse("hello", result_encoding_opts=encoding_opts)
b"olleh"

# `snex.cast` can be invoked by passing `cast=True`
>>> await Elixir.Kernel.send("registered_name", "hello!", cast=True)
None

# Make a custom proxy object for a non-Elixir module
>>> erlang = BeamModuleProxy("erlang")
>>> await erlang.float_to_binary(3.14)
"3.14000000000000012434e+00"

snex.DistinctAtom()

@type snex.DistinctAtom() :: snex.Atom()

DistinctAtom represents an Elixir atom that is not equal to a bare str.

Will be converted to atom when decoded in Elixir.

Unlike snex.Atom, snex.DistinctAtom does not compare equal to a bare str (or, by extension, snex.Atom). In other words, "foo" != snex.DistinctAtom("foo") while "foo" == snex.Atom("foo"). This allows to mix atom and string keys in a dictionary.

snex.EncodingOpts()

@type snex.EncodingOpts() :: dict()

Configuration for encoding Elixir terms into Python objects.

Attributes

  • binary_as (NotRequired[EncodingOpt.BinaryAs]) - How Elixir binaries are encoded
  • set_as (NotRequired[EncodingOpt.SetAs]) - How Elixir MapSets are encoded
  • atom_as (NotRequired[EncodingOpt.AtomAs]) - How Elixir atoms are encoded

snex.Term()

@type snex.Term() :: object()

Term represents an Elixir term, opaque on Python side.

Created on Elixir side by encoding otherwise unserializable terms, or wrapping a term with Snex.Serde.term/1. It's not supposed to be created or modified on Python side. It's decoded back to the original term on Elixir side.

Python Functions

snex.call(module, function, args, *, node, result_encoding_opts)

Call a function in BEAM, returning the result. Thread-safe.

async def call(
    module: str | Atom | Term,
    function: str | Atom | Term,
    args: Iterable[object],
    *,
    node: str | Atom | Term | None = None,
    result_encoding_opts: EncodingOpts | None = None
) -> Any

module, function and node can be given as strings, atoms, or snex.Term that resolves either to a string, or an atom. They will be converted to atoms on the BEAM side.

The function will be called in a new process on the BEAM side.

Args

  • module - The module to call the function from
  • function - The function to call
  • args - The arguments to pass to the function
  • node - The node to call the function on; defaults to the current node
  • result_encoding_opts - The options to use for encoding the result. They will override the interpreter's :encoding_opts.

snex.cast(module, function, args, *, node)

Call a function in BEAM without waiting for a response. Thread-safe.

async def cast(
    module: str | Atom | Term,
    function: str | Atom | Term,
    args: Iterable[object],
    *,
    node: str | Atom | Term | None = None
) -> None

module, function and node can be given as strings, atoms, or snex.Term that resolves either to a string, or an atom. They will be converted to atoms on the BEAM side.

The function will be called in a new process on the BEAM side.

Args

  • module - The module to call the function from
  • function - The function to call
  • args - The arguments to pass to the function
  • node - The node to call the function on; defaults to the current node

snex.io_loop_for_connection(sub_reader, sub_writer)

Run an IO loop for a reader/writer pair of asyncio streams. Thread-safe.

async def io_loop_for_connection(
    sub_reader: asyncio.StreamReader,
    sub_writer: asyncio.StreamWriter
) -> None

Can be used to connect a subprocess to the main Snex loop. See snex.serve for an example.

Closes the writer on the return path and waits until it's closed.

Examples

await asyncio.start_unix_server(snex.io_loop_for_connection, socket_path)

snex.send(to, data)

Send data to a BEAM process. Thread-safe.

async def send(to: object, data: object) -> None

Args

  • to - The BEAM process to send the data to; can be any identifier that can be used as destination in Kernel.send/2
  • data - The data to send; will be encoded and sent to the BEAM process

Examples

# Send to a registered process with an atom name
snex.send("registered_name", "hello to a registered process!")

# Send to a remote process with a tuple name and node
snex.send(("myname", "mynode@localhost"), "hello to a remote process!")

# Use `snex.Term` passed through with `Snex.pyeval/4`
# e.g. `Snex.pyeval(env, "snex.cast(self, 'hello self!')", %{"self" => self()}`)
snex.send(self, "hello self!")

snex.serve(reader, writer)

Run Snex server loop as a context manager.

def serve(
    reader: asyncio.StreamReader,
    writer: asyncio.StreamWriter
) -> AsyncGenerator[None, None]

The server loop is necessary to use snex.call/snex.cast from external Python processes. Usually, you'll want to run it in a subprocess, and connect the other end of the reader/writer pair to the main Snex process.

Examples

import asyncio
from asyncio import start_unix_server
from concurrent.futures import ProcessPoolExecutor

import snex


async def in_subprocess_loop(sock_path: str) -> int:
    reader, writer = await asyncio.open_unix_connection(sock_path)
    async with snex.serve(reader, writer):
        return await snex.call("Elixir.System", "pid", [])


def in_subprocess(sock_path: str) -> int:
    return asyncio.run(in_subprocess_loop(sock_path))


async def main():
    sock_path = "/tmp/snex.sock"
    loop = asyncio.get_running_loop()
    async with await start_unix_server(snex.io_loop_for_connection, sock_path):
        with ProcessPoolExecutor() as pool:
            return await loop.run_in_executor(pool, in_subprocess, sock_path)

snex.serve_forever(reader, writer, *, on_ready)

Run Snex server loop until canceled.

async def serve_forever(
    reader: asyncio.StreamReader,
    writer: asyncio.StreamWriter,
    *,
    on_ready: Callable[[], None] | None = None
) -> None

Similar to async with snex.serve(reader, writer).

snex.set_custom_encoder(encoder_fun)

Set a custom encoding function for objects that are not supported by default.

def set_custom_encoder(
    encoder_fun: Callable[[object], object]
) -> None

Args

  • encoder_fun - The function to use to encode the objects

snex.EncodingOpt

Options for encoding Elixir terms into Python objects.

EncodingOpt.AtomAs()

@type EncodingOpt.AtomAs() :: enum.StrEnum()

The representation of Elixir atoms on the Python side.

  • ATOM ("atom") encodes atom as snex.Atom
  • DISTINCT_ATOM ("distinct_atom") encodes atom as snex.DistinctAtom

EncodingOpt.BinaryAs()

@type EncodingOpt.BinaryAs() :: enum.StrEnum()

The representation of Elixir binaries on the Python side.

  • STR ("str") encodes binary as str
  • BYTES ("bytes") encodes binary as bytes
  • BYTEARRAY ("bytearray") encodes binary as bytearray

EncodingOpt.SetAs()

@type EncodingOpt.SetAs() :: enum.StrEnum()

The representation of Elixir MapSets on the Python side.

  • SET ("set") encodes MapSet as set
  • FROZENSET ("frozenset") encodes MapSet as frozenset

snex.LoggingHandler

A logging handler that logs messages to the Elixir logger.

The Elixir log level will be set according to the numeric log level in Python. The standard log levels in Python (DEBUG, INFO, WARNING, ERROR, CRITICAL) translate directly; custom log levels are translated to the next higher log level.

Log levels between INFO and WARNING are translated to :notice. Log levels higher than CRITICAL are translated to :alert. Log levels 10 higher than CRITICAL are translated to :emergency.

Adds the following metadata to the log:

  • file - The file the log was emitted from (full path)
  • line - The line the log was emitted from
  • time - The time the log was emitted, in microseconds since the epoch. This meta will get picked up by Elixir logger and set as the log timestamp.
  • python_logger_name - The name of the logger
  • python_log_level - The original log level in Python
  • python_module - The module the log was emitted from
  • python_function - The function the log was emitted from
  • python_process_id - The OS PID of the process (if available)
  • python_thread_id - The OS TID of the thread (if available)
  • python_thread_name - The name of the thread (if available)
  • python_task_name - The name of the task (if available)
  • python_exception - The name of the exception type (if available)

LoggingHandler.__init__(self, level, *, default_metadata, extra_metadata_keys)

Initialize the logging handler.

def __init__(
    self,
    level: int | str = 0,
    *,
    default_metadata: Mapping[str, object] | None = None,
    extra_metadata_keys: Iterable[str] | None = None
) -> None

Args

  • level - The log level to use
  • default_metadata - The default metadata to send to the Elixir logger.
  • extra_metadata_keys - Extra keys to read from the log record and send to the Elixir logger as metadata.