Charms.Defm.Definition (charms v0.1.4)

Charms.Defm.Definition provides functions to define and compile defm functions that can be JIT/AOT compiled to native targets.

The compilation process

The principle of the compilation process:

  • convert the Elixir code into MLIR IR while preserving the Elixir semantics.
  • work with Elixir's code analysis infrastructure and LSP-like tools.

With this in mind, the compilation process is divided into these steps:

  1. Convert the Elixir code into MLIR IR, then optimize the IR.
  2. Analyze the AST and IR and generate proper Elixir side effects.
  3. Finally convert the IR into a bytecode representation and persist it as an Elixir string.
graph TD
  A[Elixir Code] -->|Step 1: Convert to MLIR IR| B[MLIR IR]
  B -->|Optimize IR| C[Optimized MLIR IR]
  C -->|Step 2: Analyze AST and IR| D[Generate Elixir Side Effects]
  D --> E[Elixir Function Definitions]
  D --> F[Remote Function Calls]
  C -->|Step 3: Convert IR to Bytecode| G[Bytecode Representation]
  G -->|Persist as Elixir String| H[Persisted Bytecode]
  H -->|Consumed by JIT/AOT Engine| I[Further Optimization]
  I --> J[Native Target Code]

The optimization before JIT/AOT should focus on:

  • Complete the semantics which usually absent in the Elixir code, such as the return type of a function.
  • Fill the IR with Elixir's debug information, including code location and function arity. Elixir side effects include:
  • Elixir function definitions as placeholder
  • Remote function calls to build the module dependencies The IR will be later consumed and further optimized by a JIT/AOT engine. It should be as target independent as possible.

Summary

Functions

compile(definitions)

Compile definitions into MLIR module. Compile definitions into an MLIR module.

declare(env, call, body)

Declare a function that can be JIT compiled and generate Elixir invoker function.

do_compile(ctx, definitions)

Functions

compile(definitions)

Compile definitions into MLIR module. Compile definitions into an MLIR module.

Partial and lazy compilation

When compiling defm, Charms performs preprocessing before the definition is fully translated to the target language or IR. Notable preprocessing steps include:

  • Extracting the return type of the defm definition, and wrapping it as an anonymous function to be called on-demand at the invocation site.
  • Determine the MLIR ops available in the definition.

declare(env, call, body)

Declare a function that can be JIT compiled and generate Elixir invoker function.

The call signature will be decomposed and transformed into a normalized form.

do_compile(ctx, definitions)