Metastatic.AST (Metastatic v0.10.4)

View Source

M2: Meta-Model for Programming Language Abstract Syntax.

This module defines the meta-types that all language ASTs (M1 level) must conform to. Think of this as the "UML" for programming language ASTs.

Meta-Modeling Hierarchy

  • M3: Elixir type system (@type, @spec)
  • M2: This module (MetaAST) - defines what AST nodes CAN be
  • M1: Python AST, JavaScript AST, Elixir AST - what specific code IS
  • M0: Runtime execution - what code DOES

Node Structure

All MetaAST nodes are uniform 3-element tuples:

{type_atom, keyword_meta, children_or_value}

Where:

  • type_atom - Node type (e.g., :literal, :container, :function_def)
  • keyword_meta - Keyword list containing metadata (line, column, subtype, etc.)
  • children_or_value - Either a value (for leaf nodes) or list of child nodes

Third Element Semantics

The third element varies by node type:

  • Leaf nodes (literal, variable): The actual value (42, "x")
  • Composite nodes (binary_op, function_call): List of child AST nodes
  • Container nodes (container, function_def): List of body statements

Examples

# Literal integer
{:literal, [subtype: :integer, line: 10], 42}

# Variable
{:variable, [line: 5], "x"}

# Binary operation
{:binary_op, [category: :arithmetic, operator: :+],
 [{:variable, [], "x"}, {:literal, [subtype: :integer], 5}]}

# Map with pairs
{:map, [],
 [{:pair, [], [{:literal, [subtype: :symbol], :name},
               {:literal, [subtype: :string], "Alice"}]}]}

Semantic Enrichment (op_kind)

Function call nodes may include semantic metadata via the op_kind key. This metadata is added during M1 -> M2 transformation by the semantic enricher, which detects known patterns (e.g., database operations).

When present, op_kind is a keyword list with:

  • :domain - Semantic domain (e.g., :db, :http, :cache)
  • :operation - Operation type within the domain (e.g., :retrieve, :create)
  • :target - Optional target entity (e.g., "User", "orders")
  • :async - Whether the operation is asynchronous
  • :framework - Source framework (e.g., :ecto, :sqlalchemy)

Example:

{:function_call,
 [name: "Repo.get", op_kind: [domain: :db, operation: :retrieve, target: "User"]],
 [{:variable, [], "User"}, {:variable, [], "id"}]}

Analyzers can use op_kind for precise semantic detection instead of heuristics. See Metastatic.Semantic.OpKind for the full type specification.

Traversal

Use traverse/4 for walking and transforming ASTs:

AST.traverse(ast, acc, &pre/2, &post/2)

This mirrors Macro.traverse/4 from Elixir's standard library.

Summary

Types

Container type classification.

Node type atoms for M2.1 Core Layer - universal concepts.

Database operation types for op_kind metadata.

Node type atoms for M2.2 Extended Layer - common patterns.

Semantic subtype for literals.

A MetaAST node is a 3-tuple: {type, metadata, children_or_value}.

Node type atoms for M2.3 Native Layer - language-specific escape hatch.

All valid node type atoms.

Semantic operation kind metadata added to function_call nodes. Provides precise semantic information about what a function does.

Category for binary/unary operators.

Semantic domain for op_kind metadata. Domains categorize operations by their high-level purpose.

Node type atoms for M2.2s Structural Layer - organizational constructs.

Visibility modifier.

Functions

Create a block node.

Extract the children or value from a MetaAST node.

Validate that a term conforms to the M2 meta-model.

Extract the name from a container node.

Create a function call node.

Create a function definition node.

Extract the name from a function_def node.

Get the visibility of a function_def node.

Get a specific metadata value by key.

Check if a container has state (classes typically have state).

Create an inline match node.

Get the layer classification for a node type.

Check if a node is a leaf node (no children to traverse).

Extract location information from a MetaAST node.

Create a map node with pairs.

Extract the metadata keyword list from a MetaAST node.

Extract all metadata from a MetaAST node as a keyword list.

Extract arity from node metadata.

Extract container name from node metadata.

Extract file path from node metadata.

Extract function name from node metadata.

Extract module name from node metadata.

Extract visibility from node metadata.

Create a key-value pair node for maps.

Traverse a MetaAST with only a post function (pre is identity).

Traverse a MetaAST with only a pre function (post is identity).

Put a metadata value by key.

Traverse a MetaAST, applying pre and post functions.

Extract the node type from a MetaAST node.

Update the children/value of a node.

Update multiple metadata keys at once.

Create a variable node.

Extract all variable names referenced in an AST.

Merge context metadata into a node's metadata.

Attach location information to a MetaAST node.

Types

container_type()

@type container_type() :: :module | :class | :namespace

Container type classification.

core_type()

@type core_type() ::
  :literal
  | :variable
  | :list
  | :map
  | :pair
  | :tuple
  | :binary_op
  | :unary_op
  | :function_call
  | :conditional
  | :early_return
  | :block
  | :assignment
  | :inline_match

Node type atoms for M2.1 Core Layer - universal concepts.

db_operation()

@type db_operation() ::
  :retrieve
  | :retrieve_all
  | :query
  | :create
  | :update
  | :delete
  | :transaction
  | :preload
  | :aggregate

Database operation types for op_kind metadata.

extended_type()

@type extended_type() ::
  :loop
  | :lambda
  | :collection_op
  | :pattern_match
  | :match_arm
  | :exception_handling
  | :async_operation

Node type atoms for M2.2 Extended Layer - common patterns.

literal_subtype()

@type literal_subtype() ::
  :integer | :float | :string | :boolean | :null | :symbol | :regex

Semantic subtype for literals.

meta_ast()

@type meta_ast() :: {atom(), keyword(), term()}

A MetaAST node is a 3-tuple: {type, metadata, children_or_value}.

The type atom identifies the node kind. The metadata is a keyword list with location, subtype, and other info. The third element is either a value (leaf nodes) or list of children.

native_type()

@type native_type() :: :language_specific

Node type atoms for M2.3 Native Layer - language-specific escape hatch.

node_type()

@type node_type() :: core_type() | extended_type() | structural_type() | native_type()

All valid node type atoms.

op_kind()

@type op_kind() :: [
  domain: semantic_domain(),
  operation: atom(),
  target: String.t() | nil,
  async: boolean(),
  framework: atom() | nil
]

Semantic operation kind metadata added to function_call nodes. Provides precise semantic information about what a function does.

Fields:

  • :domain - High-level semantic category (required)
  • :operation - Specific operation within the domain (required)
  • :target - Target entity or resource (optional)
  • :async - Whether the operation is asynchronous (optional)
  • :framework - Source framework that provides this operation (optional)

Example: [domain: :db, operation: :retrieve, target: "User", framework: :ecto]

operator_category()

@type operator_category() :: :arithmetic | :comparison | :boolean

Category for binary/unary operators.

semantic_domain()

@type semantic_domain() :: :db | :http | :cache | :queue | :file | :external

Semantic domain for op_kind metadata. Domains categorize operations by their high-level purpose.

structural_type()

@type structural_type() ::
  :container
  | :function_def
  | :param
  | :attribute_access
  | :augmented_assignment
  | :property

Node type atoms for M2.2s Structural Layer - organizational constructs.

visibility()

@type visibility() :: :public | :private | :protected

Visibility modifier.

Functions

binary_op(category, operator, left, right, extra_meta \\ [])

@spec binary_op(operator_category(), atom(), meta_ast(), meta_ast(), keyword()) ::
  meta_ast()

Create a binary operation node.

Examples

iex> left = {:variable, [], "x"}
iex> right = {:literal, [subtype: :integer], 5}
iex> Metastatic.AST.binary_op(:arithmetic, :+, left, right)
{:binary_op, [category: :arithmetic, operator: :+], [{:variable, [], "x"}, {:literal, [subtype: :integer], 5}]}

block(statements, meta \\ [])

@spec block(
  [meta_ast()],
  keyword()
) :: meta_ast()

Create a block node.

Examples

iex> stmts = [{:variable, [], "x"}, {:literal, [subtype: :integer], 42}]
iex> Metastatic.AST.block(stmts)
{:block, [], [{:variable, [], "x"}, {:literal, [subtype: :integer], 42}]}

children(arg)

@spec children(meta_ast()) :: term()

Extract the children or value from a MetaAST node.

For leaf nodes (literal, variable), returns the value. For composite nodes, returns the list of children.

Examples

iex> Metastatic.AST.children({:literal, [subtype: :integer], 42})
42

iex> left = {:variable, [], "x"}
iex> right = {:literal, [subtype: :integer], 5}
iex> Metastatic.AST.children({:binary_op, [operator: :+], [left, right]})
[{:variable, [], "x"}, {:literal, [subtype: :integer], 5}]

conforms?(ast)

@spec conforms?(term()) :: boolean()

Validate that a term conforms to the M2 meta-model.

Checks that the structure is a valid 3-tuple MetaAST node with proper type, metadata, and children.

Examples

iex> Metastatic.AST.conforms?({:literal, [subtype: :integer], 42})
true

iex> Metastatic.AST.conforms?({:binary_op, [category: :arithmetic, operator: :+],
...>   [{:variable, [], "x"}, {:literal, [subtype: :integer], 5}]})
true

iex> Metastatic.AST.conforms?({:invalid_node, "data"})
false

iex> Metastatic.AST.conforms?("not a tuple")
false

container(type, name, body, extra_meta \\ [])

@spec container(container_type(), String.t(), [meta_ast()], keyword()) :: meta_ast()

Create a container node.

Examples

iex> func_def1 = {:function_def, [name: "add", params: ["x", "y"]], []}
iex> func_def2 = {:function_def, [name: "sub", params: ["x", "y"]], []}
iex> Metastatic.AST.container(:module, "MyApp.Math", [func_def1, func_def2])
{:container, [container_type: :module, name: "MyApp.Math"], [{:function_def, [name: "add", params: ["x", "y"]], []}, {:function_def, [name: "sub", params: ["x", "y"]], []}]}

container_name(arg1)

@spec container_name(meta_ast()) :: String.t() | nil

Extract the name from a container node.

Examples

iex> ast = {:container, [container_type: :module, name: "MyApp.Math"], []}
iex> Metastatic.AST.container_name(ast)
"MyApp.Math"

function_call(name, args, extra_meta \\ [])

@spec function_call(String.t(), [meta_ast()], keyword()) :: meta_ast()

Create a function call node.

Examples

iex> args = [{:variable, [], "x"}, {:literal, [subtype: :integer], 5}]
iex> Metastatic.AST.function_call("add", args)
{:function_call, [name: "add"], [{:variable, [], "x"}, {:literal, [subtype: :integer], 5}]}

function_def(name, params, body, extra_meta \\ [])

@spec function_def(String.t(), [term()], [meta_ast()], keyword()) :: meta_ast()

Create a function definition node.

Examples

iex> body = [{:binary_op, [category: :arithmetic, operator: :+],
...>   [{:variable, [], "x"}, {:variable, [], "y"}]}]
iex> Metastatic.AST.function_def("add", ["x", "y"], body)
{:function_def, [name: "add", params: ["x", "y"]], [{:binary_op, [category: :arithmetic, operator: :+], [{:variable, [], "x"}, {:variable, [], "y"}]}]}

function_name(arg1)

@spec function_name(meta_ast()) :: String.t() | nil

Extract the name from a function_def node.

Examples

iex> body = {:binary_op, [operator: :+], [{:variable, [], "x"}, {:variable, [], "y"}]}
iex> ast = {:function_def, [name: "add", params: ["x", "y"]], [body]}
iex> Metastatic.AST.function_name(ast)
"add"

function_visibility(arg1)

@spec function_visibility(meta_ast()) :: visibility()

Get the visibility of a function_def node.

Examples

iex> body = {:binary_op, [operator: :+], [{:variable, [], "x"}, {:variable, [], "y"}]}
iex> ast = {:function_def, [name: "add", visibility: :public], [body]}
iex> Metastatic.AST.function_visibility(ast)
:public

get_meta(arg, key, default \\ nil)

@spec get_meta(meta_ast(), atom(), term()) :: term()

Get a specific metadata value by key.

Examples

iex> ast = {:literal, [subtype: :integer, line: 10], 42}
iex> Metastatic.AST.get_meta(ast, :line)
10

iex> ast = {:variable, [], "x"}
iex> Metastatic.AST.get_meta(ast, :line)
nil

iex> ast = {:variable, [], "x"}
iex> Metastatic.AST.get_meta(ast, :line, 0)
0

has_state?(arg1)

@spec has_state?(meta_ast()) :: boolean()

Check if a container has state (classes typically have state).

Examples

iex> ast = {:container, [container_type: :class, name: "Counter"], []}
iex> Metastatic.AST.has_state?(ast)
true

iex> ast = {:container, [container_type: :module, name: "Math"], []}
iex> Metastatic.AST.has_state?(ast)
false

inline_match(pattern, value, meta \\ [])

@spec inline_match(meta_ast(), meta_ast(), keyword()) :: meta_ast()

Create an inline match node.

Examples

iex> pattern = {:variable, [], "x"}
iex> value = {:literal, [subtype: :integer], 42}
iex> Metastatic.AST.inline_match(pattern, value)
{:inline_match, [], [{:variable, [], "x"}, {:literal, [subtype: :integer], 42}]}

layer(type)

@spec layer(atom()) :: :core | :extended | :structural | :native | :unknown

Get the layer classification for a node type.

Returns :core, :extended, :structural, or :native.

Examples

iex> Metastatic.AST.layer(:literal)
:core

iex> Metastatic.AST.layer(:lambda)
:extended

iex> Metastatic.AST.layer(:container)
:structural

iex> Metastatic.AST.layer(:language_specific)
:native

leaf?(arg1)

@spec leaf?(meta_ast()) :: boolean()

Check if a node is a leaf node (no children to traverse).

Examples

iex> Metastatic.AST.leaf?({:literal, [subtype: :integer], 42})
true

iex> Metastatic.AST.leaf?({:variable, [], "x"})
true

iex> left = {:variable, [], "x"}
iex> right = {:literal, [subtype: :integer], 5}
iex> Metastatic.AST.leaf?({:binary_op, [], [left, right]})
false

literal(subtype, value, extra_meta \\ [])

@spec literal(literal_subtype(), term(), keyword()) :: meta_ast()

Create a literal node.

Examples

iex> Metastatic.AST.literal(:integer, 42)
{:literal, [subtype: :integer], 42}

iex> Metastatic.AST.literal(:string, "hello", line: 5)
{:literal, [subtype: :string, line: 5], "hello"}

location(arg1)

@spec location(meta_ast()) :: map() | nil

Extract location information from a MetaAST node.

Returns a map with :line, :col, :end_line, :end_col if present in metadata.

Examples

iex> ast = {:literal, [subtype: :integer, line: 10, col: 5], 42}
iex> Metastatic.AST.location(ast)
%{line: 10, col: 5}

iex> ast = {:variable, [], "x"}
iex> Metastatic.AST.location(ast)
nil

map_node(pairs, meta \\ [])

@spec map_node(
  [meta_ast()],
  keyword()
) :: meta_ast()

Create a map node with pairs.

Examples

iex> pairs = [Metastatic.AST.pair(
...>   {:literal, [subtype: :symbol], :name},
...>   {:literal, [subtype: :string], "Alice"})]
iex> Metastatic.AST.map_node(pairs)
{:map, [], [{:pair, [], [{:literal, [subtype: :symbol], :name}, {:literal, [subtype: :string], "Alice"}]}]}

meta(arg)

@spec meta(meta_ast()) :: keyword()

Extract the metadata keyword list from a MetaAST node.

Examples

iex> Metastatic.AST.meta({:literal, [subtype: :integer, line: 10], 42})
[subtype: :integer, line: 10]

iex> Metastatic.AST.meta({:variable, [], "x"})
[]

metadata(arg1)

@spec metadata(meta_ast()) :: keyword()

Extract all metadata from a MetaAST node as a keyword list.

Returns the full metadata including location, M1 context (module, function, arity, etc.), and node-specific metadata (subtype, operator, etc.).

Examples

iex> ast = {:literal, [subtype: :integer, line: 10], 42}
iex> Metastatic.AST.metadata(ast)
[subtype: :integer, line: 10]

iex> ast = {:variable, [line: 5, module: "MyApp", function: "create"], "x"}
iex> Metastatic.AST.metadata(ast)
[line: 5, module: "MyApp", function: "create"]

iex> ast = {:variable, [], "x"}
iex> Metastatic.AST.metadata(ast)
[]

node_arity(ast)

@spec node_arity(meta_ast()) :: non_neg_integer() | nil

Extract arity from node metadata.

node_container(ast)

@spec node_container(meta_ast()) :: String.t() | nil

Extract container name from node metadata.

node_file(ast)

@spec node_file(meta_ast()) :: String.t() | nil

Extract file path from node metadata.

node_function(ast)

@spec node_function(meta_ast()) :: String.t() | nil

Extract function name from node metadata.

node_module(ast)

@spec node_module(meta_ast()) :: String.t() | nil

Extract module name from node metadata.

Examples

iex> ast = {:variable, [module: "MyApp.Controller"], "x"}
iex> Metastatic.AST.node_module(ast)
"MyApp.Controller"

node_visibility(ast)

@spec node_visibility(meta_ast()) :: visibility() | nil

Extract visibility from node metadata.

pair(key, value, meta \\ [])

@spec pair(meta_ast(), meta_ast(), keyword()) :: meta_ast()

Create a key-value pair node for maps.

Examples

iex> key = {:literal, [subtype: :symbol], :name}
iex> value = {:literal, [subtype: :string], "Alice"}
iex> Metastatic.AST.pair(key, value)
{:pair, [], [{:literal, [subtype: :symbol], :name}, {:literal, [subtype: :string], "Alice"}]}

postwalk(ast, acc, fun)

@spec postwalk(meta_ast() | term(), acc, (meta_ast() | term(), acc ->
                                      {meta_ast() | term(), acc})) ::
  {meta_ast() | term(), acc}
when acc: term()

Traverse a MetaAST with only a post function (pre is identity).

Examples

{new_ast, acc} = AST.postwalk(ast, [], fn node, acc -> {node, acc} end)

prewalk(ast, acc, fun)

@spec prewalk(meta_ast() | term(), acc, (meta_ast() | term(), acc ->
                                     {meta_ast() | term(), acc})) ::
  {meta_ast() | term(), acc}
when acc: term()

Traverse a MetaAST with only a pre function (post is identity).

Examples

{new_ast, acc} = AST.prewalk(ast, [], fn node, acc -> {node, acc} end)

put_meta(arg, key, value)

@spec put_meta(meta_ast(), atom(), term()) :: meta_ast()

Put a metadata value by key.

Examples

iex> ast = {:literal, [subtype: :integer], 42}
iex> Metastatic.AST.put_meta(ast, :line, 10)
{:literal, [line: 10, subtype: :integer], 42}

iex> ast = {:variable, [line: 5], "x"}
iex> Metastatic.AST.put_meta(ast, :line, 10)
{:variable, [line: 10], "x"}

traverse(ast, acc, pre, post)

@spec traverse(meta_ast() | term(), acc, pre_fun, post_fun) ::
  {meta_ast() | term(), acc}
when acc: term(),
     pre_fun: (meta_ast() | term(), acc -> {meta_ast() | term(), acc}),
     post_fun: (meta_ast() | term(), acc -> {meta_ast() | term(), acc})

Traverse a MetaAST, applying pre and post functions.

This mirrors Macro.traverse/4 from Elixir's standard library.

  • pre is called before visiting children, receives {ast, acc}, returns {ast, acc}
  • post is called after visiting children, receives {ast, acc}, returns {ast, acc}

The traversal visits children based on the node type:

  • Leaf nodes (literal, variable): No children to visit
  • Composite nodes: Visits each child in the list
  • Non-AST values: Passed through unchanged

Examples

# Count all nodes
{_ast, count} = AST.traverse(ast, 0, fn node, acc -> {node, acc + 1} end, fn node, acc -> {node, acc} end)

# Collect all variable names
{_ast, vars} = AST.traverse(ast, [], fn
  {:variable, _, name} = node, acc -> {node, [name | acc]}
  node, acc -> {node, acc}
end, fn node, acc -> {node, acc} end)

# Transform all integers by doubling them
{new_ast, _} = AST.traverse(ast, nil, fn node, acc -> {node, acc} end, fn
  {:literal, meta, value} = node, acc ->
    if Keyword.get(meta, :subtype) == :integer do
      {{:literal, meta, value * 2}, acc}
    else
      {node, acc}
    end
  node, acc -> {node, acc}
end)

type(arg)

@spec type(meta_ast()) :: atom()

Extract the node type from a MetaAST node.

Examples

iex> Metastatic.AST.type({:literal, [subtype: :integer], 42})
:literal

iex> left = {:variable, [], "x"}
iex> right = {:literal, [subtype: :integer], 5}
iex> Metastatic.AST.type({:binary_op, [operator: :+], [left, right]})
:binary_op

update_children(arg, new_children)

@spec update_children(meta_ast(), term()) :: meta_ast()

Update the children/value of a node.

Examples

iex> old_left = {:variable, [], "x"}
iex> old_right = {:literal, [subtype: :integer], 5}
iex> ast = {:binary_op, [operator: :+], [old_left, old_right]}
iex> new_left = {:variable, [], "y"}
iex> new_right = {:literal, [subtype: :integer], 10}
iex> Metastatic.AST.update_children(ast, [new_left, new_right])
{:binary_op, [operator: :+], [{:variable, [], "y"}, {:literal, [subtype: :integer], 10}]}

update_meta(arg, updates)

@spec update_meta(
  meta_ast(),
  keyword()
) :: meta_ast()

Update multiple metadata keys at once.

Examples

iex> ast = {:literal, [subtype: :integer], 42}
iex> Metastatic.AST.update_meta(ast, line: 10, col: 5)
{:literal, [subtype: :integer, line: 10, col: 5], 42}

variable(name, meta \\ [])

@spec variable(
  String.t(),
  keyword()
) :: meta_ast()

Create a variable node.

Examples

iex> Metastatic.AST.variable("x")
{:variable, [], "x"}

iex> Metastatic.AST.variable("count", line: 10)
{:variable, [line: 10], "count"}

variables(ast)

@spec variables(meta_ast()) :: MapSet.t(String.t())

Extract all variable names referenced in an AST.

Uses traverse/4 internally to collect all variable nodes.

Examples

iex> ast = {:binary_op, [category: :arithmetic, operator: :+],
...>   [{:variable, [], "x"}, {:variable, [], "y"}]}
iex> Metastatic.AST.variables(ast)
MapSet.new(["x", "y"])

iex> ast = {:literal, [subtype: :integer], 42}
iex> Metastatic.AST.variables(ast)
MapSet.new([])

with_context(ast, context)

@spec with_context(meta_ast(), map()) :: meta_ast()

Merge context metadata into a node's metadata.

Used for attaching M1-level context like module name, function name, etc. Note: Order of merged keys is not guaranteed due to map iteration order.

Examples

iex> ast = {:variable, [line: 10], "x"}
iex> context = %{module: "MyApp"}
iex> {:variable, meta, "x"} = Metastatic.AST.with_context(ast, context)
iex> Keyword.get(meta, :module)
"MyApp"
iex> Keyword.get(meta, :line)
10

with_location(ast, loc)

@spec with_location(meta_ast(), map() | nil) :: meta_ast()

Attach location information to a MetaAST node.

Examples

iex> ast = {:literal, [subtype: :integer], 42}
iex> Metastatic.AST.with_location(ast, %{line: 10, col: 5})
{:literal, [subtype: :integer, line: 10, col: 5], 42}

iex> ast = {:variable, [], "x"}
iex> Metastatic.AST.with_location(ast, nil)
{:variable, [], "x"}