Getting Started with Metastatic Development

Welcome to Metastatic! This guide will help you get up and running with the development environment.

Prerequisites

Required

Elixir 1.19+ and Erlang/OTP 27+
Git for version control

Current Status

Metastatic is production-ready with comprehensive analysis capabilities.

Test coverage: 1764 tests passing (1523 tests + 241 doctests)
Language adapters: Python, Elixir, Erlang, Ruby, Haskell
Static analyzers: 9 core analyzers + 32 business logic analyzers
CWE coverage: 15 CWE Top 25 vulnerabilities detected

Current Capabilities:

Parse and transform code across Python, Elixir, Erlang, Ruby, and Haskell
Analyze function purity and side effects
Measure code complexity (6 comprehensive metric types)
Detect dead code and unreachable branches
Track unused variables with scope awareness
Generate control flow graphs (DOT/D3.js formats)
Perform taint analysis for security vulnerabilities
Scan for security issues with CWE identifiers (CWE Top 25 coverage)
Detect code smells and maintainability issues
32 language-agnostic business logic analyzers
Semantic operation detection via OpKind (DB, HTTP, file, cache, auth, queue, external API)
15+ CLI tools for all analysis operations

Optional (for extended language support)

Python 3.9+ for Python adapter
Node.js 16+ for JavaScript adapter (future)
Go 1.19+ for Go adapter (future)
Rust 1.65+ for Rust adapter (future)
Ruby 3.0+ for Ruby adapter

Quick Setup

# Clone the repository
cd /home/am/Proyectos/Oeditus/metastatic

# Install dependencies
mix deps.get

# Run all tests
mix test

# Generate documentation
mix docs

# Run static analysis (optional)
mix format --check-formatted

Project Structure

metastatic/
├── lib/
│   └── metastatic/
│       ├── ast.ex                  # Core MetaAST type definitions (3-tuple format)
│       ├── document.ex             # Document wrapper with metadata
│       ├── builder.ex              # High-level API
│       ├── adapter.ex              # Adapter behaviour
│       ├── validator.ex            # Conformance validation
│       ├── adapters/               # 5 language adapters
│       │   ├── python/             # Full Python support
│       │   ├── elixir/             # Full Elixir support
│       │   ├── erlang/             # Full Erlang support
│       │   ├── ruby/               # Full Ruby support
│       │   └── haskell/            # Full Haskell support
│       ├── supplemental/           # Cross-language construct support
│       │   ├── registry.ex         # Supplemental module registry
│       │   ├── transformer.ex      # Transformation helper
│       │   └── python/             # Pykka (actors), Asyncio
│       ├── semantic/               # Semantic metadata systems
│       │   ├── op_kind.ex          # Operation kind metadata (DB, HTTP, file, etc.)
│       │   └── enricher.ex         # Semantic enrichment for AST nodes
│       ├── analysis/               # Complete analysis suite
│       │   ├── purity.ex           # Purity analyzer
│       │   ├── complexity.ex       # Complexity analyzer (6 metrics)
│       │   ├── duplication.ex      # Code duplication detection
│       │   ├── dead_code.ex        # Dead code detection
│       │   ├── unused_variables.ex # Unused variable analysis
│       │   ├── control_flow.ex     # CFG generation
│       │   ├── taint.ex            # Taint analysis
│       │   ├── security.ex         # Security scanning
│       │   ├── smells.ex           # Code smell detection
│       │   └── business_logic/     # 32 language-agnostic analyzers
│       │       ├── callback_hell.ex
│       │       ├── sql_injection.ex
│       │       ├── xss_vulnerability.ex
│       │       └── ... (32 total)
│       └── mix/tasks/              # CLI tools (15+ tasks)
│           ├── metastatic.translate.ex
│           ├── metastatic.inspect.ex
│           ├── metastatic.purity_check.ex
│           ├── metastatic.complexity.ex
│           └── ... (15+ total)
├── test/
│   └── metastatic/                 # 1764 tests (1523 + 241 doctests)
│       ├── ast_test.exs
│       ├── adapters/               # Python, Elixir, Erlang, Ruby, Haskell
│       ├── supplemental/           # Supplemental modules
│       ├── analysis/               # All analyzers
│       └── mix/tasks/              # CLI tools
├── RESEARCH.md                     # Research and architecture
├── THEORETICAL_FOUNDATIONS.md      # Formal theory
├── IMPLEMENTATION_PLAN.md          # Detailed roadmap
├── GETTING_STARTED.md              # Developer guide (this file)
└── README.md                       # Project overview

Development Workflow

1. Understanding the Architecture

Before diving in, read these documents in order:

README.md - High-level overview and current status
RESEARCH.md - Deep dive into the MetaAST design decisions
THEORETICAL_FOUNDATIONS.md - Formal meta-modeling theory with proofs
IMPLEMENTATION_PLAN.md - Roadmap and milestones

2. Running Tests

# Run all tests (1764 tests: 1523 tests + 241 doctests)
mix test

# Run specific test file
mix test test/metastatic/ast_test.exs

# Run with verbose output
mix test --trace

# Generate documentation
mix docs

# Open documentation in browser
open doc/index.html

3. Working on a Feature

Follow this process:

# 1. Create a feature branch
git checkout -b feature/my-feature

# 2. Make your changes
# Edit files in lib/ and test/

# 3. Run tests frequently
mix test

# 4. Format code
mix format

# 5. Run static analysis
mix credo

# 6. Commit with descriptive messages
git commit -m "Add support for X in MetaAST"

# 7. Push and create PR
git push origin feature/my-feature

4. Code Style

We follow standard Elixir conventions:

Formatting: Use mix format (configured in .formatter.exs)
Documentation: All public functions must have @doc and examples
Typespecs: All public functions must have @spec
Tests: Aim for >90% coverage
Naming: Use descriptive names, avoid abbreviations

Example:

@doc """
Transform a Python binary operation to MetaAST.

## Examples

    iex> transform_binop(%{"_type" => "Add"})
    {:binary_op, :arithmetic, :+, left, right}

"""
@spec transform_binop(map()) :: {:ok, MetaAST.node()} | {:error, term()}
def transform_binop(%{"_type" => op_type, "left" => left, "right" => right}) do
  # Implementation
end

Common Tasks

Working with MetaAST

MetaAST uses a uniform 3-tuple format: {type_atom, keyword_meta, children_or_value}

alias Metastatic.{AST, Document, Validator}

# Create a MetaAST manually (3-tuple format)
ast = {:binary_op, [category: :arithmetic, operator: :+], [
  {:variable, [], "x"},
  {:literal, [subtype: :integer], 5}
]}

# Check conformance
AST.conforms?(ast)  # => true

# Extract variables
AST.variables(ast)  # => MapSet.new(["x"])

# Wrap in a document
doc = Document.new(ast, :python)

# Validate with metadata
{:ok, meta} = Validator.validate(doc)
meta.level  # => :core
meta.depth  # => 2
meta.variables  # => MapSet.new(["x"])

Using Language Adapters

Elixir Adapter

alias Metastatic.Adapters.Elixir, as: ElixirAdapter
alias Metastatic.Builder

# Parse Elixir source to MetaAST
source = "x + 5"
{:ok, doc} = Builder.from_source(source, ElixirAdapter)

# doc.ast uses the uniform 3-tuple format:
# {:binary_op, [category: :arithmetic, operator: :+], [
#   {:variable, [], "x"},
#   {:literal, [subtype: :integer], 5}
# ]}

# Convert back to Elixir source
{:ok, result} = Builder.to_source(doc)
# => "x + 5"

# Round-trip validation
{:ok, doc} = Builder.round_trip(source, ElixirAdapter)

Erlang Adapter

alias Metastatic.Adapters.Erlang, as: ErlangAdapter

# Parse Erlang source to MetaAST
source = "X + 5."
{:ok, doc} = Builder.from_source(source, ErlangAdapter)

# Same MetaAST structure as Elixir (only variable name differs)!
# {:binary_op, [category: :arithmetic, operator: :+], [
#   {:variable, [], "X"},
#   {:literal, [subtype: :integer], 5}
# ]}

# Convert to Erlang source
{:ok, result} = Builder.to_source(doc)
# => "X + 5"

Cross-Language Equivalence

# Parse Elixir
elixir_source = "x + 5"
{:ok, elixir_doc} = Builder.from_source(elixir_source, ElixirAdapter)

# Parse semantically equivalent Erlang
erlang_source = "X + 5."
{:ok, erlang_doc} = Builder.from_source(erlang_source, ErlangAdapter)

# Normalize variable names for comparison
elixir_vars = elixir_doc.ast |> normalize_vars()
erlang_vars = erlang_doc.ast |> normalize_vars()

# Same MetaAST structure!
assert elixir_vars == erlang_vars

Using Advanced Analyzers

Metastatic includes nine core static analysis capabilities:

Dead Code Detection

alias Metastatic.Analysis.DeadCode

# Detect code after return (3-tuple format)
ast = {:block, [], [
  {:early_return, [], [{:literal, [subtype: :integer], 42}]},
  {:function_call, [name: "print"], [{:literal, [subtype: :string], "hello"}]}  # unreachable!
]}
doc = Document.new(ast, :python)
{:ok, result} = DeadCode.analyze(doc)

result.has_dead_code?  # => true
result.issues          # => [{:code_after_return, :high, "Code after return statement", ...}]

# CLI usage
# mix metastatic.dead_code my_file.py
# mix metastatic.dead_code my_file.ex --format json

Unused Variables

alias Metastatic.Analysis.UnusedVariables

# Track variable usage (3-tuple format)
ast = {:block, [], [
  {:assignment, [], [{:variable, [], "x"}, {:literal, [subtype: :integer], 5}]},
  {:assignment, [], [{:variable, [], "y"}, {:literal, [subtype: :integer], 10}]},
  {:binary_op, [category: :arithmetic, operator: :+], [
    {:variable, [], "y"},
    {:literal, [subtype: :integer], 1}
  ]}
]}
doc = Document.new(ast, :elixir)
{:ok, result} = UnusedVariables.analyze(doc)

result.has_unused?  # => true
result.unused       # => MapSet.new(["x"])
result.defined      # => MapSet.new(["x", "y"])
result.used         # => MapSet.new(["y"])

# CLI usage
# mix metastatic.unused_vars my_file.ex
# mix metastatic.unused_vars my_file.py --ignore-underscore

Control Flow Graph

alias Metastatic.Analysis.ControlFlow

# Build CFG (3-tuple format)
ast = {:conditional, [], [
  {:variable, [], "x"},
  {:early_return, [], [{:literal, [subtype: :integer], 1}]},
  {:literal, [subtype: :integer], 2}
]}
doc = Document.new(ast, :python)
{:ok, result} = ControlFlow.analyze(doc)

result.node_count   # => 5
result.edge_count   # => 4
result.has_cycles?  # => false

# Export to DOT for Graphviz
dot_graph = result.to_dot()
# "digraph CFG {\n  0 [label=\"ENTRY\"];\n  ...

# Export to D3.js JSON
json_data = result.to_d3_json()
# %{nodes: [%{id: 0, label: "ENTRY", type: "entry", group: 1}, ...],
#   links: [%{source: 0, target: 1, label: nil, type: "normal"}, ...]}

# CLI usage
# mix metastatic.control_flow my_file.py --format dot
# mix metastatic.control_flow my_file.ex --format d3 --output cfg.json

Taint Analysis

alias Metastatic.Analysis.Taint

# Detect taint vulnerabilities (3-tuple format)
ast = {:function_call, [name: "eval"], [
  {:function_call, [name: "input"], []}  # Dangerous: eval(input())
]}
doc = Document.new(ast, :python)
{:ok, result} = Taint.analyze(doc)

result.has_vulnerabilities?  # => true
result.vulnerabilities       # => [{:code_injection, "eval called with untrusted source", :high}]

# CLI usage
# mix metastatic.taint_check my_file.py
# mix metastatic.taint_check my_file.ex --format json

Security Scanning

alias Metastatic.Analysis.Security

# Detect security issues (3-tuple format)
ast = {:assignment, [], [{:variable, [], "password"}, {:literal, [subtype: :string], "admin123"}]}
doc = Document.new(ast, :python)
{:ok, result} = Security.analyze(doc)

result.has_vulnerabilities?  # => true
vuln = hd(result.vulnerabilities)
vuln.type      # => :hardcoded_secret
vuln.severity  # => :high
vuln.cwe       # => "CWE-798"
vuln.location  # => "Variable: password"

# CLI usage
# mix metastatic.security_scan my_file.py
# mix metastatic.security_scan my_file.ex --format json

Code Smell Detection

alias Metastatic.Analysis.Smells

# Detect code smells (3-tuple format)
ast = {:block, [], [
  {:conditional, [], [{:variable, [], "a"}, {:literal, [subtype: :integer], 1}, {:literal, [subtype: :integer], 2}]},
  {:conditional, [], [{:variable, [], "b"}, {:literal, [subtype: :integer], 3}, {:literal, [subtype: :integer], 4}]},
  # ... many more statements creating long function and deep nesting
]}
doc = Document.new(ast, :python)
{:ok, result} = Smells.analyze(doc)

result.has_smells?  # => true (if thresholds exceeded)
result.smells       # => [:long_function, :deep_nesting] (if detected)
result.severity     # => :medium or :high

# CLI usage
# mix metastatic.code_smells my_file.py
# mix metastatic.code_smells my_file.ex --format detailed

Business Logic Analyzers (32 analyzers)

Metastatic includes 32 language-agnostic business logic analyzers that detect anti-patterns across all supported languages. These include:

Security (CWE Top 25 coverage):

SQLInjection (CWE-89), XSSVulnerability (CWE-79), PathTraversal (CWE-22)
MissingAuthorization (CWE-862), SSRFVulnerability (CWE-918)
SensitiveDataExposure (CWE-200), UnrestrictedFileUpload (CWE-434)
MissingAuthentication (CWE-306), MissingCSRFProtection (CWE-352)
IncorrectAuthorization (CWE-863), ImproperInputValidation (CWE-20)
InsecureDirectObjectReference (CWE-639)

Anti-patterns:

CallbackHell, MissingErrorHandling, SilentErrorCase, SwallowingException
HardcodedValue, NPlusOneQuery, InefficientFilter, UnmanagedTask
BlockingInPlug, SyncOverAsync, DirectStructUpdate, MissingPreload
InlineJavascript, MissingThrottle, TOCTOU, and more

alias Metastatic.Analysis.Runner
alias Metastatic.Document

# Run all business logic analyzers
ast = {:function_call, [name: "execute"], [
  {:binary_op, [category: :arithmetic, operator: :+], [
    {:literal, [subtype: :string], "SELECT * FROM users WHERE id = "},
    {:variable, [], "user_input"}
  ]}
]}
doc = Document.new(ast, :python)

{:ok, issues} = Runner.run(doc)
# Returns SQLInjection warning about string concatenation in SQL

Adding a New Language Adapter

See existing Elixir and Erlang adapters as reference implementations.

Adding a New Mutator

Create mutator module: lib/metastatic/mutators/my_mutator.ex
Implement mutation logic: Use Macro.postwalk/2
Add tests: Test on multiple languages
Document: Include examples

Adding Test Fixtures

# Create fixture directory
mkdir -p test/fixtures/elixir/

# Add source file
echo 'x + y' > test/fixtures/elixir/simple_add.ex

# Add expected MetaAST
cat > test/fixtures/elixir/expected/simple_add.exs << 'EOF'
{:binary_op, :arithmetic, :+, {:variable, "x"}, {:variable, "y"}}
EOF

Testing Philosophy

Unit Tests

Test individual transformations and functions:

test "transforms Elixir addition to MetaAST" do
  elixir_ast = {:+, [], [{:x, [], nil}, 5]}
  {:ok, meta_ast} = Metastatic.Adapters.Elixir.ToMeta.transform(elixir_ast)
  # 3-tuple format: {type, keyword_meta, children_or_value}
  assert {:binary_op, [category: :arithmetic, operator: :+], [
    {:variable, [], "x"},
    {:literal, [subtype: :integer], 5}
  ]} = meta_ast
end

Integration Tests

Test full round-trips:

test "round-trip Elixir source through MetaAST" do
  source = "x + 5"
  alias Metastatic.Adapters.Elixir, as: ElixirAdapter
  {:ok, doc} = Builder.from_source(source, ElixirAdapter)
  {:ok, result} = Builder.to_source(doc)
  assert result == source
end

Property Tests

Use StreamData for property-based testing:

property "all arithmetic mutations are valid" do
  check all ast <- ast_generator() do
    mutations = Mutator.arithmetic_inverse(ast)
    assert Enum.all?(mutations, &valid_ast?/1)
  end
end

Debugging Tips

Inspecting ASTs

# In IEx
iex> alias Metastatic.Adapters.Elixir, as: ElixirAdapter
iex> source = "x + 5"
iex> {:ok, doc} = Metastatic.Builder.from_source(source, ElixirAdapter)
iex> IO.inspect(doc.ast, label: "MetaAST")
iex> IO.inspect(doc.metadata, label: "Metadata")

Using IEx for Development

# Start IEx with project loaded
iex -S mix

# Reload changed modules
iex> recompile()

# Run specific test
iex> ExUnit.run()

Testing Adapters

# Test Elixir adapter
mix test test/metastatic/adapters/elixir_test.exs

# Test Erlang adapter
mix test test/metastatic/adapters/erlang_test.exs

# Test specific feature
mix test test/metastatic/adapters/elixir_test.exs:45

Documentation

Writing Docs

All public functions must have:

@doc """
Brief one-line description.

Longer explanation if needed. Explain what the function does,
not how it does it.

## Examples

    iex> MyModule.my_function(arg)
    expected_result

## Options

- `:option1` - Description
- `:option2` - Description

"""
@spec my_function(arg_type()) :: return_type()
def my_function(arg) do
  # Implementation
end

Generating Docs

# Generate HTML documentation
mix docs

# Open in browser
open doc/index.html

Performance Considerations

Profiling

# Use :fprof for profiling
alias Metastatic.Adapters.Elixir, as: ElixirAdapter
source = "x + 5"
:fprof.apply(&Metastatic.Builder.from_source/2, [source, ElixirAdapter])
:fprof.profile()
:fprof.analyse()

Benchmarking

# Use Benchee for benchmarking
alias Metastatic.Adapters.{Elixir, Erlang}
source_ex = "x + 5"
source_erl = "X + 5."

Benchee.run(%{
  "parse elixir" => fn -> Metastatic.Builder.from_source(source_ex, Elixir) end,
  "parse erlang" => fn -> Metastatic.Builder.from_source(source_erl, Erlang) end
})

Troubleshooting

Common Issues

Issue: Elixir parse error

Error: Code.string_to_quoted/1 failed with syntax error

Solution: Ensure Elixir source is syntactically valid

Issue: Erlang parse error

Error: :erl_parse.parse_exprs failed

Solution: Ensure Erlang expressions end with a period (.)

Issue: Tests failing after changes

Error: test/metastatic/adapters/... failed

Solution: Check MetaAST structure matches expected format; run mix format to ensure consistent formatting

Getting Help

Issues: Open a GitHub issue for bugs or feature requests
Discussions: Use GitHub Discussions for questions
Slack: Join #metastatic channel (internal)
Documentation: Check RESEARCH.md and IMPLEMENTATION_PLAN.md

Contributing Checklist

Before submitting a PR:

[ ] Code is formatted (mix format)
[ ] Tests pass (mix test)
[ ] Coverage > 90% for new code
[ ] Credo passes (mix credo --strict)
[ ] Dialyzer passes (mix dialyzer)
[ ] Documentation added/updated
[ ] CHANGELOG.md updated
[ ] Commit messages are descriptive

Next Steps

Read the research: Start with RESEARCH.md to understand the "why"
Pick a task: Check IMPLEMENTATION_PLAN.md for current priorities
Set up environment: Install required runtimes
Run tests: Make sure everything works
Start coding: Pick an issue or feature from the roadmap

Welcome aboard!

← Previous Page Metastatic

Next Page → Theoretical Foundations