Macula HTTP/3 Mesh - Documentation Root

A distributed mesh networking platform for BEAM, built on HTTP/3/QUIC

Vision

Build a unique, standards-based distributed networking layer for Erlang/Elixir applications that:

• Uses HTTP/3 (QUIC) for NAT-friendly, firewall-friendly transport
• Forms self-organizing mesh topologies at the edge
• Provides pub/sub and RPC primitives
• Scales to thousands of nodes
• Supports multi-tenancy and realm isolation
• Delivers "Wow! How do they do it?" factor

Target Use Cases:

• Edge-first IoT networks
• Decentralized energy trading platforms
• Multi-tenant SaaS applications
• Partner data exchange networks
• Hybrid cloud-edge systems

Documentation Index

Core Architecture Documents

1. Technical Roadmap ⭐ START HERE

20-week implementation plan with detailed technical specifications

Contents:

• Understanding QUIC and HTTP/3 (protocol deep dive)
• QUIC/HTTP/3 libraries for BEAM (comparison matrix)
• Complete 5-layer architecture
• Phase-by-phase implementation (Weeks 1-20)
• Code examples for each phase
• Success criteria and benchmarks
• Technical deep dives (QUIC vs TCP, SWIM gossip, Kademlia DHT)

Who should read this: Engineers implementing the platform, technical leads evaluating feasibility

Key decisions documented:

• QUIC library choice: quicer (Erlang NIF wrapper for MsQuic)
• Topology: k-regular graph with SWIM gossip
• Routing: Kademlia DHT (O(log N) lookups)
• NAT traversal: STUN/ICE + UDP hole punching

2. C4 Architecture Diagrams

Multi-level architecture visualization using C4 model

Contents:

• Level 1 - System Context: Ecosystem view (developers, nodes, infrastructure)
• Level 2 - Container: Technology stack, BEAM VM internals
• Level 3 - Component: Mesh Services and Protocol Layer components
• Level 4 - Code: Detailed macula_connection GenServer implementation
• Supplementary - Deployment: Physical deployment scenarios

Who should read this: Architects, new team members, stakeholders wanting visual overview

Diagram formats: ASCII (easy to view in terminal, version control friendly)

3. Isolation Mechanisms

Multi-tenancy, realms, and cross-realm communication

Contents:

• Realm concept and namespacing
• Identity layer (node-level realm membership)
• Topic namespacing and validation
• Routing table partitioning by realm
• Three cross-realm communication models:
  • Model A: Gateway Nodes (recommended)
  • Model B: Federation Protocol
  • Model C: Strict Isolation
• Protocol-level realm support
• SWIM membership per realm
• Pub/sub with realm scoping
• Certificate-based security
• Use cases (SaaS, energy markets, IoT, partners)
• Implementation roadmap (Weeks 21-27)

Who should read this: SaaS platform engineers, security architects, multi-tenant deployment teams

Key features:

• Isolation by default
• Policy-based gateways for controlled data sharing
• Certificate validation with realm in SAN
• ACL enforcement
• Audit logging

4. Module Dependencies and Architecture ⭐

How the pieces fit together: From QUIC to your application

Contents:

• Layered architecture diagram - Visual representation of all components
• Module responsibilities - What each component does and why
• Message flow examples - Follow an RPC call and pub/sub message through the stack
• The philosophy - Gateway as "dumb transport" vs RPC/PubSub as "smart orchestration"
• Current implementation status - What works, what's in progress, what's planned

Who should read this: All developers working on Macula, architects understanding the system

Key insight: Macula separates transport concerns (gateway) from business logic (RPC/PubSub servers), similar to how nginx handles HTTP routing while your application handles business logic. This separation enables independent evolution, testing, and scaling of each layer.

Why this matters: Understanding the layered architecture prevents confusion about where functionality belongs. For example, topic pattern matching belongs in macula_pubsub_server (business logic), not macula_gateway (transport).

Quick Start Guides

5. Quick Start Guide 🚀 ⚠️ TODO

Get a 3-node mesh running in 15 minutes

Planned contents:

• Prerequisites (Erlang/OTP 26+, quicer installation)
• Download and build Macula
• Start first node
• Start and join second node
• Start and join third node
• Send first pub/sub message
• Make first RPC call
• Verify mesh topology
• Common troubleshooting

Target audience: Developers wanting hands-on experience

6. Hello World Tutorial 📚 ⚠️ TODO

Build your first Macula application

Planned contents:

• Create new Elixir/Mix project
• Add Macula dependency
• Configure node identity and realm
• Implement simple pub/sub chat application
• Deploy across multiple nodes
• Add RPC endpoint (echo service)
• Monitor with Observer

Target audience: Application developers new to Macula

API and Protocol Specifications

7. Wire Protocol Specification 📋 ⚠️ TODO

Complete wire protocol documentation

Planned contents:

• Packet format and framing
• Message types (handshake, heartbeat, send, pub, sub, rpc, etc.)
• Encoding/decoding rules (Erlang term format)
• Handshake protocol flow
• Stream multiplexing
• Error handling
• Protocol versioning
• Compatibility matrix

Target audience: Protocol implementers, interoperability teams

8. API Reference 📖 ⚠️ TODO

Complete Erlang/Elixir API documentation

Planned contents:

• macula:start/1 - Start node
• macula:connect/2 - Connect to peer
• macula:publish/2,3 - Publish message
• macula:subscribe/1,2 - Subscribe to topic
• macula:call/3,4 - RPC call
• macula:register/2 - Register RPC endpoint
• Gateway APIs
• Policy configuration APIs
• Metrics and monitoring APIs

Target audience: Application developers

Advanced Topics

9. NAT Traversal Deep Dive 🌐 ⚠️ TODO

How Macula works behind NATs and firewalls

Planned contents:

• NAT types and challenges (full cone, symmetric, etc.)
• STUN protocol for public IP discovery
• ICE protocol for connectivity establishment
• UDP hole punching techniques
• TURN relay fallback
• Connection migration on IP change
• Mobile/cellular network considerations
• Enterprise firewall traversal

Target audience: Network engineers, DevOps, deployment teams

10. Security Model 🔒 ⚠️ TODO

Comprehensive security architecture

Planned contents:

• Threat model
• TLS 1.3 integration with QUIC
• Certificate-based node authentication
• Realm isolation via certificates
• ACL enforcement
• Message signing and verification
• Audit logging
• Denial-of-service protection
• Rate limiting
• Security best practices
• Penetration testing results

Target audience: Security teams, compliance officers, architects

11. Performance Tuning Guide ⚡ ⚠️ TODO

Optimize for throughput and latency

Planned contents:

• Benchmarking methodology
• OS-level tuning (UDP buffers, file descriptors)
• BEAM VM tuning (schedulers, memory)
• QUIC connection parameters
• Stream multiplexing configuration
• Routing table optimization
• SWIM protocol tuning
• Gateway throughput optimization
• Monitoring and profiling tools
• Load testing scenarios

Target audience: Performance engineers, SREs

12. Observability Guide 📊 ⚠️ TODO

Monitor, trace, and debug Macula networks

Planned contents:

• Prometheus metrics (all available metrics)
• Grafana dashboards (pre-built templates)
• OpenTelemetry tracing integration
• Log aggregation (structured logging)
• Mesh topology visualization
• Real-time message flow visualization
• Health checks and alerts
• Debugging tools (observer, recon, etc.)
• Common issues and diagnostics

Target audience: SREs, DevOps, operations teams

13. Deployment Patterns 🚀 ⚠️ TODO

Production deployment architectures

Planned contents:

• Single-region mesh
• Multi-region with gateways
• Hybrid cloud-edge
• Kubernetes deployment (Helm charts)
• Docker Compose examples
• Bare metal / VM deployment
• DNS/discovery configuration
• Load balancing strategies
• High availability patterns
• Disaster recovery
• Migration strategies (zero-downtime updates)

Target audience: DevOps, platform engineers, architects

14. Gateway Operations Manual 🌉 ⚠️ TODO

Deploy and operate gateway nodes

Planned contents:

• Gateway node requirements
• Policy configuration (YAML/JSON schemas)
• Topic translation setup
• Rate limiting configuration
• Audit log management
• Certificate management for gateways
• High availability setup (active-active)
• Monitoring gateway health
• Troubleshooting cross-realm issues
• Performance optimization
• Security hardening

Target audience: Gateway operators, platform admins

Comparisons and Design Decisions

15. Macula vs Distributed Erlang ⭐

Does Macula augment or replace Distributed Erlang?

Answer: Macula replaces Distributed Erlang for internet-scale, multi-tenant scenarios.

Contents:

• The key differences - Security, network assumptions, multi-tenancy, scalability
• Architectural comparison - Cookie-based mesh vs TLS-based selective connectivity
• When to use each - Tightly-coupled clusters (disterl) vs loosely-coupled services (Macula)
• Code comparison - Side-by-side examples showing explicit vs implicit communication
• Hybrid approach - Can you use both? (Yes, with gateways between data centers)
• Migration path - Moving from disterl to Macula incrementally

Who should read this: Teams familiar with Distributed Erlang, architects deciding between approaches

Key takeaway: Distributed Erlang excels at <50 node, single-datacenter, fully-trusted clusters. Macula excels at 100s-1000s of nodes across the internet with realm isolation and NAT traversal. They solve different problems.

16. Comparison with WAMP/Bondy 🔄 ⚠️ TODO

Why build Macula when WAMP exists?

Planned contents:

• WAMP strengths and weaknesses
• Why WAMP over WebSocket doesn't work well for edge
• NAT traversal comparison
• Bondy clustering (Partisan) vs Macula mesh
• Protocol overhead comparison
• Latency and throughput benchmarks
• When to use WAMP/Bondy vs Macula
• Migration path from WAMP to Macula
• Interoperability (WAMP compatibility layer)

Target audience: Teams familiar with WAMP, decision makers

17. Comparison with libp2p 🔄 ⚠️ TODO

Macula vs libp2p (IPFS networking stack)

Planned contents:

• libp2p architecture overview
• BEAM libp2p implementations (ex_libp2p)
• Why not use libp2p?
  • Complexity
  • Maturity in BEAM ecosystem
  • Distributed Erlang incompatibility
• Protocol comparison (QUIC, GossipSub, Kademlia)
• Use case fit analysis
• Performance comparison

Target audience: P2P networking engineers, architects

18. Design Decision Log 📝 ⚠️ TODO

Why we made the choices we did

Planned contents:

• ADR 001: Why QUIC instead of TCP?
• ADR 002: Why quicer (MsQuic) instead of pure Erlang?
• ADR 003: Why Kademlia instead of Chord/Pastry?
• ADR 004: Why SWIM instead of Raft for membership?
• ADR 005: Why k-regular graph instead of full mesh?
• ADR 006: Why gateway pattern for cross-realm?
• ADR 007: Why certificate-based auth instead of API keys?
• ADR 008: Why UDP hole punching instead of TURN-only?

Format: Architecture Decision Records (ADRs)

Target audience: Architects, long-term maintainers

Reference Materials

19. Glossary 📖 ⚠️ TODO

Terms and definitions

Planned contents:

• QUIC, HTTP/3, UDP, DTLS, TLS 1.3
• Mesh, topology, k-regular graph
• SWIM, gossip, failure detection
• Kademlia, DHT, XOR distance
• Realm, tenant, gateway
• Pub/sub, RPC, stream
• NAT, STUN, ICE, TURN, hole punching
• Connection migration, 0-RTT
• ACL, policy, audit

Target audience: Everyone (reference)

20. FAQ ❓ ⚠️ TODO

Frequently asked questions

Planned contents:

• What is Macula?
• Why HTTP/3 instead of traditional distributed Erlang?
• Can it work behind NAT?
• How many nodes can it scale to?
• What's the latency overhead?
• Is it production-ready?
• How does it compare to X? (where X = WAMP, libp2p, Partisan, gRPC, etc.)
• Can I use it with Phoenix/LiveView?
• Does it work in Kubernetes?
• What license is it?

Target audience: Everyone (first questions)

21. Troubleshooting Guide 🔧 ⚠️ TODO

Common issues and solutions

Planned contents:

• Nodes can't discover each other
• Connection timeout / handshake failure
• NAT traversal failures
• Certificate validation errors
• SWIM membership flapping
• Routing table inconsistencies
• Gateway policy denials
• Performance issues (high latency, low throughput)
• Memory leaks
• Crash dumps analysis

Format: Problem → Diagnosis → Solution

Target audience: Operations, support teams

Contributing and Community

22. Contributing Guide 🤝 ⚠️ TODO

How to contribute to Macula

Planned contents:

• Code of conduct
• Development setup
• Testing requirements (unit, integration, property-based)
• Code style guide (Erlang/Elixir conventions)
• Documentation requirements
• Pull request process
• Release process
• Community channels (Discord, mailing list, etc.)
• Roadmap and feature requests

Target audience: Contributors, open source community

Document Status

Priority Levels:

• P0: Must have before any code (architecture)
• P1: Required for MVP release
• P2: Important for production use
• P3: Nice to have

Reading Paths

For Evaluators (Decision Makers)

1. This document (overview)
2. Technical Roadmap - Sections: Vision, Architecture Overview, Timeline
3. C4 Diagrams - Level 1 and Level 2
4. Isolation Mechanisms - Use Cases section

Time: ~1 hour

For Architects (System Design)

1. Technical Roadmap - Complete read
2. C4 Diagrams - All levels
3. Module Dependencies - Understand the layers
4. Isolation Mechanisms - Complete read
5. Macula vs Distributed Erlang - Architectural trade-offs
6. NAT Traversal Deep Dive ⚠️ TODO
7. Security Model ⚠️ TODO
8. Design Decision Log ⚠️ TODO

Time: ~5 hours

For Implementers (Engineers)

1. Module Dependencies - Understand where code belongs
2. Technical Roadmap - Focus on code examples
3. Wire Protocol Spec ⚠️ TODO
4. API Reference ⚠️ TODO
5. Quick Start Guide ⚠️ TODO
6. Hello World Tutorial ⚠️ TODO

Time: ~3 hours + hands-on

For Operators (DevOps/SRE)

1. Quick Start Guide ⚠️ TODO
2. Deployment Patterns ⚠️ TODO
3. Observability Guide ⚠️ TODO
4. Performance Tuning ⚠️ TODO
5. Troubleshooting Guide ⚠️ TODO
6. Gateway Operations ⚠️ TODO

Time: ~2 hours + practice

For Security Teams

1. Security Model ⚠️ TODO
2. Isolation Mechanisms - Security sections
3. NAT Traversal Deep Dive ⚠️ TODO
4. Gateway Operations ⚠️ TODO - Security hardening

Time: ~3 hours

Additional Topics to Document

Based on the comprehensive nature of this project, here are additional topics that should be documented:

21. Testing Strategy 🧪

Comprehensive testing approach

Contents:

• Unit testing (EUnit, ExUnit)
• Property-based testing (PropEr, StreamData)
• Integration testing (multi-node scenarios)
• Chaos engineering (partition testing, node crashes)
• Load testing (Tsung, k6)
• Security testing (penetration testing)
• Fuzz testing (protocol fuzzing)
• Continuous integration setup

22. Migration Guide 🔄

Moving from other systems to Macula

Contents:

• Migrating from WAMP/Bondy
• Migrating from distributed Erlang
• Migrating from RabbitMQ/Kafka
• Migrating from gRPC
• Co-existence strategies (gradual migration)
• Data migration patterns
• Rollback procedures

23. Scaling Patterns 📈

How to scale from 10 to 10,000 nodes

Contents:

• Topology evolution (full mesh → k-regular → hierarchical)
• Region sharding
• DHT bucket optimization
• SWIM tuning for large networks
• Gateway scaling (horizontal)
• Database scaling (if persistence layer added)
• Cost analysis at scale

24. Protocol Evolution 🔬

Versioning and backward compatibility

Contents:

• Protocol version negotiation
• Backward compatibility guarantees
• Deprecation policy
• Feature flags
• Upgrade paths (rolling upgrades)
• Breaking changes process

25. Mobile and Browser Support 📱

Extending Macula to constrained environments

Contents:

• WebAssembly BEAM (lumen, AtomVM)
• Browser WebTransport (QUIC in browsers)
• React Native integration
• Mobile battery optimization
• Offline-first patterns
• Connection resumption on network change

26. Plugin Architecture 🔌

Extend Macula with custom behaviors

Contents:

• Hook system for message interception
• Custom discovery plugins (Consul, etcd)
• Custom transport plugins (WebRTC, Bluetooth)
• Custom routing strategies
• Custom serialization formats
• Plugin development guide

27. Cost Analysis 💰

TCO comparison vs alternatives

Contents:

• Infrastructure costs (vs cloud load balancers)
• Bandwidth costs (UDP vs TCP, compression)
• Operational costs (automation, monitoring)
• Development costs (time to market)
• Licensing costs (vs commercial solutions)
• ROI calculator

28. Regulatory Compliance 📜

GDPR, HIPAA, SOC2 considerations

Contents:

• Data residency (realm isolation for regions)
• Right to be forgotten (message expiry)
• Audit logging requirements
• Encryption at rest/in transit
• Access controls
• Compliance checklists

29. Interoperability 🔗

Connect Macula to other systems

Contents:

• WAMP compatibility layer (adapter)
• gRPC bridge
• REST/GraphQL gateway
• MQTT bridge (for IoT)
• Kafka/RabbitMQ connectors
• Database change data capture (CDC)

30. Case Studies 📚

Real-world deployments (when available)

Contents:

• Example Platform energy trading platform
• Industrial IoT deployment
• Multi-tenant SaaS platform
• Gaming backend
• Financial data mesh
• Lessons learned, metrics, testimonials

Contributing to Documentation

Documentation is as important as code! To contribute:

1. Choose a TODO document from the status table above
2. Follow the template provided in the document outline
3. Include code examples (working, tested code)
4. Add diagrams (ASCII art for version control friendliness)
5. Get review from at least one core team member
6. Update this index when document is complete

Documentation Standards:

• Use Markdown (.md)
• ASCII diagrams (box drawing characters: ┌─┐│└┘)
• Code examples must be syntactically correct
• Include both Erlang and Elixir examples where applicable
• Cross-reference related documents
• Keep language clear and concise (avoid jargon, or define it)

License

[To be determined - likely Apache 2.0 or MIT]

Contact and Community

• GitHub: macula-io/macula ⚠️ TODO
• Discord: Join our Discord ⚠️ TODO
• Mailing List: macula-dev@googlegroups.com ⚠️ TODO
• Twitter: @MaculaMesh ⚠️ TODO

Last Updated: 2025-01-08 Maintainers: Macula Core Team Status: Living Document (updated as project evolves)