Snakepit.Pool.Worker.Starter (Snakepit v0.13.0)

Copy Markdown View Source

Supervisor wrapper for individual workers that provides automatic restart capability.

This module implements the "Permanent Wrapper" pattern for managing workers that control external OS processes (Python gRPC servers).

Architecture Decision

See: docs/architecture/adr-001-worker-starter-supervision-pattern.md for detailed rationale, alternatives considered, and trade-offs.

Why This Pattern?

TL;DR: Workers manage external Python processes, not just Elixir state. This pattern provides:

  • Automatic restart without Pool intervention
  • Atomic resource cleanup (worker + Python process)
  • Future extensibility for per-worker resources

Trade-off: Extra process (~1KB) per worker for better encapsulation.

Architecture

DynamicSupervisor (WorkerSupervisor)
 Worker.Starter (Supervisor, :permanent)
     GRPCWorker (GenServer, :transient)
         Port  Python grpc_server.py

Lifecycle

When GRPCWorker crashes:

  1. Worker.Starter detects crash via :one_for_one strategy
  2. Worker.Starter automatically restarts GRPCWorker
  3. Pool notified via :DOWN but doesn't manage restart
  4. New GRPCWorker spawns new Python process and re-registers

When Worker.Starter terminates:

  1. GRPCWorker receives shutdown signal
  2. GRPCWorker.terminate sends SIGTERM to Python
  3. Python process exits gracefully
  4. Worker.Starter confirms all children stopped
  5. Clean atomic shutdown

This decouples Pool (availability management) from Worker lifecycle (crash/restart).

  • Issue #2: Community feedback questioning this complexity
  • ADR-001: Full architecture decision record with alternatives
  • External Process Design: docs/20251007_external_process_supervision_design.md

Summary

Functions

Returns a specification to start this module under a supervisor.

Starts a worker starter supervisor.

Returns a via tuple for this starter supervisor.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

start_link(worker_id)

Starts a worker starter supervisor.

Parameters

  • worker_id - Unique identifier for the worker

via_name(worker_id)

Returns a via tuple for this starter supervisor.