Snakepit.WorkerProfile.Thread (Snakepit v0.6.10)

Multi-threaded worker profile (Python 3.13+ optimized).

Each worker is a Python process with a thread pool, providing:

• Shared memory: Zero-copy data sharing within worker
• CPU parallelism: True multi-threading without GIL (Python 3.13+)
• Lower memory: One interpreter vs many
• High throughput: Optimal for CPU-bound tasks

Configuration

config :snakepit,
  pools: [
    %{
      name: :hpc_pool,
      worker_profile: :thread,
      pool_size: 4,                    # 4 processes
      threads_per_worker: 16,          # 16 threads each = 64 total capacity
      adapter_module: Snakepit.Adapters.GRPCPython,
      adapter_args: ["--mode", "threaded", "--max-workers", "16"],
      adapter_env: [
        # Allow multi-threading in libraries
        {"OPENBLAS_NUM_THREADS", "16"},
        {"OMP_NUM_THREADS", "16"}
      ],
      worker_ttl: {3600, :seconds},    # Recycle hourly
      worker_max_requests: 1000,       # Or after 1000 requests
      thread_safety_checks: true       # Enable runtime validation
    }
  ]

Requirements

• Python 3.13+ for optimal performance (free-threading)
• Thread-safe Python adapters
• Thread-safe ML libraries (NumPy, PyTorch, etc.)

Status

Phase 1 (Current): Stub implementation that returns :not_implemented

Full implementation planned for Phase 2-3 of v0.6.0 development.

Implementation Notes

The thread profile will:

1. Start fewer Python processes (4-16 instead of 100+)
2. Each process runs a ThreadPoolExecutor with N threads
3. Track in-flight requests per worker for capacity management
4. Support concurrent requests to same worker via HTTP/2 multiplexing