Snakepit.WorkerProfile.Thread (Snakepit v0.8.7)

Multi-threaded worker profile (Python 3.13+ optimized).

Each worker is a Python process with a thread pool, providing:

• Shared memory: Zero-copy data sharing within worker
• CPU parallelism: True multi-threading without GIL (Python 3.13+)
• Lower memory: One interpreter vs many
• High throughput: Optimal for CPU-bound tasks

Configuration

config :snakepit,
  pools: [
    %{
      name: :hpc_pool,
      worker_profile: :thread,
      pool_size: 4,                    # 4 processes
      threads_per_worker: 16,          # 16 threads each = 64 total capacity
      adapter_module: Snakepit.Adapters.GRPCPython,
      adapter_args: ["--mode", "threaded", "--max-workers", "16"],
      adapter_env: [
        # Allow multi-threading in libraries
        {"OPENBLAS_NUM_THREADS", "16"},
        {"OMP_NUM_THREADS", "16"}
      ],
      worker_ttl: {3600, :seconds},    # Recycle hourly
      worker_max_requests: 1000,       # Or after 1000 requests
      thread_safety_checks: true       # Enable runtime validation
    }
  ]

Requirements

• Python 3.13+ for optimal performance (free-threading)
• Thread-safe Python adapters
• Thread-safe ML libraries (NumPy, PyTorch, etc.)

Status

Thread profile is fully supported when paired with Python 3.13+ and thread-safe adapters.

Implementation Notes

The thread profile:

1. Starts fewer Python processes (4-16 instead of 100+)
2. Runs a ThreadPoolExecutor per worker process
3. Tracks per-worker capacity via threads_per_worker for pool scheduling
4. Supports optional CapacityStore telemetry with capacity_strategy: :hybrid
5. Allows concurrent requests to the same worker via HTTP/2 multiplexing