Snakepit.WorkerProfile.Process (Snakepit v0.6.10)

View Source

Multi-process worker profile (default).

Each worker is a separate OS process, providing:

  • Process isolation: Crashes don't affect other workers
  • GIL compatibility: Works with all Python versions
  • High concurrency: Optimal for 100+ workers with I/O-bound tasks

This is the default profile and maintains 100% backward compatibility with Snakepit v0.5.x configurations.

Configuration

config :snakepit,
  pools: [
    %{
      name: :default,
      worker_profile: :process,  # Explicit (or omit for default)
      pool_size: 100,
      adapter_module: Snakepit.Adapters.GRPCPython,
      adapter_env: [
        {"OPENBLAS_NUM_THREADS", "1"},
        {"OMP_NUM_THREADS", "1"}
      ]
    }
  ]

Implementation Details

  • Each worker runs a single-threaded Python process
  • Workers are single-capacity (one request at a time)
  • Environment variables enforce single-threading in scientific libraries
  • Startup is batched to prevent resource exhaustion