Poolex (poolex v1.5.0-rc.0)

Usage

In the most typical use of Poolex, you only need to start pool of workers as a child of your application.

children = [
  {Poolex,
    pool_id: :worker_pool,
    worker_module: SomeWorker,
    workers_count: 5}
]

Supervisor.start_link(children, strategy: :one_for_one)

Then you can execute any code on the workers with run/3:

Poolex.run(:worker_pool, &(is_pid?(&1)), checkout_timeout: 1_000)
{:ok, true}

For more information see Getting Started

Summary

Types

pool_id()

Any valid GenServer's name. It may be an atom like :some_pool or a tuple {:via, Registry, {MyApp.Registry, "pool"} if you want to use Registry.

poolex_option()

run_option()

worker()

Process id of worker.

Functions

acquire(pool_id, options \\ [])

Acquires a worker from the pool for manual management.

add_idle_workers!(pool_id, workers_count)

Adds some idle workers to existing pool.

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

release(pool_id, worker_pid)

Releases a manually acquired worker back to the pool.

remove_idle_workers!(pool_id, workers_count)

Removes some idle workers from existing pool. If the number of workers to remove is greater than the number of idle workers, all idle workers will be removed.

run(pool_id, fun, options \\ [])

The main function for working with the pool.

start(opts)

Starts a Poolex process without links (outside of a supervision tree).

start_link(opts)

Starts a Poolex process linked to the current process.

Types

pool_id()

@type pool_id() :: GenServer.name() | pid()

Any valid GenServer's name. It may be an atom like :some_pool or a tuple {:via, Registry, {MyApp.Registry, "pool"} if you want to use Registry.

poolex_option()

@type poolex_option() ::
  {:busy_workers_impl, module()}
  | {:failed_workers_retry_interval, timeout()}
  | {:idle_overflowed_workers_impl, module()}
  | {:idle_workers_impl, module()}
  | {:max_overflow, non_neg_integer()}
  | {:pool_id, pool_id()}
  | {:pool_size_metrics, boolean()}
  | {:waiting_callers_impl, module()}
  | {:worker_args, [any()]}
  | {:worker_module, module()}
  | {:worker_shutdown_delay, timeout()}
  | {:worker_start_fun, atom()}
  | {:workers_count, non_neg_integer()}

Option	Description	Example	Default value
`busy_workers_impl`	Module that describes how to work with busy workers	`SomeBusyWorkersImpl`	`Poolex.Workers.Impl.List`
`failed_workers_retry_interval`	Interval in milliseconds between retry attempts for failed workers	`5_000`	`1_000`
`idle_workers_impl`	Module that describes how to work with idle workers	`SomeIdleWorkersImpl`	`Poolex.Workers.Impl.List`
`idle_overflowed_workers_impl`	Module that describes how to work with idle overflowed workers	`SomeIdleOverflowedWorkersImpl`	`Poolex.Workers.Impl.List`
`max_overflow`	How many workers can be created over the limit	`2`	`0`
`worker_shutdown_delay`	Delay (ms) before shutting down overflow worker after release	`5000`	`0`
`pool_id`	Identifier by which you will access the pool	`:my_pool`	`worker_module` value
`pool_size_metrics`	Whether to dispatch pool size metrics	`true`	`false`
`waiting_callers_impl`	Module that describes how to work with callers queue	`WaitingCallersImpl`	`Poolex.Callers.Impl.ErlangQueue`
`worker_args`	List of arguments passed to the start function	`[:gg, "wp"]`	`[]`
`worker_module`	Name of module that implements our worker	`MyApp.Worker`	option is required
`worker_start_fun`	Name of the function that starts the worker	`:run`	`:start_link`
`workers_count`	How many workers should be running in the pool	`5`	option is required

run_option()

@type run_option() :: {:checkout_timeout, timeout()}

Option	Description	Example	Default value
checkout_timeout	How long we can wait for a worker on the call site	`60_000`	`5000`

worker()

@type worker() :: pid()

Process id of worker.

Workers are processes launched in a pool.

Functions

acquire(pool_id, options \\ [])

@spec acquire(pool_id(), [run_option()]) ::
  {:ok, worker()} | {:error, :checkout_timeout}

Acquires a worker from the pool for manual management.

This function checks out a worker from the pool and returns its PID. Unlike run/3, the worker must be manually released using release/2. If the calling process crashes before releasing the worker, it will be automatically returned to the pool.

This is useful for long-running operations where you need to hold onto a worker for an extended period, such as maintaining a database connection for the lifetime of a TCP session.

Options

Same as run/3:

checkout_timeout - How long to wait for a worker (default: 5000ms)

Returns

{:ok, worker_pid} - Successfully acquired a worker
{:error, :checkout_timeout} - No worker available within timeout

Examples

iex> Poolex.start_link(pool_id: :my_pool, worker_module: Agent, worker_args: [fn -> 0 end], workers_count: 2)
iex> {:ok, worker} = Poolex.acquire(:my_pool)
iex> Agent.get(worker, & &1)
0
iex> Poolex.release(:my_pool, worker)
:ok

Safety

If the caller process crashes before calling release/2, the worker will be automatically killed and restarted by the supervisor. This ensures that the next caller gets a clean worker, not one potentially stuck in a long-running operation.

For graceful cleanup, always explicitly call release/2 when done with the worker.

Multiple Workers

A single process can acquire multiple workers from the same pool:

{:ok, worker1} = Poolex.acquire(:my_pool)
{:ok, worker2} = Poolex.acquire(:my_pool)
# ... use both workers ...
Poolex.release(:my_pool, worker1)
Poolex.release(:my_pool, worker2)

add_idle_workers!(pool_id, workers_count)

@spec add_idle_workers!(pool_id(), pos_integer()) :: :ok | no_return()

Adds some idle workers to existing pool.

child_spec(init_arg)

@spec child_spec([poolex_option()]) :: Supervisor.child_spec()

Returns a specification to start this module under a supervisor.

Options

Option	Description	Example	Default value
`busy_workers_impl`	Module that describes how to work with busy workers	`SomeBusyWorkersImpl`	`Poolex.Workers.Impl.List`
`failed_workers_retry_interval`	Interval in milliseconds between retry attempts for failed workers	`5_000`	`1_000`
`idle_workers_impl`	Module that describes how to work with idle workers	`SomeIdleWorkersImpl`	`Poolex.Workers.Impl.List`
`idle_overflowed_workers_impl`	Module that describes how to work with idle overflowed workers	`SomeIdleOverflowedWorkersImpl`	`Poolex.Workers.Impl.List`
`max_overflow`	How many workers can be created over the limit	`2`	`0`
`worker_shutdown_delay`	Delay (ms) before shutting down overflow worker after release	`5000`	`0`
`pool_id`	Identifier by which you will access the pool	`:my_pool`	`worker_module` value
`pool_size_metrics`	Whether to dispatch pool size metrics	`true`	`false`
`waiting_callers_impl`	Module that describes how to work with callers queue	`WaitingCallersImpl`	`Poolex.Callers.Impl.ErlangQueue`
`worker_args`	List of arguments passed to the start function	`[:gg, "wp"]`	`[]`
`worker_module`	Name of module that implements our worker	`MyApp.Worker`	option is required
`worker_start_fun`	Name of the function that starts the worker	`:run`	`:start_link`
`workers_count`	How many workers should be running in the pool	`5`	option is required

Examples

children = [
  Poolex.child_spec(worker_module: SomeWorker, workers_count: 5),
  # or in another way
  {Poolex, worker_module: SomeOtherWorker, workers_count: 5}
]

Supervisor.start_link(children, strategy: :one_for_one)

release(pool_id, worker_pid)

@spec release(pool_id(), worker()) :: :ok

Releases a manually acquired worker back to the pool.

This function returns a worker that was previously acquired with acquire/2 back to the pool, making it available for other callers.

Parameters

pool_id - The pool identifier
worker_pid - The PID of the worker to release

Returns

:ok - Always returns :ok, even if the worker was already released or doesn't exist

Examples

{:ok, worker} = Poolex.acquire(:my_pool)
# ... use worker ...
Poolex.release(:my_pool, worker)

Notes

It's safe to call release/2 multiple times for the same worker
If there are callers waiting for a worker, the released worker is provided to them
Otherwise, the worker is returned to the idle pool
The monitor process created during acquire/2 is automatically cleaned up

remove_idle_workers!(pool_id, workers_count)

@spec remove_idle_workers!(pool_id(), pos_integer()) :: :ok | no_return()

Removes some idle workers from existing pool. If the number of workers to remove is greater than the number of idle workers, all idle workers will be removed.

run(pool_id, fun, options \\ [])

@spec run(pool_id(), (worker :: pid() -> any()), [run_option()]) ::
  {:ok, any()} | {:error, :checkout_timeout}

The main function for working with the pool.

It takes a pool identifier, a function that takes a worker process id as an argument and returns any value. When executed, an attempt is made to find a free worker with specified timeout (5 seconds by default). You can set the timeout using the checkout_timeout option.

Returns:

{:ok, result} if the worker was found and the function was executed successfully.
{:error, :checkout_timeout} if no free worker was found before the timeout.

Examples

iex> Poolex.start_link(pool_id: :some_pool, worker_module: Agent, worker_args: [fn -> 5 end], workers_count: 1)
iex> Poolex.run(:some_pool, fn pid -> Agent.get(pid, &(&1)) end)
{:ok, 5}

start(opts)

@spec start([poolex_option()]) :: GenServer.on_start()

Starts a Poolex process without links (outside of a supervision tree).

See start_link/1 for more information.

Examples

iex> Poolex.start(pool_id: :my_pool, worker_module: Agent, worker_args: [fn -> 0 end], workers_count: 5)
iex> %Poolex.Private.State{worker_module: worker_module} = :sys.get_state(:my_pool)
iex> worker_module
Agent

start_link(opts)

@spec start_link([poolex_option()]) :: GenServer.on_start()

Starts a Poolex process linked to the current process.

This is often used to start the Poolex as part of a supervision tree.

After the process is started, you can access it using the previously specified pool_id.

Options

Option	Description	Example	Default value
`busy_workers_impl`	Module that describes how to work with busy workers	`SomeBusyWorkersImpl`	`Poolex.Workers.Impl.List`
`failed_workers_retry_interval`	Interval in milliseconds between retry attempts for failed workers	`5_000`	`1_000`
`idle_workers_impl`	Module that describes how to work with idle workers	`SomeIdleWorkersImpl`	`Poolex.Workers.Impl.List`
`idle_overflowed_workers_impl`	Module that describes how to work with idle overflowed workers	`SomeIdleOverflowedWorkersImpl`	`Poolex.Workers.Impl.List`
`max_overflow`	How many workers can be created over the limit	`2`	`0`
`worker_shutdown_delay`	Delay (ms) before shutting down overflow worker after release	`5000`	`0`
`pool_id`	Identifier by which you will access the pool	`:my_pool`	`worker_module` value
`pool_size_metrics`	Whether to dispatch pool size metrics	`true`	`false`
`waiting_callers_impl`	Module that describes how to work with callers queue	`WaitingCallersImpl`	`Poolex.Callers.Impl.ErlangQueue`
`worker_args`	List of arguments passed to the start function	`[:gg, "wp"]`	`[]`
`worker_module`	Name of module that implements our worker	`MyApp.Worker`	option is required
`worker_start_fun`	Name of the function that starts the worker	`:run`	`:start_link`
`workers_count`	How many workers should be running in the pool	`5`	option is required

Examples

iex> Poolex.start_link(pool_id: :other_pool, worker_module: Agent, worker_args: [fn -> 0 end], workers_count: 5)
iex> %Poolex.Private.State{worker_module: worker_module} = :sys.get_state(:other_pool)
iex> worker_module
Agent