ParallelStream (Parallel Stream v1.1.0) View Source
Parallel stream implementation for Elixir.
Link to this section Summary
Functions
Creates a stream that will apply the given function on enumeration in parallel. The functions return value will be thrown away, hence this is useful for producing side-effects.
Creates a stream that will apply the given function on enumeration in parallel and only pass the values for which the function returns truthy downstream.
Creates a stream that will apply the given function on enumeration in parallel and return the functions return value.
Creates a stream that will apply the given function on enumeration in parallel and only pass the values for which the function returns falsy downstream.
Link to this section Functions
Creates a stream that will apply the given function on enumeration in parallel. The functions return value will be thrown away, hence this is useful for producing side-effects.
Options
These are the options:
:num_workers– The number of parallel operations to run when running the stream.:worker_work_ratio– The available work per worker, defaults to 5. Higher rates will mean more work sharing, but might also lead to work fragmentation slowing down the queues.
Examples
Iterate and write the numbers to stdout:
iex> parallel_stream = 1..5 |> ParallelStream.each(&IO.write/1)
iex> parallel_stream |> Stream.run
:ok # 12345 appears on stdoutCreates a stream that will apply the given function on enumeration in parallel and only pass the values for which the function returns truthy downstream.
Options
These are the options:
:num_workers– The number of parallel operations to run when running the stream.:worker_work_ratio– The available work per worker, defaults to 5. Higher rates will mean more work sharing, but might also lead to work fragmentation slowing down the queues.
Examples
Map and filter the even numbers:
iex> parallel_stream = 1..5 |> ParallelStream.filter(fn i -> i |> rem(2) == 0 end)
iex> parallel_stream |> Enum.to_list
[2,4]Creates a stream that will apply the given function on enumeration in parallel and return the functions return value.
Options
These are the options:
:num_workers– The number of parallel operations to run when running the stream.:worker_work_ratio– The available work per worker, defaults to 5. Higher rates will mean more work sharing, but might also lead to work fragmentation slowing down the queues.
Examples
Map and duplicate the numbers:
iex> parallel_stream = 1..5 |> ParallelStream.map(fn i -> i * 2 end)
iex> parallel_stream |> Enum.to_list
[2, 4, 6, 8, 10]Creates a stream that will apply the given function on enumeration in parallel and only pass the values for which the function returns falsy downstream.
Options
These are the options:
:num_workers– The number of parallel operations to run when running the stream.:worker_work_ratio– The available work per worker, defaults to 5. Higher rates will mean more work sharing, but might also lead to work fragmentation slowing down the queues.
Examples
Map and reject the even numbers:
iex> parallel_stream = 1..5 |> ParallelStream.reject(fn i -> i |> rem(2) == 0 end)
iex> parallel_stream |> Enum.to_list
[1,3,5]