| Overview |  |
Abstract iterator library.
It allows to define your own iterators over some continuous source of data and also build compositions, chains, filters and transformations of those iterators.
Iterators are lazy, so, chains of iterators would use O(1) of memory. Also, infinite iterators are possible.
It tries to mimick some of the APIs of thelists module where possible.
close_fun() = fun((any()) -> ok)
abstract datatype: iterator(Type)
yield_fun(Type) = fun((St::any()) -> {Type, St::any()} | done)
| append/1 | Joins multiple iterators to a single steam (non-recursive, only top-level)
Similar to lists:append/1 |
| chunks/2 | Takes up to N items from sub-iterator at a time and yields them as non-empty list
Last chunk may contain less than N elements. |
| close/1 | |
| concat/1 | deprecated. |
| dropwhile/2 | Skips elements of inner iterator until Fun returns true
Similar to lists:dropwhile/2 |
| eterm_fd_iterator/1 | Iterator over already opened .eterm file It's up to the caller to make sure file is opened with correct options and closed when done. |
| eterm_file_iterator/1 | Iterator over .eterm file (file containing dot-terminated Erlang terms) XXX: never abandon this iterator from long-running processes! |
| filter/2 | Returns a new iterator that yields elements of InnerIterator which were allowed by Fun
Similar to lists:filter/2 |
| filtermap/2 | Returns a new iterator that yields (optionally modified) elements of InnerIterator
if Fun returns true or {true, NewValue} and skips the element if Fun returns false. |
| flatmap/2 | Consumes the iterator that returns a list item and yields list elements one-by-one
Opposite of chunks/1 |
| flatten1/1 | Converts iterator that yields lists to flat list (non-recursive). |
| fold/3 | Folds over the iterator, returning the fold accumulator
Similar to lists:foldl/2 |
| foldl/3 | Convenient alias for fold/3 |
| foreach/2 | Applies Fun to each element of the iterator, discarding the results
It does not return a new iterator, but atom ok and always consumes the iterator to the end. |
| from_list/1 | Converts list to iterator that yields the elements of this list. |
| from_map/1 | Converts map to the iterator that yields {Key, Value} tuples in unspecified order. |
| is_iterator/1 | Returns true if argument provided is iterator() |
| map/2 | Returns a new iterator that yields results of application of the Fun to the each element
of InnerIterator. |
| mapfoldl/3 | Stateful map |
| new/2 | |
| new/3 | Creates a new iterator that needs to be explicitly "closed" when fully consumed or when patially consumed but then abandoned. |
| next/1 | Returns next element of the iterator and new iterator, or done when iterator is empty. |
| nthtail/2 | Discards first N elements of inner iterator
If inner iterator procuced less than N elements, it fails with badarg error. |
| pv/3 | Passthrough iterator one can use to periodically report the progress of the inner iterator. |
| report/3 | Alias for pv/3 |
| search/2 | Returns the first element that matches the predicate, otherwise false |
| sublist/2 | Yields at most N elements of the inner iterator
Similar to lists:sublist/2 |
| takewhile/2 | Yields elements of inner iterator as long as Fun returns true
Similar to lists:takewhile/2 |
| to_list/1 | Converts the iterator to the list. |
| zip/3 | Consumes 2 iterators at the same time, returns an iterator that yields 2-tuples containing next element of each iterator. |
Joins multiple iterators to a single steam (non-recursive, only top-level)
Similar to lists:append/1
Takes up to N items from sub-iterator at a time and yields them as non-empty list
Last chunk may contain less than N elements.
Kind of similar to lists:split/2 but repeated recursively and not throwing errors.
close(Iter) -> any()
concat(Inner) -> any()
deprecated
Skips elements of inner iterator until Fun returns true
Similar to lists:dropwhile/2
eterm_fd_iterator(Fd::file:io_device()) -> iterator(any())
Iterator over already opened .eterm file It's up to the caller to make sure file is opened with correct options and closed when done.
eterm_file_iterator(Filename::file:name_all()) -> iterator(any())
Iterator over .eterm file (file containing dot-terminated Erlang terms)
XXX: never abandon this iterator from long-running processes! It would leak file descriptor!
Either consume it to the end or close with iterator:close/1 explicitly.
Returns a new iterator that yields elements of InnerIterator which were allowed by Fun
Similar to lists:filter/2
filtermap(Fun, InnerIterator) -> any()
Returns a new iterator that yields (optionally modified) elements of InnerIterator
if Fun returns true or {true, NewValue} and skips the element if Fun returns false.
lists:filtermap/2
flatmap(Fun::fun((InType) -> [OutType]), InnerIterator::iterator(InType)) -> iterator(OutType)
InType = any()OutType = any()
Consumes the iterator that returns a list item and yields list elements one-by-one
Opposite of chunks/1
Converts iterator that yields lists to flat list (non-recursive)
Do not confuse withappend/1. This function takes SINGLE iterator that yields
lists and returns list elements one-by-one.
fold(Fun::fun((IterType, Acc) -> Acc), Acc, Iterator::iterator:iterator(IterType)) -> Acc
Acc = any()IterType = any()
Folds over the iterator, returning the fold accumulator
Similar to lists:foldl/2
foldl(Fun, Acc, Iterator) -> any()
Convenient alias for fold/3
Applies Fun to each element of the iterator, discarding the results
It does not return a new iterator, but atom ok and always consumes the iterator to the end.
Only makes sense for side-effecting functions.
Similar to lists:foreach/2
Converts list to iterator that yields the elements of this list
from_map(Map::#{Key => Val}) -> iterator({Key, Val})
Converts map to the iterator that yields {Key, Value} tuples in unspecified order
maps:to_list/1
is_iterator(Iter::any()) -> boolean()
Returns true if argument provided is iterator()
map(Fun::fun((InType) -> OutType), InnerIterator::iterator(InType)) -> iterator(OutType)
InType = any()OutType = any()
Returns a new iterator that yields results of application of the Fun to the each element
of InnerIterator.
Similar to lists:map/2
mapfoldl(Fun::fun((A, AccIn) -> {B, AccOut}), AccIn, InnerIterator::iterator(A)) -> iterator(B)
A = term()B = term()AccIn = term()AccOut = term()
Stateful map
Somewhat equivalent of lists:mapfoldl/2---transforms each element
of the sequence while keeping a stateful context.
lists:mapfoldl/2 does,
for simplicity.
new(YieldFun::yield_fun(Type), IterState, CloseFun::close_fun()) -> iterator(Type)
Type = any()IterState = any()
Creates a new iterator that needs to be explicitly "closed" when fully consumed or when patially consumed but then abandoned.
close_fun() should be idempotent (it could be called multiple times).
There is no 100% guarantee that close_fun() will be called! But other high-order iterators
from this module do their best to make sure they are closed.
Returns next element of the iterator and new iterator, or done when iterator is empty
close/1 on it
before abandoning.
Discards first N elements of inner iterator
If inner iterator procuced less than N elements, it fails with badarg error.
lists:nthtail/2
pv(F::fun((Type, TimePassed::integer(), ItemsPassed::integer(), TotalItems::integer()) -> any()), Opts::#{for_each_n => pos_integer(), every_s => pos_integer()}, InnerIter::iterator:iterator(Type)) -> iterator:iterator(Type)
Type = any()F: function to call when one of the conditions triggers. Function arguments:
- Data - current element of the inner iterator (sample)
- TimePassed - time passed since the last report in native units
- ItemsPassed - number of items passed since the last report
- TotalItems - total number of items passed since the start
TimePassed + ItemsPassed are convenient to calculate the speed of the stream.
Opts: trigger condition options:
- for_each_n - trigger every N-th element
- every_s - trigger every S seconds
InnerIter:
inner iterator to wrap
Keep in mind that whichever trigger condition is met first, the F function will be called and
counters/timers will reset. So if you set for_each_n to 1000 and every_s to 30, then the
function will be called either as counter reaches 1000 or 30 seconds pass since the last call.
every_s seconds to process a single element, the function will be called
with additional delay.Passthrough iterator one can use to periodically report the progress of the inner iterator.
Name comes from pv (pipe view) Unix utility. See man pv.
report(F, Opts, InnerIter) -> any()
Alias for pv/3
search(Fun::fun((Type) -> boolean()), Iterator::iterator(Type)) -> {value, Type} | false
Type = any()
Returns the first element that matches the predicate, otherwise false
Yields at most N elements of the inner iterator
Similar to lists:sublist/2
Yields elements of inner iterator as long as Fun returns true
Similar to lists:takewhile/2
Converts the iterator to the list
XXX: Might run OOM if called over the infinite iterator!zip(Iter1::iterator(Data1), Iter2::iterator(Data2), How::trim | {pad, {Default1, Default2}}) -> iterator({Data1 | Default1, Data2 | Default2})
Data1 = any()Data2 = any()Default1 = any()Default2 = any()
Consumes 2 iterators at the same time, returns an iterator that yields
2-tuples containing next element of each iterator.
If one of the iterators is done before the other, the behaviour would depend
on How parameter:
* If it is trim, then the other iterator will be closed and zip/3 would finish.
* If it is {pad, {Default1, Default2}, then the values of the finished iterator
would be replaced with Default until the 2nd iterator is done.
lists:zip/3, but doesn't support fail behaviour (just because
it rarely makes sense for lazy sequences; feel free to file an issue otherwise).
Generated by EDoc