| Overview |  |
This module provides lazy, composable producer streams that can be considered counterparts to Archiver's consumer pipes and therefore can facilitate testing
Also it comes with an implementation of binary data stream which is able to produce sufficiently large amounts of plausibly pseudorandom binary payload in a deterministic way. It also contains routines to check binary blobs via samplingbinary_payload() = {binary(), _ChunkNum::non_neg_integer(), _ChunkCnt::non_neg_integer()}
abstract datatype: chunk_state()
abstract datatype: cont(Data)
payload() = {Seed::term(), Size::integer()}
stream(Data) = maybe_improper_list(Data, cont(Data)) | '?end_of_stream'
tagged_binstream() = stream({Tag::term(), Payload::chunk_state()})
| binary_stream_gen/1 | Proper generator that creates a binary stream. |
| check_consistency/3 | |
| check_file_consistency/3 | |
| consume/1 | Equivalent to consume(Stream, fun (A) -> A end). |
| consume/2 | Consume all elements of the stream and feed them into a callback (e.g. |
| generate_chunk/2 | Stream of binary chunks. |
| generate_chunks/3 | Generate chunks of data and feed them into
Callback |
| generator_fun/2 | First argument is a chunk number, the second one is a seed. |
| get_byte/2 | Get byte at offset N |
| interleave_chunks/1 | Take a list of payload()s and a callback function, and start
producing the payloads in random order. |
| interleave_chunks/2 | Take a list of payload()s and a callback function, and start
consuming the payloads in a random order. |
| interleave_streams/1 | Take a list of tagged streams and return a stream where elements of the streams are tagged and randomly interleaved. |
| interleaved_binary_gen/1 | |
| interleaved_list_gen/1 | |
| interleaved_streams_gen/1 | Equivalent to interleaved_streams_gen(10, Type). |
| list_to_stream/1 | |
| mb/1 | Return number of bytes in N megabytes. |
| next/1 | Consume one element from the stream. |
| retransmits/2 | Take an arbitrary stream and add repetitions of the elements. |
binary_stream_gen(_ChunkSize::non_neg_integer()) -> proper_types:type()
Proper generator that creates a binary stream
check_consistency(X1::payload(), SampleSize::integer(), Callback::fun((integer()) -> {ok, binary()} | undefined)) -> ok
check_file_consistency(Payload::payload(), SampleSize::integer(), FileName::file:filename()) -> ok
consume(Stream::stream(A)) -> [A]
Equivalent to consume(Stream, fun (A) -> A end).
consume(Stream::stream(A), Callback::fun((A) -> Ret)) -> [Ret]
Consume all elements of the stream and feed them into a callback (e.g. brod:produce)
generate_chunk(X1::payload(), ChunkSize::integer()) -> stream(binary_payload())
Stream of binary chunks. Limitation: both payload size and
ChunkSize should be dividable by ?hash_size
generate_chunks(Payload::payload(), ChunkSize::integer(), Callback::fun((binary()) -> A)) -> [A]
Generate chunks of data and feed them into
Callback
generator_fun(N::integer(), Seed::binary()) -> binary()
First argument is a chunk number, the second one is a seed. This implementation is hardly efficient, but it was chosen for clarity reasons
get_byte(N::integer(), Seed::term()) -> byte()
Get byte at offset N
interleave_chunks(Streams0::[{payload(), ChunkSize::non_neg_integer()}]) -> tagged_binstream()
Take a list of payload()s and a callback function, and start
producing the payloads in random order. Seed is used as a tag
See also: interleave_streams/4.
interleave_chunks(Streams0::[payload()], ChunkSize::non_neg_integer()) -> tagged_binstream()
Take a list of payload()s and a callback function, and start
consuming the payloads in a random order. Seed is used as a
tag. All streams use the same chunk size
See also: interleave_streams/2.
Take a list of tagged streams and return a stream where elements of the streams are tagged and randomly interleaved.
Note: this function is more or less generic and it's compatible
with this module's generate_chunks function family, as well as
ets:next, lists and what not
interleaved_binary_gen(ChunkSize::non_neg_integer()) -> proper_types:type()
interleaved_list_gen(Type::proper_types:type()) -> proper_types:type()
interleaved_streams_gen(Type::proper_types:type()) -> proper_types:type()
Equivalent to interleaved_streams_gen(10, Type).
list_to_stream(L::[A]) -> stream(A)
mb(N::integer()) -> integer()
Return number of bytes in N megabytes
Consume one element from the stream.
Take an arbitrary stream and add repetitions of the elements
Generated by EDoc