data_source v0.1.4 Datasource.File
Can be used in Datasource
as the function-argument. It streams a given file
line by line.
Link to this section Summary
Functions
Get next line from the file(stream)
Link to this section Functions
Link to this function
next(filename)
Get next line from the file(stream)
Examples:
Read through datasource reads the file as stream and should give the same result as reading the file directly.
iex> expected = File.read!("README.md")
...> num_lines = (String.split(expected, "\n") |> Enum.count) - 1
...>
...> # Start the datasource for file "README"
...> {:ok, file_source} = Datasource.start_link("README.md", &Datasource.File.next/1)
...>
...> # Call `next()` number of lines times
...> subject =
...> Enum.map((1..num_lines), fn(_) -> Datasource.next(file_source) end)
...> |> Enum.join
...>
...> assert subject == expected
true
You can use Enum.flat_map_reduce
to read from the datasource until it returns
:no_data
.
iex> {:ok, file_source} = Datasource.start_link("README.md", &Datasource.File.next/1)
...> { _, cnt } = Enum.flat_map_reduce((0..1000), 0, fn x, acc ->
...> if Datasource.next(file_source) == :no_data, do: {:halt, acc}, else: {[x], acc + 1}
...> end)
...> assert cnt <= 1000
true
When next
is called for the first time, the filename is passed
as a string. In this case next
will create the Stream
for this
file and calls next
again, this time with the stream to use and
reads the first line.