Defines how data can be acquired in a piecemeal fashion, perhaps by reading only a few pages from the disk at a time or only a few MBs of data from an open socket.

The Source behaviour defines three callbacks that must be implemented by conforming modules:

1. validate/1, which is called to check the initialisation argument.
2. init/1, which is called to instantiate the source and return its state.
3. read/1, which is called to read data from the source, given the state.

Representing Sources

Sources are represented in Manifest Entries as tuples such as {:file, path} or {:url, url}. This form of representation is called a Source Entry.

The Source Entry is a stable locator of the underlying data which has no runtime implications. The Encoder hydrates the Source Entry into whatever the Source module implements internally, when it is time to pull data from that source.

The first element in the tuple is the Source Name, and the second element is called the Initialisation Argument (init_arg).

Source Name

The Source names can be special atoms (short names) or full module names:

1. :file resolves to Packmatic.Source.File.
2. :url resolves to Packmatic.Source.URL.
3. :dynamic resolves to Packmatic.Source.Dynamic.
4. :random resolves to Packmatic.Source.Random.

When using Custom Sources, the module name can be passed (as an atom) as well.

Initialisation Argument

The Initialisation Argument is usually a basic Elixir type, but in the case of Dynamic Sources, it is a function which resolves to a Source Entry understood by either the File or URL source.

Examples

The Source Entry {:file, path} is resolved during encoding:

iex(1)> {:ok, file_path} = Briefly.create()
iex(2)> {:ok, {module, state}} = Packmatic.Source.build({:file, file_path})
iex(3)> module
Packmatic.Source.File
iex(3)> state.__struct__
Packmatic.Source.File

Notes

When implementing a custom Source which uses an external data provider (for example reading from a file), remember to perform any cleanup required within the read/1 callback if the Source is not expected to return any further data, for example if the file has been read completely or if there has been an error.