datastream

datastream is a compositional, resource-safe stream library for Gleam. The cross-target core (datastream/source, datastream/stream, datastream/fold, datastream/sink, datastream/chunk, datastream/text, datastream/binary) runs on both the Erlang and JavaScript targets. The datastream/erlang/* extension modules are quarantined to the Erlang target.

This module defines the foundational types every later module builds on: the opaque pipeline value Stream(a) and the pull-result enum Step(a, state).

Types

Step

</>

The result of pulling one element from a stream-like producer.

Next(element, state) carries the produced element together with the continuation needed to pull the next one. Done signals end-of-stream. Once Done is observed the producer MUST NOT be pulled again.

There is intentionally no Error variant: the core does not impose a failure model. Callers that need errors carry them as element-shaped values (Stream(Result(a, e)), Stream(Validated(a, e)), …).

Unlike Stream(a), Step is a transparent public type: callers return Next / Done from the next callback passed to source.resource and source.try_resource. The enum shape is stable from v0.1.0 onward.

pub type Step(a, state) {
  Next(element: a, state: state)
  Done
}

Constructors

```
Next(element: a, state: state)
```
```
Done
```

Stream

opaque </>

A lazy, pull-based pipeline that yields elements of type a.

Values of this type are pipeline definitions, not materialised collections. Each terminal operation (in fold / sink) re-runs the pipeline from its source — there is no implicit caching. Callers that want replay should materialise once with fold.to_list.

The internal representation is intentionally hidden so the library is free to change it (list-backed, function-batched, chunk-batched, …) without a breaking change. == / != carry no specified meaning; observational equality is via terminal reduction.

The library threads a cleanup callback through every stream value so resource-safe sources from source.resource can be released on every termination path (normal end, downstream early-exit, early-exit folds, try_each failure). Pure (unfold-built) streams use a no-op cleanup.

pub opaque type Stream(a)