ExArrow.Stream
(ex_arrow v0.4.0)
View Source
Opaque handle to a native Arrow record-batch stream.
Provides a unified iterator interface over three backing sources:
| Backend | Created by |
|---|---|
:ipc | ExArrow.IPC.Reader — Arrow IPC stream or file format |
:parquet | ExArrow.Parquet.Reader — lazy row-group Parquet reader |
:adbc | ExArrow.ADBC.Statement.execute/1 — SQL result streams |
Flight do_get results also use the :ipc backend (the Flight client
returns an IPC stream resource).
All three backends expose the same three functions:
schema/1— inspect the Arrow schema without consuming any batchesnext/1— read the next batch on demand (nilwhen exhausted)to_list/1— collect all remaining batches into a list
Record batch data stays in native Arrow memory until consumed. Callers
never set the backend field directly; it is assigned by the function that
opens the stream.
Summary
Functions
Returns the next record batch from the stream, or nil when done.
Returns {:error, message} on read error.
Returns the schema of this stream (without consuming it).
Returns {:error, message} if the stream is invalid (e.g. poisoned lock).
Collects all remaining batches from the stream into a list.
Types
Functions
@spec next(t()) :: ExArrow.RecordBatch.t() | nil | {:error, String.t()}
Returns the next record batch from the stream, or nil when done.
Returns {:error, message} on read error.
@spec schema(t()) :: {:ok, ExArrow.Schema.t()} | {:error, String.t()}
Returns the schema of this stream (without consuming it).
Returns {:error, message} if the stream is invalid (e.g. poisoned lock).
@spec to_list(t()) :: [ExArrow.RecordBatch.t()]
Collects all remaining batches from the stream into a list.
Stops at the first error and raises. Returns an empty list for an already-exhausted stream.