SqlKit.DuckDB (sql_kit v0.2.0)
View SourceDuckDB support for SqlKit.
Provides two ways to use DuckDB with SqlKit:
Direct Connection (BYO)
For simple use cases, scripts, or explicit control:
{:ok, conn} = SqlKit.DuckDB.connect(":memory:")
SqlKit.query_all(conn, "SELECT 1 as num", [])
# => [%{num: 1}]
SqlKit.DuckDB.disconnect(conn)Pooled Connection (Recommended for Production)
For production use, add the pool to your supervision tree:
children = [
{SqlKit.DuckDB.Pool,
name: MyApp.AnalyticsPool,
database: "priv/analytics.duckdb",
pool_size: 4}
]
# Then use the pool name with SqlKit functions
SqlKit.query_all(MyApp.AnalyticsPool, "SELECT * FROM events", [])Loading Extensions
DuckDB extensions are loaded via SQL (SQL-first philosophy):
SqlKit.query!(conn, "INSTALL 'parquet';", [])
SqlKit.query!(conn, "LOAD 'parquet';", [])
SqlKit.query_all(conn, "SELECT * FROM 'data.parquet'", [])Notes
- Uses PostgreSQL-style
$1, $2, ...parameter placeholders - In-memory database: use
":memory:"string (not:memoryatom) - Hugeint values are automatically converted to Elixir integers
Summary
Functions
Opens a DuckDB database and creates a connection.
Opens a DuckDB database and creates a connection. Raises on error.
Converts hugeint tuples to integers in result rows.
Converts a single value, handling hugeint tuples.
Closes a DuckDB connection and releases the database.
Executes a SQL query and returns columns and rows.
Executes a SQL query and returns columns and rows. Raises on error.
Executes a SQL query and returns a stream of result chunks.
Like stream!/3 but also returns column names.
Types
@type connect_opts() :: [{:config, struct()}]
Functions
@spec connect(String.t(), connect_opts()) :: {:ok, SqlKit.DuckDB.Connection.t()} | {:error, term()}
Opens a DuckDB database and creates a connection.
Arguments
database- Path to database file or":memory:"for in-memory database
Options
:config- ADuckdbex.Configstruct for advanced configuration
Examples
# In-memory database
{:ok, conn} = SqlKit.DuckDB.connect(":memory:")
# File-based database
{:ok, conn} = SqlKit.DuckDB.connect("analytics.duckdb")
# With configuration
{:ok, conn} = SqlKit.DuckDB.connect("analytics.duckdb",
config: %Duckdbex.Config{threads: 4})
@spec connect!(String.t(), connect_opts()) :: SqlKit.DuckDB.Connection.t()
Opens a DuckDB database and creates a connection. Raises on error.
See connect/2 for options.
Converts hugeint tuples to integers in result rows.
Duckdbex represents HUGEINT (128-bit integers) as {upper, lower} tuples.
This function recursively converts all such tuples to Elixir integers.
This is safe because other duckdbex tuple types have different arities:
- DATE:
{year, month, day} - TIME:
{hour, minute, second, microsecond} - DECIMAL:
{value, precision, scale} - TIMESTAMP:
{{y, m, d}, {h, m, s, us}}
If duckdbex adds another 2-integer-tuple type in the future, this would need to be updated. Check duckdbex changelog on upgrades.
Converts a single value, handling hugeint tuples.
See convert_hugeints/1 for details.
@spec disconnect(SqlKit.DuckDB.Connection.t()) :: :ok
Closes a DuckDB connection and releases the database.
Examples
{:ok, conn} = SqlKit.DuckDB.connect(":memory:")
:ok = SqlKit.DuckDB.disconnect(conn)
@spec query(SqlKit.DuckDB.Connection.t(), String.t(), list()) :: {:ok, {[String.t()], [[term()]]}} | {:error, term()}
Executes a SQL query and returns columns and rows.
This is a low-level function. Users should typically use
SqlKit.query_all/3, SqlKit.query_one!/3, etc. instead.
Examples
{:ok, {columns, rows}} = SqlKit.DuckDB.query(conn, "SELECT 1 as num", [])
# => {:ok, {["num"], [[1]]}}
@spec query!(SqlKit.DuckDB.Connection.t(), String.t(), list()) :: {[String.t()], [[term()]]}
Executes a SQL query and returns columns and rows. Raises on error.
See query/3 for details.
@spec stream!(SqlKit.DuckDB.Connection.t(), String.t(), list()) :: Enumerable.t()
Executes a SQL query and returns a stream of result chunks.
Unlike query!/3 which loads all results into memory, stream!/3 returns
a lazy Stream that fetches results in chunks. This is useful for large
result sets that would otherwise consume too much memory.
Each chunk is a list of rows (each row is a list of values). Use
Stream.flat_map/2 to iterate over individual rows.
Examples
# Stream large result set
conn
|> SqlKit.DuckDB.stream!("SELECT * FROM large_table", [])
|> Stream.flat_map(& &1)
|> Enum.take(100)
# With column names (first element of tuple)
{columns, row_stream} = SqlKit.DuckDB.stream_with_columns!(conn, sql, [])
rows = row_stream |> Stream.flat_map(& &1) |> Enum.to_list()Notes
- The connection is held for the duration of stream consumption
- Hugeint values are automatically converted to Elixir integers
- For pooled connections, use
SqlKit.DuckDB.Pool.with_stream!/5
@spec stream_with_columns!(SqlKit.DuckDB.Connection.t(), String.t(), list()) :: {[String.t()], Enumerable.t()}
Like stream!/3 but also returns column names.
Returns {columns, row_chunk_stream} where columns is a list of column
names and row_chunk_stream is a stream of row chunks.
Examples
{columns, stream} = SqlKit.DuckDB.stream_with_columns!(conn, "SELECT * FROM users", [])
# columns => ["id", "name", "age"]
rows = stream |> Stream.flat_map(& &1) |> Enum.to_list()