@spec encode_string(
  [[binary()]],
  separator(),
  escape(),
  binary() | atom(),
  term(),
  term(),
  [binary()]
) ::
  iodata()

Encode rows to CSV using SIMD-accelerated scanning.

Uses portable SIMD to scan 16-32 bytes at a time for characters that need escaping. On platforms without SIMD hardware, portable_simd automatically degrades to scalar operations. Falls back to a general encoder for multi-byte separator/escape sequences.

Accepts a list of rows, where each row is a list of binary fields. Returns iodata (nested lists) — clean fields are passed through as zero-copy references, only dirty fields requiring quoting are allocated.

Parameters

• rows - List of rows (list of lists of binaries)
• separator - Field separator (see "Separator Format" above)
• escape - Escape character (see "Escape Format" above)
• line_separator - Line separator binary or :default for "\n"

Examples

iex> RustyCSV.Native.encode_string([["a", "b"], ["1", "2"]], 44, 34, :default)
"a,b\n1,2\n"

@spec encode_string_parallel(
  [[binary()]],
  separator(),
  escape(),
  binary() | atom(),
  term(),
  term(),
  [
    binary()
  ]
) :: iodata()

Encode rows to CSV in parallel using rayon, returning iodata (list of binaries).

Uses multiple threads to encode chunks of rows simultaneously. Copies all field data into Rust-owned memory before dispatching to worker threads.

Best for quoting-heavy data — fields that frequently contain commas, quotes, or newlines (e.g., user-generated content, free-text descriptions). The per-field quoting work parallelizes well and outweighs the copy overhead.

For typical/clean data where most fields pass through unquoted, prefer encode_string/4 which avoids the copy via zero-copy term references.

Only supports single-byte separator/escape. Raises ArgumentError for multi-byte configurations — use encode_string/4 instead.

Parameters

• rows - List of rows (list of lists of binaries)
• separator - Field separator (see "Separator Format" above)
• escape - Escape character (see "Escape Format" above)
• line_separator - Line separator binary or :default for "\n"

Examples

iex> RustyCSV.Native.encode_string_parallel([["a", "b"], ["1", "2"]], [","], "\"", "\r\n")
...> |> IO.iodata_to_binary()
"a,b\r\n1,2\r\n"

@spec get_rust_memory() :: non_neg_integer()

Get current Rust heap allocation in bytes.

Note: Requires the memory_tracking Cargo feature to be enabled. Without the feature, this returns 0 with no overhead.

To enable memory tracking, set the feature in native/rustycsv/Cargo.toml:

[features]
default = ["mimalloc", "memory_tracking"]

Examples

bytes = RustyCSV.Native.get_rust_memory()

@spec get_rust_memory_peak() :: non_neg_integer()

Get peak Rust heap allocation since last reset, in bytes.

Note: Requires the memory_tracking Cargo feature. Returns 0 otherwise.

Examples

peak_bytes = RustyCSV.Native.get_rust_memory_peak()

@spec parse_string(binary()) :: rows()

Parse CSV using the SIMD structural boundary scanner. Runs on a dirty CPU scheduler.

Uses a portable-SIMD scanner to find all field and row boundaries in a single pass, then builds Elixir terms using sub-binary references where possible (hybrid Cow approach — zero-copy for clean fields, copy only when unescaping is needed).

Functionally equivalent to parse_string_fast/1, parse_string_indexed/1, and parse_string_zero_copy/1 — all use the same code path. Multiple function names are retained for backward API compatibility.

Returns {:error, :input_too_large} if the input exceeds 4 GiB.

Examples

iex> RustyCSV.Native.parse_string("a,b\n1,2\n")
[["a", "b"], ["1", "2"]]

@spec parse_string_fast(binary()) :: rows()

Parse CSV using the SIMD structural boundary scanner. Runs on a dirty CPU scheduler.

Equivalent to parse_string/1 — same code path, retained for backward API compatibility.

Returns {:error, :input_too_large} if the input exceeds 4 GiB.

Examples

iex> RustyCSV.Native.parse_string_fast("a,b\n1,2\n")
[["a", "b"], ["1", "2"]]

@spec parse_string_fast_with_config(binary(), separator(), escape(), term()) :: rows()

Parse CSV with configurable separator(s) and escape. Equivalent to parse_string_with_config/4.

Examples

# TSV parsing with integer separator
iex> RustyCSV.Native.parse_string_fast_with_config("a\tb\n1\t2\n", 9, 34)
[["a", "b"], ["1", "2"]]

# TSV parsing with binary separator
iex> RustyCSV.Native.parse_string_fast_with_config("a\tb\n1\t2\n", <<9>>, 34)
[["a", "b"], ["1", "2"]]

@spec parse_string_indexed(binary()) :: rows()

Parse CSV using the SIMD structural boundary scanner. Runs on a dirty CPU scheduler.

Equivalent to parse_string/1 — same code path, retained for backward API compatibility.

Returns {:error, :input_too_large} if the input exceeds 4 GiB.

Examples

iex> RustyCSV.Native.parse_string_indexed("a,b\n1,2\n")
[["a", "b"], ["1", "2"]]

@spec parse_string_indexed_with_config(binary(), separator(), escape(), term()) ::
  rows()

Parse CSV with configurable separator(s) and escape. Equivalent to parse_string_with_config/4.

Examples

# TSV parsing with integer separator
iex> RustyCSV.Native.parse_string_indexed_with_config("a\tb\n1\t2\n", 9, 34)
[["a", "b"], ["1", "2"]]

# TSV parsing with binary separator
iex> RustyCSV.Native.parse_string_indexed_with_config("a\tb\n1\t2\n", <<9>>, 34)
[["a", "b"], ["1", "2"]]

@spec parse_string_parallel(binary()) :: rows()

Parse CSV in parallel using multiple threads.

Uses the rayon thread pool to parse rows in parallel. This is most beneficial for very large files (100MB+) where the parallelization overhead is outweighed by the parsing speedup.

This function runs on a dirty CPU scheduler to avoid blocking the normal BEAM schedulers.

Examples

iex> RustyCSV.Native.parse_string_parallel("a,b\n1,2\n")
[["a", "b"], ["1", "2"]]

@spec parse_string_parallel_with_config(binary(), separator(), escape(), term()) ::
  rows()

Parse CSV in parallel with configurable separator(s) and escape.

Examples

# TSV parallel parsing with integer separator
iex> RustyCSV.Native.parse_string_parallel_with_config("a\tb\n1\t2\n", 9, 34)
[["a", "b"], ["1", "2"]]

# TSV parallel parsing with binary separator
iex> RustyCSV.Native.parse_string_parallel_with_config("a\tb\n1\t2\n", <<9>>, 34)
[["a", "b"], ["1", "2"]]

@spec parse_string_with_config(binary(), separator(), escape(), term()) :: rows()

Parse CSV with configurable separator(s) and escape characters.

Parameters

• csv - The CSV binary to parse
• separator - Integer byte, binary, or list of binaries (see "Separator Format" above)
• escape - Integer byte or binary (see "Escape Format" above)

Examples

# TSV parsing with integer separator
iex> RustyCSV.Native.parse_string_with_config("a\tb\n1\t2\n", 9, 34)
[["a", "b"], ["1", "2"]]

# TSV parsing with binary separator
iex> RustyCSV.Native.parse_string_with_config("a\tb\n1\t2\n", <<9>>, 34)
[["a", "b"], ["1", "2"]]

# Multi-separator parsing (comma or semicolon)
iex> RustyCSV.Native.parse_string_with_config("a,b;c\n1;2,3\n", [<<44>>, <<59>>], 34)
[["a", "b", "c"], ["1", "2", "3"]]

# Multi-byte separator
iex> RustyCSV.Native.parse_string_with_config("a::b::c\n", "::", 34)
[["a", "b", "c"]]

# Multi-byte escape
iex> RustyCSV.Native.parse_string_with_config("$$hello$$,world\n", 44, "$$")
[["hello", "world"]]

@spec parse_string_zero_copy(binary()) :: rows()

Parse CSV using the SIMD structural boundary scanner. Runs on a dirty CPU scheduler.

Equivalent to parse_string/1 — same code path, retained for backward API compatibility.

Returns {:error, :input_too_large} if the input exceeds 4 GiB.

Examples

iex> RustyCSV.Native.parse_string_zero_copy("a,b\n1,2\n")
[["a", "b"], ["1", "2"]]

@spec parse_string_zero_copy_with_config(binary(), separator(), escape(), term()) ::
  rows()

Parse CSV with configurable separator(s) and escape. Equivalent to parse_string_with_config/4.

Examples

# TSV parsing with integer separator
iex> RustyCSV.Native.parse_string_zero_copy_with_config("a\tb\n1\t2\n", 9, 34)
[["a", "b"], ["1", "2"]]

# TSV parsing with binary separator
iex> RustyCSV.Native.parse_string_zero_copy_with_config("a\tb\n1\t2\n", <<9>>, 34)
[["a", "b"], ["1", "2"]]

@spec parse_to_maps(
  binary(),
  separator(),
  escape(),
  term(),
  atom(),
  atom() | list(),
  boolean()
) :: [
  map()
]

Parse CSV and return list of maps, dispatching to the specified strategy.

Parameters

• input - The CSV binary to parse
• separator - Separator(s) (see "Separator Format" above)
• escape - Escape sequence (see "Escape Format" above)
• strategy - Atom: :basic, :simd, :indexed, or :zero_copy
• header_mode - Atom :true (first row = keys) or list of key terms
• skip_first - Whether to skip the first row when using explicit keys

@spec parse_to_maps_parallel(
  binary(),
  separator(),
  escape(),
  term(),
  atom() | list(),
  boolean()
) :: [map()]

Parse CSV in parallel and return list of maps.

Uses the rayon thread pool on a dirty CPU scheduler.

Parameters

• input - The CSV binary to parse
• separator - Separator(s) (see "Separator Format" above)
• escape - Escape sequence (see "Escape Format" above)
• header_mode - Atom :true (first row = keys) or list of key terms
• skip_first - Whether to skip the first row when using explicit keys

@spec reset_rust_memory_stats() :: {non_neg_integer(), non_neg_integer()}

Reset memory tracking statistics.

Note: Requires the memory_tracking Cargo feature. Returns {0, 0} otherwise.

Returns {current_bytes, previous_peak_bytes}.

Examples

{current, peak} = RustyCSV.Native.reset_rust_memory_stats()

@spec streaming_feed(parser_ref(), binary()) :: {non_neg_integer(), non_neg_integer()}

Feed a chunk of CSV data to the streaming parser. Runs on a dirty CPU scheduler.

Returns {available_rows, buffer_size} indicating the number of complete rows ready to be taken and the current buffer size.

Raises

• :buffer_overflow — the chunk would push the internal buffer past the maximum size (default 256 MB). Use streaming_set_max_buffer/2 to adjust the limit.
• :mutex_poisoned — a previous NIF call panicked while holding the parser lock. The parser should be discarded.

Examples

{available, buffer_size} = RustyCSV.Native.streaming_feed(parser, chunk)

@spec streaming_finalize(parser_ref()) :: rows()

Finalize the streaming parser and get any remaining rows. Runs on a dirty CPU scheduler.

This should be called after all data has been fed to get any partial row that was waiting for a terminating newline.

Examples

final_rows = RustyCSV.Native.streaming_finalize(parser)

@spec streaming_new() :: parser_ref()

Create a new streaming parser instance.

The streaming parser maintains internal state and can process CSV data in chunks, making it suitable for large files with bounded memory usage.

The returned reference is safe to share across BEAM processes — the underlying Rust state is protected by a mutex.

Examples

parser = RustyCSV.Native.streaming_new()
RustyCSV.Native.streaming_feed(parser, "a,b\n")
RustyCSV.Native.streaming_feed(parser, "1,2\n")
rows = RustyCSV.Native.streaming_next_rows(parser, 100)

@spec streaming_new_with_config(separator(), escape(), term()) :: parser_ref()

Create a new streaming parser with configurable separator(s) and escape.

Parameters

• separator - Integer byte, binary, or list of binaries (see "Separator Format" above)
• escape - Integer byte or binary (see "Escape Format" above)

Examples

# TSV streaming parser with integer separator
parser = RustyCSV.Native.streaming_new_with_config(9, 34)

# TSV streaming parser with binary separator
parser = RustyCSV.Native.streaming_new_with_config(<<9>>, 34)

# Multi-separator streaming parser
parser = RustyCSV.Native.streaming_new_with_config([<<44>>, <<59>>], 34)

# Multi-byte separator streaming parser
parser = RustyCSV.Native.streaming_new_with_config("::", 34)

@spec streaming_next_rows(parser_ref(), non_neg_integer()) :: rows()

Take up to max complete rows from the streaming parser. Runs on a dirty CPU scheduler.

Returns the rows as a list of lists of binaries.

Examples

rows = RustyCSV.Native.streaming_next_rows(parser, 100)

@spec streaming_set_max_buffer(parser_ref(), non_neg_integer()) :: :ok

Set the maximum buffer size (in bytes) for the streaming parser. Default is 256 MB. Raises on overflow during streaming_feed/2.

@spec streaming_status(parser_ref()) ::
  {non_neg_integer(), non_neg_integer(), boolean()}

Get the current status of the streaming parser.

Returns {available_rows, buffer_size, has_partial}:

• available_rows - Number of complete rows ready to be taken
• buffer_size - Current size of the internal buffer in bytes
• has_partial - Whether there's an incomplete row in the buffer

Examples

{available, buffer, has_partial} = RustyCSV.Native.streaming_status(parser)