RustyCSV is an ultra-fast CSV parsing and dumping library powered by purpose-built Rust NIFs.

It provides a drop-in replacement for NimbleCSV with the same API, while offering multiple parsing strategies optimized for different use cases.

Quick Start

Use the pre-defined RustyCSV.RFC4180 parser:

alias RustyCSV.RFC4180, as: CSV

CSV.parse_string("name,age\njohn,27\n")
#=> [["john", "27"]]

CSV.parse_string("name,age\njohn,27\n", skip_headers: false)
#=> [["name", "age"], ["john", "27"]]

Defining Custom Parsers

You can define custom CSV parsers with define/2:

RustyCSV.define(MyParser,
  separator: ",",
  escape: "\"",
  line_separator: "\n"
)

MyParser.parse_string("a,b\n1,2\n")
#=> [["1", "2"]]

Parsing Strategies

RustyCSV supports multiple parsing strategies via the :strategy option:

• :simd - SIMD-accelerated scanning via memchr (default, fastest for most files)
• :basic - Simple byte-by-byte parsing (good for debugging)
• :indexed - Two-phase index-then-extract (good for re-extracting rows)
• :parallel - Multi-threaded via rayon (best for very large files 500MB+ with complex quoting)
• :zero_copy - Sub-binary references (NimbleCSV-like memory profile, max speed)

Example:

CSV.parse_string(large_csv, strategy: :parallel)

Scheduling

All parsing NIFs run on BEAM dirty CPU schedulers, so they never block normal schedulers. Parallel parsing (:parallel strategy) additionally runs on a dedicated rustycsv-* rayon thread pool to avoid contention with other Rayon users in the same VM.

Streaming

For large files, use parse_stream/2 which uses a bounded-memory streaming parser:

"huge.csv"
|> File.stream!()
|> CSV.parse_stream()
|> Stream.each(&process_row/1)
|> Stream.run()

Streaming parsers are safe to share across processes — the underlying Rust resource is protected by a mutex. However, concurrent access is serialized, so for maximum throughput use one parser per process.

Encoding (Dumping)

Convert rows back to CSV format:

CSV.dump_to_iodata([["name", "age"], ["john", "27"]])
#=> "name,age\njohn,27\n"

Encoding uses a SIMD-accelerated Rust NIF that writes all CSV bytes into a single flat binary. The NIF handles four modes: plain UTF-8, UTF-8 with formula escaping, non-UTF-8 encoding, and both combined.

Difference from NimbleCSV: NimbleCSV's dump_to_iodata/1 returns an iodata list (a nested list of small binaries) that callers typically flatten back into a single binary via IO.iodata_to_binary/1 before writing to a file, sending as a download, or passing to an API. RustyCSV skips that roundtrip — it returns the final binary directly, ready for use with IO.binwrite/2, Conn.send_resp/3, :gen_tcp.send/2, File.write/2, etc. The output bytes are identical; there is nothing to traverse or flatten.

Code that pattern-matches on the return value expecting a list will need adjustment. This is a deliberate trade-off: building an iodata list across the NIF boundary requires allocating one Erlang term per field, separator, and newline, which is 18–63% slower and uses 3–6x more NIF memory than returning the bytes directly.

Encoding Strategies

dump_to_iodata/2 accepts a :strategy option:

• default (no option) — Single-threaded SIMD-accelerated encoder. Writes all CSV bytes into a single flat binary. Best for most workloads.

• :parallel — Multi-threaded encoding via rayon. Copies all field data into Rust-owned memory, splits rows into chunks, and encodes each chunk on a separate thread. Returns a short list of large binaries. Best for quoting-heavy data (user-generated content with embedded commas/quotes/newlines).

Example:

# Default (recommended for most cases)
CSV.dump_to_iodata(rows)

# Parallel (opt in for quoting-heavy data)
CSV.dump_to_iodata(rows, strategy: :parallel)

High-Throughput Concurrent Exports

The encoding NIF runs on dirty CPU schedulers with per-thread mimalloc arenas, making it suitable for concurrent export workloads — e.g., thousands of users downloading CSV reports simultaneously:

# Phoenix controller — each request encodes independently
rows = MyApp.Reports.fetch_rows(user_id)
csv = MyCSV.dump_to_iodata(rows)
send_download(conn, {:binary, csv}, filename: "report.csv")

For very large exports, use chunked NIF encoding for bounded memory:

MyApp.Reports.stream_rows(user_id)
|> Stream.chunk_every(5_000)
|> Stream.map(&MyCSV.dump_to_iodata/1)
|> Enum.each(&Conn.chunk(conn, &1))

NimbleCSV Compatibility

RustyCSV is designed as a drop-in replacement for NimbleCSV. The API is identical:

• parse_string/2 - Parse CSV string to list of rows
• parse_stream/2 - Lazily parse a stream
• parse_enumerable/2 - Parse any enumerable
• dump_to_iodata/2 - Convert rows to iodata (returns a flat binary, not an iodata list — see "Encoding" section)
• dump_to_stream/1 - Lazily convert rows to iodata stream
• to_line_stream/1 - Convert arbitrary chunks to lines
• options/0 - Return module configuration

RustyCSV extends NimbleCSV with additional options:

• :strategy on parse_string/2 - Select the parsing approach (:simd, :basic, :indexed, :parallel, :zero_copy)
• :strategy on dump_to_iodata/2 - Select the encoding approach (default or :parallel)
• :headers - Return rows as maps instead of lists

Headers-to-Maps

Use the :headers option to get maps instead of lists:

CSV.parse_string("name,age\njohn,27\n", headers: true)
#=> [%{"name" => "john", "age" => "27"}]

CSV.parse_string("name,age\njohn,27\n", headers: [:name, :age])
#=> [%{name: "john", age: "27"}]

CSV.parse_string("name,age\njohn,27\n", headers: ["n", "a"])
#=> [%{"n" => "john", "a" => "27"}]

Streaming also supports headers:

"huge.csv"
|> File.stream!()
|> CSV.parse_stream(headers: true)
|> Stream.each(&process_map/1)
|> Stream.run()

How :headers interacts with :skip_headers

With headers: true, the first row is always consumed as keys — :skip_headers has no effect.

With headers: [keys], the :skip_headers option controls whether the first row is skipped (default: true). Most CSV files have a header row, so skipping it avoids mapping the header row itself into a map. If your file has no header row, pass skip_headers: false:

# File with header row (typical) — first row skipped by default
CSV.parse_string("name,age\njohn,27\n", headers: [:n, :a])
#=> [%{n: "john", a: "27"}]

# File without header row — include all rows
CSV.parse_string("john,27\njane,30\n", headers: [:n, :a], skip_headers: false)
#=> [%{n: "john", a: "27"}, %{n: "jane", a: "30"}]

Edge cases

• Fewer columns than keys — missing values are nil
• More columns than keys — extra columns are ignored
• Duplicate headers — last column wins
• Empty header field — key is ""

Multi-Separator Support

Like NimbleCSV, RustyCSV supports multiple separator characters. Separators can be single-byte or multi-byte:

RustyCSV.define(MyParser,
  separator: [",", ";"],
  escape: "\""
)

# Any separator in the list is recognized when parsing
MyParser.parse_string("a,b;c\\n1;2,3\\n", skip_headers: false)
#=> [["a", "b", "c"], ["1", "2", "3"]]

# Only the FIRST separator is used when dumping
MyParser.dump_to_iodata([["a", "b", "c"]]) |> IO.iodata_to_binary()
#=> "a,b,c\\n"

Multi-byte separators are supported:

RustyCSV.define(MyParser,
  separator: "::",
  escape: "\""
)

MyParser.parse_string("a::b::c\\n", skip_headers: false)
#=> [["a", "b", "c"]]

You can also mix single-byte and multi-byte separators:

RustyCSV.define(MyParser,
  separator: [",", "::"],
  escape: "\""
)

Multi-Byte Escape Support

Escape sequences can also be multi-byte:

RustyCSV.define(MyParser,
  separator: ",",
  escape: "$$"
)

MyParser.parse_string("$$hello$$,world\\n", skip_headers: false)
#=> [["hello", "world"]]

Encoding Support

RustyCSV supports character encoding conversion via the :encoding option. This is useful when exporting CSVs with non-ASCII characters (accents, CJK, emoji) that need to open correctly in spreadsheet applications:

alias RustyCSV.Spreadsheet

# Export data with international characters for Excel/Google Sheets/Numbers
rows = [["名前", "年齢"], ["田中", "27"], ["Müller", "35"]]
csv = Spreadsheet.dump_to_iodata(rows) |> IO.iodata_to_binary()
File.write!("export.csv", csv)

The pre-defined RustyCSV.Spreadsheet module outputs UTF-16 LE with BOM, which spreadsheet applications auto-detect correctly. You can also define custom encodings:

RustyCSV.define(MySpreadsheet,
  separator: "\t",
  encoding: {:utf16, :little},
  trim_bom: true,
  dump_bom: true
)

Supported encodings:

• :utf8 - UTF-8 (default, no conversion overhead)
• :latin1 - ISO-8859-1 / Latin-1
• {:utf16, :little} - UTF-16 Little Endian
• {:utf16, :big} - UTF-16 Big Endian
• {:utf32, :little} - UTF-32 Little Endian
• {:utf32, :big} - UTF-32 Big Endian