NimbleCSV is a small and fast parsing and dumping library.

It works by building highly-inlined CSV parsers, designed to work with strings, enumerables and streams. At the top of your file (and not inside a function), you can define your own parser module:

NimbleCSV.define(MyParser, separator: "\t", escape: "\"")

Once defined, we can parse data accordingly:

iex> MyParser.parse_string "name\tage\njohn\t27"
[["john","27"]]

See the define/2 function for the list of functions that would be defined in MyParser.

Parsing

NimbleCSV is by definition restricted in scope to do only parsing (and dumping). For example, the example above discarded the headers when parsing the string, as NimbleCSV expects developers to handle those explicitly later. For example:

"name\tage\njohn\t27"
|> MyParser.parse_string
|> Enum.map(fn [name, age] ->
  %{name: name, age: String.to_integer(age)}
end)

This is particularly useful with the parse_stream functionality that receives and returns a stream. For example, we can use it to parse files line by line lazily:

"path/to/csv/file"
|> File.stream!(read_ahead: 100_000)
|> MyParser.parse_stream
|> Stream.map(fn [name, age] ->
  %{name: name, age: String.to_integer(age)}
end)

By default this library ships with NimbleCSV.RFC4180, which is the most common implementation of CSV parsing/dumping available using comma as separators and double-quote as escape. If you want to use it in your codebase, simply alias it to CSV and enjoy:

iex> alias NimbleCSV.RFC4180, as: CSV
iex> CSV.parse_string "name,age\njohn,27"
[["john","27"]]

Binary references

One of the reasons behind NimbleCSV performance is that it performs parsing by matching on binaries and extracting those fields as binary references. Therefore if you have a row such as:

one,two,three,four,five

NimbleCSV will return a list of ["one", "two", "three", "four", "five"] where each element references the original row. For this reason, if you plan to keep the parsed data around in the parsing process or even send it to another process, you may want to copy the data before doing the transfer.

For example, in the parse_stream example in the previous section, we could rewrite the Stream.map/2 operation to explicitly copy any field that is stored as a binary:

"path/to/csv/file"
|> File.stream!(read_ahead: 100_000)
|> MyParser.parse_stream
|> Stream.map(fn [name, age] ->
  %{name: :binary.copy(name),
    age: String.to_integer(age)}
end)

Dumping

NimbleCSV can dump any enumerable to either iodata or to streams:

iex> IO.iodata_to_binary MyParser.dump_to_iodata([~w(name age), ~w(mary 28)])
"name\tage\nmary\t28\n"

iex> MyParser.dump_to_stream([~w(name age), ~w(mary 28)])
#Stream<...>