@spec from_binary(
  binary(),
  :float | :integer | :boolean | :date | :time | :datetime,
  keyword()
) :: t()

Builds a series of dtype from binary.

All binaries must be in native endianness.

options

Options

• :backend - The backend to allocate the series on.

examples

Examples

Integers and floats follow their native encoding:

iex> Explorer.Series.from_binary(<<1.0::float-64-native, 2.0::float-64-native>>, :float)
#Explorer.Series<
  Polars[2]
  float [1.0, 2.0]
>

iex> Explorer.Series.from_binary(<<-1::signed-64-native, 1::signed-64-native>>, :integer)
#Explorer.Series<
  Polars[2]
  integer [-1, 1]
>

Booleans are unsigned integers:

iex> Explorer.Series.from_binary(<<1, 0, 1>>, :boolean)
#Explorer.Series<
  Polars[3]
  boolean [true, false, true]
>

Dates are encoded as i32 representing days from the Unix epoch (1970-01-01):

iex> binary = <<-719162::signed-32-native, 0::signed-32-native, 6129::signed-32-native>>
iex> Explorer.Series.from_binary(binary, :date)
#Explorer.Series<
  Polars[3]
  date [0001-01-01, 1970-01-01, 1986-10-13]
>

Times are encoded as i64 representing microseconds from midnight:

iex> binary = <<0::signed-64-native, 86399999999::signed-64-native>>
iex> Explorer.Series.from_binary(binary, :time)
#Explorer.Series<
  Polars[2]
  time [00:00:00.000000, 23:59:59.999999]
>

Datetimes are encoded as i64 representing microseconds from the Unix epoch (1970-01-01):

iex> binary = <<0::signed-64-native, 529550625987654::signed-64-native>>
iex> Explorer.Series.from_binary(binary, :datetime)
#Explorer.Series<
  Polars[2]
  datetime [1970-01-01 00:00:00.000000, 1986-10-13 01:23:45.987654]
>

@spec from_list(list :: list(), opts :: Keyword.t()) :: t()

Creates a new series from a list.

The list must consist of a single data type and nils. It is possible to have a list of only nil values. In this case, the list will have the :dtype of float.

options

Options

• :backend - The backend to allocate the series on.
• :dtype - Cast the series to a given :dtype. By default this is nil, which means that Explorer will infer the type from the values in the list.

examples

Examples

Explorer will infer the type from the values in the list:

iex> Explorer.Series.from_list([1, 2, 3])
#Explorer.Series<
  Polars[3]
  integer [1, 2, 3]
>

Series are nullable, so you may also include nils:

iex> Explorer.Series.from_list([1.0, nil, 2.5, 3.1])
#Explorer.Series<
  Polars[4]
  float [1.0, nil, 2.5, 3.1]
>

A mix of integers and floats will be cast to a float:

iex> Explorer.Series.from_list([1, 2.0])
#Explorer.Series<
  Polars[2]
  float [1.0, 2.0]
>

Floats series can accept NaN, Inf, and -Inf values:

iex> Explorer.Series.from_list([1.0, 2.0, :nan, 4.0])
#Explorer.Series<
  Polars[4]
  float [1.0, 2.0, NaN, 4.0]
>

iex> Explorer.Series.from_list([1.0, 2.0, :infinity, 4.0])
#Explorer.Series<
  Polars[4]
  float [1.0, 2.0, Inf, 4.0]
>

iex> Explorer.Series.from_list([1.0, 2.0, :neg_infinity, 4.0])
#Explorer.Series<
  Polars[4]
  float [1.0, 2.0, -Inf, 4.0]
>

Trying to create a "nil" series will, by default, result in a series of floats:

iex> Explorer.Series.from_list([nil, nil])
#Explorer.Series<
  Polars[2]
  float [nil, nil]
>

You can specify the desired dtype for a series with the :dtype option.

iex> Explorer.Series.from_list([nil, nil], dtype: :integer)
#Explorer.Series<
  Polars[2]
  integer [nil, nil]
>

iex> Explorer.Series.from_list([1, nil], dtype: :string)
#Explorer.Series<
  Polars[2]
  string ["1", nil]
>

The dtype option is particulary important if a :binary series is desired, because by default binary series will have the dtype of :string:

iex> Explorer.Series.from_list([<<228, 146, 51>>, <<42, 209, 236>>], dtype: :binary)
#Explorer.Series<
  Polars[2]
  binary [<<228, 146, 51>>, <<42, 209, 236>>]
>

A series mixing UTF8 strings and binaries is possible:

iex> Explorer.Series.from_list([<<228, 146, 51>>, "Elixir"], dtype: :binary)
#Explorer.Series<
  Polars[2]
  binary [<<228, 146, 51>>, "Elixir"]
>

Another option is to create a categorical series from a list of strings:

iex> Explorer.Series.from_list(["EUA", "Brazil", "Poland"], dtype: :category)
#Explorer.Series<
  Polars[3]
  category ["EUA", "Brazil", "Poland"]
>

It is possible to create a series of :datetime from a list of microseconds since Unix Epoch.

iex> Explorer.Series.from_list([1649883642 * 1_000 * 1_000], dtype: :datetime)
#Explorer.Series<
  Polars[1]
  datetime [2022-04-13 21:00:42.000000]
>

It is possible to create a series of :time from a list of microseconds since midnight.

iex> Explorer.Series.from_list([123 * 1_000 * 1_000], dtype: :time)
#Explorer.Series<
  Polars[1]
  time [00:02:03.000000]
>

Mixing non-numeric data types will raise an ArgumentError:

iex> Explorer.Series.from_list([1, "a"])
** (ArgumentError) the value "a" does not match the inferred series dtype :integer

@spec from_tensor(tensor :: Nx.Tensor.t(), opts :: Keyword.t()) :: t()

Converts a Nx.Tensor.t/0 to a series.

Warning

Nx is an optional dependency. You will need to ensure it's installed to use this function.

options

Options

• :backend - The backend to allocate the series on.
• :dtype - The dtype of the series, it must match the underlying tensor type.

examples

Examples

Integers and floats:

iex> tensor = Nx.tensor([1, 2, 3])
iex> Explorer.Series.from_tensor(tensor)
#Explorer.Series<
  Polars[3]
  integer [1, 2, 3]
>

iex> tensor = Nx.tensor([1.0, 2.0, 3.0], type: :f64)
iex> Explorer.Series.from_tensor(tensor)
#Explorer.Series<
  Polars[3]
  float [1.0, 2.0, 3.0]
>

Unsigned 8-bit tensors are assumed to be booleans:

iex> tensor = Nx.tensor([1, 0, 1], type: :u8)
iex> Explorer.Series.from_tensor(tensor)
#Explorer.Series<
  Polars[3]
  boolean [true, false, true]
>

Signed 32-bit tensors are assumed to be dates:

iex> tensor = Nx.tensor([-719162, 0, 6129], type: :s32)
iex> Explorer.Series.from_tensor(tensor)
#Explorer.Series<
  Polars[3]
  date [0001-01-01, 1970-01-01, 1986-10-13]
>

Times are signed 64-bit and therefore must have their dtype explicitly given:

iex> tensor = Nx.tensor([0, 86399999999])
iex> Explorer.Series.from_tensor(tensor, dtype: :time)
#Explorer.Series<
  Polars[2]
  time [00:00:00.000000, 23:59:59.999999]
>

Datetimes are signed 64-bit and therefore must have their dtype explicitly given:

iex> tensor = Nx.tensor([0, 529550625987654])
iex> Explorer.Series.from_tensor(tensor, dtype: :datetime)
#Explorer.Series<
  Polars[2]
  datetime [1970-01-01 00:00:00.000000, 1986-10-13 01:23:45.987654]
>

@spec replace(t(), Nx.Tensor.t() | list()) :: t()

Replaces the contents of the given series by the one given in a tensor or list.

The new series will have the same dtype and backend as the current series, but the size may not necessarily match.

tensor-examples

Tensor examples

iex> s = Explorer.Series.from_list([0, 1, 2])
iex> Explorer.Series.replace(s, Nx.tensor([1, 2, 3]))
#Explorer.Series<
  Polars[3]
  integer [1, 2, 3]
>

This is particularly useful for categorical columns:

iex> s = Explorer.Series.from_list(["foo", "bar", "baz"], dtype: :category)
iex> Explorer.Series.replace(s, Nx.tensor([2, 1, 0]))
#Explorer.Series<
  Polars[3]
  category ["baz", "bar", "foo"]
>

list-examples

List examples

Similar to tensors, we can also replace by lists:

iex> s = Explorer.Series.from_list([0, 1, 2])
iex> Explorer.Series.replace(s, [1, 2, 3, 4, 5])
#Explorer.Series<
  Polars[5]
  integer [1, 2, 3, 4, 5]
>

The same considerations as above apply.

@spec to_binary(series :: t()) :: binary()

Returns a series as a fixed-width binary.

This is a shortcut around to_iovec/1. If possible, prefer to use to_iovec/1 as that avoids copying binaries.

examples

Examples

iex> series = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.to_binary(series)
<<1::signed-64-native, 2::signed-64-native, 3::signed-64-native>>

iex> series = Explorer.Series.from_list([true, false, true])
iex> Explorer.Series.to_binary(series)
<<1, 0, 1>>

@spec to_enum(series :: t()) :: Enumerable.t()

Converts a series to an enumerable.

The enumerable will lazily traverse the series.

Warning

You must avoid converting a series to enum, as that will copy the whole series in memory as you traverse it. Prefer to use the operations in this module rather than the ones in Enum whenever possible, as this module is optimized for large series.

examples

Examples

iex> series = Explorer.Series.from_list([1, 2, 3])
iex> series |> Explorer.Series.to_enum() |> Enum.to_list()
[1, 2, 3]

@spec to_iovec(series :: t()) :: [binary()]

Returns a series as a list of fixed-width binaries.

An io vector (iovec) is the Erlang VM term for a flat list of binaries. This is typically a reference to the in-memory representation of the series. If the whole series in contiguous in memory, then the list will have a single element. All binaries are in native endianness.

This operation fails if the series has nil values. Use fill_missing/1 to handle them accordingly.

To retrieve the type of the underlying io vector, use iotype/1. To convert an iovec to a binary, you can use IO.iodata_to_binary/1.

examples

Examples

Integers and floats follow their native encoding:

iex> series = Explorer.Series.from_list([-1, 0, 1])
iex> Explorer.Series.to_iovec(series)
[<<-1::signed-64-native, 0::signed-64-native, 1::signed-64-native>>]

iex> series = Explorer.Series.from_list([1.0, 2.0, 3.0])
iex> Explorer.Series.to_iovec(series)
[<<1.0::float-64-native, 2.0::float-64-native, 3.0::float-64-native>>]

Booleans are encoded as 0 and 1:

iex> series = Explorer.Series.from_list([true, false, true])
iex> Explorer.Series.to_iovec(series)
[<<1, 0, 1>>]

Dates are encoded as i32 representing days from the Unix epoch (1970-01-01):

iex> series = Explorer.Series.from_list([~D[0001-01-01], ~D[1970-01-01], ~D[1986-10-13]])
iex> Explorer.Series.to_iovec(series)
[<<-719162::signed-32-native, 0::signed-32-native, 6129::signed-32-native>>]

Times are encoded as i64 representing microseconds from midnight:

iex> series = Explorer.Series.from_list([~T[00:00:00.000000], ~T[23:59:59.999999]])
iex> Explorer.Series.to_iovec(series)
[<<0::signed-64-native, 86399999999::signed-64-native>>]

Datetimes are encoded as i64 representing microseconds from the Unix epoch (1970-01-01):

iex> series = Explorer.Series.from_list([~N[0001-01-01 00:00:00], ~N[1970-01-01 00:00:00], ~N[1986-10-13 01:23:45.987654]])
iex> Explorer.Series.to_iovec(series)
[<<-62135596800000000::signed-64-native, 0::signed-64-native, 529550625987654::signed-64-native>>]

The operation raises for binaries and strings, as they do not provide a fixed-width binary representation:

iex> s = Explorer.Series.from_list(["a", "b", "c", "b"])
iex> Explorer.Series.to_iovec(s)
** (ArgumentError) cannot convert series of dtype :string into iovec

However, if appropriate, you can convert them to categorical types, which will then return the index of each category:

iex> series = Explorer.Series.from_list(["a", "b", "c", "b"], dtype: :category)
iex> Explorer.Series.to_iovec(series)
[<<0::unsigned-32-native, 1::unsigned-32-native, 2::unsigned-32-native, 1::unsigned-32-native>>]

@spec to_list(series :: t()) :: list()

Converts a series to a list.

Warning

You must avoid converting a series to list, as that requires copying the whole series in memory. Prefer to use the operations in this module rather than the ones in Enum whenever possible, as this module is optimized for large series.

examples

Examples

iex> series = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.to_list(series)
[1, 2, 3]

@spec to_tensor(series :: t(), tensor_opts :: Keyword.t()) :: Nx.Tensor.t()

Converts a series to a Nx.Tensor.t/0.

Note that Explorer.Series are automatically converted to tensors once give to numerical definitions. The tensor type is given by iotype/1.

Warning

Nx is an optional dependency. You will need to ensure it's installed to use this function.

options

Options

• :backend - the Nx backend to allocate the tensor on

examples

Examples

iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.to_tensor(s)
#Nx.Tensor<
  s64[3]
  [1, 2, 3]
>

iex> s = Explorer.Series.from_list([true, false, true])
iex> Explorer.Series.to_tensor(s)
#Nx.Tensor<
  u8[3]
  [1, 0, 1]
>