View Source Explorer.Series (Explorer v0.8.2)
The Series struct and API.
A series can be of the following data types:
:binary- Binaries (sequences of bytes):boolean- Boolean:category- Strings but represented internally as integers:date- Date type that unwraps toElixir.Date{:datetime, precision}- DateTime type with millisecond/microsecond/nanosecond precision that unwraps toElixir.NaiveDateTime{:duration, precision}- Duration type with millisecond/microsecond/nanosecond precision that unwraps toExplorer.Duration{:f, size}- a 64-bit or 32-bit floating point number{:s, size}- a 8-bit or 16-bit or 32-bit or 64-bit signed integer number.{:u, size}- a 8-bit or 16-bit or 32-bit or 64-bit unsigned integer number.:null-nils exclusively:string- UTF-8 encoded binary:time- Time type that unwraps toElixir.Time{:list, dtype}- A recursive dtype that can store lists. Examples:{:list, :boolean}or a nested list dtype like{:list, {:list, :boolean}}.{:struct, [{key, dtype}]}- A recursive dtype that can store Arrow/Polars structs (not to be confused with Elixir's struct). This type unwraps to Elixir maps with string keys. Examples:{:struct, [{"a", :string}]}or a nested struct dtype like{:struct, [{"a", {:struct, [{"b", :string}]}}]}.
When passing a dtype as argument, aliases are supported for convenience and compatibility with the Elixir ecosystem:
- All numeric dtypes (signed integer, unsigned integer, and floats) can
be specified as an atom in the form of
:s32,:u8,:f32and o son - The atom
:floatas an alias for{:f, 64}to mirror Elixir's floats - The atom
:integeras an alias for{:s, 64}to mirror Elixir's integers
A series must consist of a single data type only. Series may have nil values in them.
The series dtype can be retrieved via the dtype/1 function or directly accessed as
series.dtype. A series.name field is also available, but it is always nil unless
the series is retrieved from a dataframe.
Many functions only apply to certain dtypes. These functions may appear on distinct categories on the sidebar. Other functions may work on several datatypes, such as comparison functions. In such cases, a "Supported dtypes" section will be available in the function documentation.
Creating series
Series can be created using from_list/2, from_binary/3, and friends:
Series can be made of numbers:
iex> Explorer.Series.from_list([1, 2, 3])
#Explorer.Series<
Polars[3]
s64 [1, 2, 3]
>Series are nullable, so you may also include nils:
iex> Explorer.Series.from_list([1.0, nil, 2.5, 3.1])
#Explorer.Series<
Polars[4]
f64 [1.0, nil, 2.5, 3.1]
>Any of the dtypes above are supported, such as strings:
iex> Explorer.Series.from_list(["foo", "bar", "baz"])
#Explorer.Series<
Polars[3]
string ["foo", "bar", "baz"]
>Casting numeric series (type promotion)
Series of integers and floats are automatically cast when executing certain operations. For example, adding a series of s64 with f64 will return a list of f64.
Numeric casting works like this:
when working with the same numeric type but of different precisions, the higher precision wins
when working with unsigned integers and signed integers, unsigned integers are cast to signed integers using double of its precision (maximum of 64 bits)
when working with integers and floats, integers are always cast floats, keep the floating number precision
Series queries
DataFrames have named columns, so their queries use column names as variables:
iex> require Explorer.DataFrame
iex> df = Explorer.DataFrame.new(col_name: [1, 2, 3])
iex> Explorer.DataFrame.filter(df, col_name > 2)
#Explorer.DataFrame<
Polars[1 x 1]
col_name s64 [3]
>Series have no named columns (a series constitutes a single column,
so no name is required). This means their queries can't use column
names as variables. Instead, series queries use the special _ variable like so:
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.filter(s, _ > 2)
#Explorer.Series<
Polars[1]
s64 [3]
>Summary
Functions: Conversion
Builds a series of dtype from binary.
Creates a new series from a list.
Converts a Nx.Tensor.t/0 to a series.
Replaces the contents of the given series by the one given in a tensor or list.
Returns a series as a fixed-width binary.
Converts a series to an enumerable.
Returns a series as a list of fixed-width binaries.
Converts a series to a list.
Converts a series to a Nx.Tensor.t/0.
Functions: Aggregation
Returns if all the values in a boolean series are true.
Returns if any of the values in a boolean series are true.
Gets the index of the maximum value of the series.
Gets the index of the minimum value of the series.
Compute the correlation between two series.
Counts the number of non-nil elements in a series.
Compute the covariance between two series.
Bins values into discrete values.
Creates a new dataframe with unique values and the frequencies of each.
Gets the maximum value of the series.
Gets the mean value of the series.
Gets the median value of the series.
Gets the minimum value of the series.
Gets the most common value(s) of the series.
Returns the number of unique values in the series.
Counts the number of nil elements in a series.
Reduce this Series to the product value.
Bins values into discrete values base on their quantiles.
Gets the given quantile of the series.
Compute the sample skewness of a series.
Gets the standard deviation of the series.
Gets the sum of the series.
Gets the variance of the series.
Functions: Element-wise
Gets the series absolute values.
Adds right to left, element-wise.
Checks equality between two entire series.
Returns a boolean mask of left and right, element-wise.
Cast the series to another type.
Categorise a series of integers or strings according to categories.
Clip (or clamp) the values in a series.
Finds the first non-missing element at each position.
Finds the first non-missing element at each position.
Divides left by right, element-wise.
Returns boolean mask of left == right, element-wise.
Calculates the exponential of all elements.
Picks values based on an Explorer.Query.
Filters a series with a callback function.
Returns boolean mask of left > right, element-wise.
Returns boolean mask of left >= right, element-wise.
Checks if each element of the series in the left exists in the series on the right, returning a boolean mask.
Returns a mask of nil values.
Returns a mask of not nil values.
Returns boolean mask of left < right, element-wise.
Returns boolean mask of left <= right, element-wise.
Calculates the natural logarithm.
Calculates the logarithm on a given base.
Maps values based on an Explorer.Query.
Maps a series with a callback function.
Filters a series with a mask.
Multiplies left and right, element-wise.
Negate the elements of a boolean series.
Returns boolean mask of left != right, element-wise.
Returns a boolean mask of left or right, element-wise.
Returns a boolean mask with true where the 'peaks' (series max or min, default max) are.
Raises a numeric series to the power of the exponent.
Element-wise integer division.
Assign ranks to data with appropriate handling of tied values.
Computes the remainder of an element-wise integer division.
Returns a series of indexes for each item (row) in the series, starting from 0.
Returns a series from two series, based on a predicate.
Converts a datetime series to a string series.
Converts a string series to a datetime series with a given format_string.
Subtracts right from left, element-wise.
Returns an Explorer.Series where each element is the result of invoking fun on each
corresponding element of series.
Functions: Datetime ops
Returns a day-of-week number starting from Monday = 1. (ISO 8601 weekday number)
Returns the day-of-year number starting from 1.
Returns the hour number from 0 to 23.
Returns the minute number from 0 to 59.
Returns the month number starting from 1. The return value ranges from 1 to 12.
Returns the second number from 0 to 59.
Returns the week-of-year number.
Returns the year number in the calendar date.
Functions: Float ops
Computes the the arccosine of a number.
The resultant series is going to be of dtype {:f, 64}, in radians, with values between 0 and pi.
Computes the the arcsine of a number.
The resultant series is going to be of dtype {:f, 64}, in radians, with values between -pi/2 and pi/2.
Computes the the arctangent of a number.
The resultant series is going to be of dtype {:f, 64}, in radians, with values between -pi/2 and pi/2.
Ceil floating point series to highest integers smaller or equal to the float value.
Computes the the cosine of a number (in radians).
The resultant series is going to be of dtype {:f, 64}, with values between 1 and -1.
Floor floating point series to lowest integers smaller or equal to the float value.
Returns a mask of finite values.
Returns a mask of infinite values.
Returns a mask of nan values.
Round floating point series to given decimal places.
Computes the the sine of a number (in radians).
The resultant series is going to be of dtype {:f, 64}, with values between 1 and -1.
Computes the tangent of a number (in radians).
The resultant series is going to be of dtype {:f, 64}.
Functions: String ops
Detects whether a string contains a substring.
Count how many times a substring appears in a string.
Converts all characters to lowercase.
Decodes a string series containing valid JSON according to dtype.
Extracts a string series using a json_path from a series.
Returns a string series where all leading Unicode whitespaces have been removed.
Returns a string series where all leading examples of the provided string have been removed.
Detects whether a string matches a pattern.
Count how many times a pattern matches a string.
Extract all capture groups as a struct for the given regex pattern.
Replaces all occurences of a pattern with replacement in string series.
Scan for all matches for the given regex pattern.
Replaces all occurences of a substring with replacement in string series.
Returns a string series where all trailing Unicode whitespaces have been removed.
Returns a string series where all trailing examples of the provided string have been removed.
Split the string by a substring.
Split a string Series into a struct of string fields.
Returns a string series where all leading and trailing Unicode whitespaces have been removed.
Returns a string series where all leading and trailing examples of the provided string have been removed.
Returns a string sliced from the offset to the end of the string, supporting negative indexing
Returns a string sliced from the offset to the length provided, supporting negative indexing
Converts all characters to uppercase.
Functions: List ops
Join all string items in a sublist and place a separator between them.
Calculates the length of each list in a list series.
Checks for the presence of a value in a list series.
Functions: Struct ops
Extracts a field from a struct series
Functions: Introspection
Return a series with the category names of a categorical series.
Returns the data type of the series.
Returns the type of the underlying fixed-width binary representation.
Returns the number of elements in the series.
Functions: Shape
Returns the indices that would sort the series.
Returns the value of the series at the given index.
Takes every nth value in this series, returned as a new series.
Concatenate one or more series.
Concatenate two series.
Returns the unique values of the series.
Returns the first element of the series.
Returns a string series with all values concatenated.
Returns the first N elements of the series.
Returns the last element of the series.
Reverses the series order.
Returns a random sample of the series.
Shifts series by offset with nil values.
Change the elements order randomly.
Slices the elements at the given indices as a new series.
Returns a slice of the series, with size elements starting at offset.
Sorts the series.
Sorts the series based on an expression.
Sorts the series based on a callback that returns a lazy series.
Returns the last N elements of the series.
Returns the unique values of the series, but does not maintain order.
Functions: Window
Calculates the cumulative maximum of the series.
Calculates the cumulative minimum of the series.
Calculates the cumulative product of the series.
Calculates the cumulative sum of the series.
Calculate the exponentially weighted moving average, given smoothing factor alpha.
Calculate the exponentially weighted moving standard deviation, given smoothing factor alpha.
Calculate the exponentially weighted moving variance, given smoothing factor alpha.
Fill missing values with the given strategy. If a scalar value is provided instead of a strategy
atom, nil will be replaced with that value. It must be of the same dtype as the series.
Calculate the rolling max, given a window size and optional list of weights.
Calculate the rolling mean, given a window size and optional list of weights.
Calculate the rolling median, given a window size and optional list of weights.
Calculate the rolling min, given a window size and optional list of weights.
Calculate the rolling standard deviation, given a window size and optional list of weights.
Calculate the rolling sum, given a window size and optional list of weights.
Functions: Conversion
@spec from_binary( binary(), :boolean | :date | :time | datetime_dtype() | duration_dtype() | float_dtype() | float_dtype_alias() | integer_dtype_alias() | signed_integer_dtype() | unsigned_integer_dtype(), keyword() ) :: t()
Builds a series of dtype from binary.
All binaries must be in native endianness.
Options
:backend- The backend to allocate the series on.
Examples
Integers and floats follow their native encoding:
iex> Explorer.Series.from_binary(<<1.0::float-64-native, 2.0::float-64-native>>, {:f, 64})
#Explorer.Series<
Polars[2]
f64 [1.0, 2.0]
>
iex> Explorer.Series.from_binary(<<-1::signed-64-native, 1::signed-64-native>>, :integer)
#Explorer.Series<
Polars[2]
s64 [-1, 1]
>Booleans are unsigned integers:
iex> Explorer.Series.from_binary(<<1, 0, 1>>, :boolean)
#Explorer.Series<
Polars[3]
boolean [true, false, true]
>Dates are encoded as s32 representing days from the Unix epoch (1970-01-01):
iex> binary = <<-719162::signed-32-native, 0::signed-32-native, 6129::signed-32-native>>
iex> Explorer.Series.from_binary(binary, :date)
#Explorer.Series<
Polars[3]
date [0001-01-01, 1970-01-01, 1986-10-13]
>Times are encoded as s64 representing nanoseconds from midnight:
iex> binary = <<0::signed-64-native, 86399999999000::signed-64-native>>
iex> Explorer.Series.from_binary(binary, :time)
#Explorer.Series<
Polars[2]
time [00:00:00.000000, 23:59:59.999999]
>Datetimes are encoded as s64 representing microseconds from the Unix epoch (1970-01-01):
iex> binary = <<0::signed-64-native, 529550625987654::signed-64-native>>
iex> Explorer.Series.from_binary(binary, {:datetime, :microsecond})
#Explorer.Series<
Polars[2]
datetime[μs] [1970-01-01 00:00:00.000000, 1986-10-13 01:23:45.987654]
>Creates a new series from a list.
The list must consist of a single data type and nils. It is possible to have
a list of only nil values. In this case, the list will have the :dtype of :null.
Options
:backend- The backend to allocate the series on.:dtype- Cast the series to a given:dtype. By default this isnil, which means that Explorer will infer the type from the values in the list. See the module docs for the list of valid dtypes and aliases.
Examples
Explorer will infer the type from the values in the list.
Integers are always treated as s64 and floats are always
treated as f64:
iex> Explorer.Series.from_list([1, 2, 3])
#Explorer.Series<
Polars[3]
s64 [1, 2, 3]
>
iex> Explorer.Series.from_list([1.0, 2.0, 3.0])
#Explorer.Series<
Polars[3]
f64 [1.0, 2.0, 3.0]
>Series are nullable, so you may also include nils:
iex> Explorer.Series.from_list([1.0, nil, 2.5, 3.1])
#Explorer.Series<
Polars[4]
f64 [1.0, nil, 2.5, 3.1]
>A mix of integers and floats will be cast to a float:
iex> Explorer.Series.from_list([1, 2.0])
#Explorer.Series<
Polars[2]
f64 [1.0, 2.0]
>Floats series can accept NaN, Inf, and -Inf values:
iex> Explorer.Series.from_list([1.0, 2.0, :nan, 4.0])
#Explorer.Series<
Polars[4]
f64 [1.0, 2.0, NaN, 4.0]
>
iex> Explorer.Series.from_list([1.0, 2.0, :infinity, 4.0])
#Explorer.Series<
Polars[4]
f64 [1.0, 2.0, Inf, 4.0]
>
iex> Explorer.Series.from_list([1.0, 2.0, :neg_infinity, 4.0])
#Explorer.Series<
Polars[4]
f64 [1.0, 2.0, -Inf, 4.0]
>Trying to create an empty series or a series of nils will, by default,
result in a series of :null type:
iex> Explorer.Series.from_list([])
#Explorer.Series<
Polars[0]
null []
>
iex> Explorer.Series.from_list([nil, nil])
#Explorer.Series<
Polars[2]
null [nil, nil]
>A list of Date, Time, NaiveDateTime, and Explorer.Duration structs
are also supported, and they will become series with the respective dtypes:
:date, :time, {:datetime, :microsecond}, and {:duration, precision}.
For example:
iex> Explorer.Series.from_list([~D[0001-01-01], ~D[1970-01-01], ~D[1986-10-13]])
#Explorer.Series<
Polars[3]
date [0001-01-01, 1970-01-01, 1986-10-13]
>You can specify the desired dtype for a series with the :dtype option.
iex> Explorer.Series.from_list([nil, nil], dtype: :integer)
#Explorer.Series<
Polars[2]
s64 [nil, nil]
>
iex> Explorer.Series.from_list([1, nil], dtype: :string)
#Explorer.Series<
Polars[2]
string ["1", nil]
>
iex> Explorer.Series.from_list([1, 2], dtype: :f32)
#Explorer.Series<
Polars[2]
f32 [1.0, 2.0]
>
iex> Explorer.Series.from_list([1, nil, 2], dtype: :float)
#Explorer.Series<
Polars[3]
f64 [1.0, nil, 2.0]
>The dtype option is particulary important if a :binary series
is desired, because by default binary series will have the dtype
of :string:
iex> Explorer.Series.from_list([<<228, 146, 51>>, <<42, 209, 236>>], dtype: :binary)
#Explorer.Series<
Polars[2]
binary [<<228, 146, 51>>, <<42, 209, 236>>]
>A series mixing UTF8 strings and binaries is possible:
iex> Explorer.Series.from_list([<<228, 146, 51>>, "Elixir"], dtype: :binary)
#Explorer.Series<
Polars[2]
binary [<<228, 146, 51>>, "Elixir"]
>Another option is to create a categorical series from a list of strings:
iex> Explorer.Series.from_list(["EUA", "Brazil", "Poland"], dtype: :category)
#Explorer.Series<
Polars[3]
category ["EUA", "Brazil", "Poland"]
>It is possible to create a series of :datetime from a list of microseconds since Unix Epoch.
iex> Explorer.Series.from_list([1649883642 * 1_000 * 1_000], dtype: {:datetime, :microsecond})
#Explorer.Series<
Polars[1]
datetime[μs] [2022-04-13 21:00:42.000000]
>It is possible to create a series of :time from a list of nanoseconds since midnight.
iex> Explorer.Series.from_list([123 * 1_000 * 1_000 * 1_000], dtype: :time)
#Explorer.Series<
Polars[1]
time [00:02:03.000000]
>Mixing non-numeric data types will raise an ArgumentError:
iex> Explorer.Series.from_list([1, "a"])
** (ArgumentError) the value "a" does not match the inferred dtype {:s, 64}@spec from_tensor(tensor :: Nx.Tensor.t(), opts :: Keyword.t()) :: t()
Converts a Nx.Tensor.t/0 to a series.
Warning
Nxis an optional dependency. You will need to ensure it's installed to use this function.
Options
:backend- The backend to allocate the series on.:dtype- The dtype of the series that must match the underlying tensor type.The series can have a different dtype if the tensor is compatible with it. For example, a tensor of
{:u, 8}can represent a series of:booleandtype.Here are the list of compatible tensor types and dtypes:
{:u, 8}tensor as a:booleanseries.{:s, 32}tensor as a:dateseries.{:s, 64}tensor as a:timeseries.{:s, 64}tensor as a{:datetime, unit}or{:duration, unit}series.
Examples
Integers and floats:
iex> tensor = Nx.tensor([1, 2, 3])
iex> Explorer.Series.from_tensor(tensor)
#Explorer.Series<
Polars[3]
s64 [1, 2, 3]
>
iex> tensor = Nx.tensor([1.0, 2.0, 3.0], type: :f64)
iex> Explorer.Series.from_tensor(tensor)
#Explorer.Series<
Polars[3]
f64 [1.0, 2.0, 3.0]
>
iex> tensor = Nx.tensor([1, 0, 1], type: :u8)
iex> Explorer.Series.from_tensor(tensor)
#Explorer.Series<
Polars[3]
u8 [1, 0, 1]
>
iex> tensor = Nx.tensor([-719162, 0, 6129], type: :s32)
iex> Explorer.Series.from_tensor(tensor)
#Explorer.Series<
Polars[3]
s32 [-719162, 0, 6129]
>Booleans can be read from a tensor of {:u, 8} type if the dtype is explicitly given:
iex> tensor = Nx.tensor([1, 0, 1], type: :u8)
iex> Explorer.Series.from_tensor(tensor, dtype: :boolean)
#Explorer.Series<
Polars[3]
boolean [true, false, true]
>Times are signed 64-bit representing nanoseconds from midnight and therefore must have their dtype explicitly given:
iex> tensor = Nx.tensor([0, 86399999999000])
iex> Explorer.Series.from_tensor(tensor, dtype: :time)
#Explorer.Series<
Polars[2]
time [00:00:00.000000, 23:59:59.999999]
>Datetimes are signed 64-bit and therefore must have their dtype explicitly given:
iex> tensor = Nx.tensor([0, 529550625987654])
iex> Explorer.Series.from_tensor(tensor, dtype: {:datetime, :microsecond})
#Explorer.Series<
Polars[2]
datetime[μs] [1970-01-01 00:00:00.000000, 1986-10-13 01:23:45.987654]
>@spec replace(t(), Nx.Tensor.t() | list()) :: t()
Replaces the contents of the given series by the one given in a tensor or list.
The new series will have the same dtype and backend as the current series, but the size may not necessarily match.
Tensor examples
iex> s = Explorer.Series.from_list([0, 1, 2])
iex> Explorer.Series.replace(s, Nx.tensor([1, 2, 3]))
#Explorer.Series<
Polars[3]
s64 [1, 2, 3]
>This is particularly useful for categorical columns:
iex> s = Explorer.Series.from_list(["foo", "bar", "baz"], dtype: :category)
iex> Explorer.Series.replace(s, Nx.tensor([2, 1, 0]))
#Explorer.Series<
Polars[3]
category ["baz", "bar", "foo"]
>List examples
Similar to tensors, we can also replace by lists:
iex> s = Explorer.Series.from_list([0, 1, 2])
iex> Explorer.Series.replace(s, [1, 2, 3, 4, 5])
#Explorer.Series<
Polars[5]
s64 [1, 2, 3, 4, 5]
>The same considerations as above apply.
Returns a series as a fixed-width binary.
This is a shortcut around to_iovec/1. If possible, prefer
to use to_iovec/1 as that avoids copying binaries.
Examples
iex> series = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.to_binary(series)
<<1::signed-64-native, 2::signed-64-native, 3::signed-64-native>>
iex> series = Explorer.Series.from_list([true, false, true])
iex> Explorer.Series.to_binary(series)
<<1, 0, 1>>@spec to_enum(series :: t()) :: Enumerable.t()
Converts a series to an enumerable.
The enumerable will lazily traverse the series.
Warning
You must avoid converting a series to enum, as that will copy the whole series in memory as you traverse it. Prefer to use the operations in this module rather than the ones in
Enumwhenever possible, as this module is optimized for large series.
Examples
iex> series = Explorer.Series.from_list([1, 2, 3])
iex> series |> Explorer.Series.to_enum() |> Enum.to_list()
[1, 2, 3]Returns a series as a list of fixed-width binaries.
An io vector (iovec) is the Erlang VM term for a flat list of binaries.
This is typically a reference to the in-memory representation of the series.
If the whole series in contiguous in memory, then the list will have a single
element. All binaries are in native endianness.
This operation fails if the series has nil values.
Use fill_missing/1 to handle them accordingly.
To retrieve the type of the underlying io vector, use iotype/1.
To convert an iovec to a binary, you can use IO.iodata_to_binary/1.
Examples
Integers and floats follow their native encoding:
iex> series = Explorer.Series.from_list([-1, 0, 1])
iex> Explorer.Series.to_iovec(series)
[<<-1::signed-64-native, 0::signed-64-native, 1::signed-64-native>>]
iex> series = Explorer.Series.from_list([1.0, 2.0, 3.0])
iex> Explorer.Series.to_iovec(series)
[<<1.0::float-64-native, 2.0::float-64-native, 3.0::float-64-native>>]Booleans are encoded as 0 and 1:
iex> series = Explorer.Series.from_list([true, false, true])
iex> Explorer.Series.to_iovec(series)
[<<1, 0, 1>>]Dates are encoded as s32 representing days from the Unix epoch (1970-01-01):
iex> series = Explorer.Series.from_list([~D[0001-01-01], ~D[1970-01-01], ~D[1986-10-13]])
iex> Explorer.Series.to_iovec(series)
[<<-719162::signed-32-native, 0::signed-32-native, 6129::signed-32-native>>]Times are encoded as s64 representing nanoseconds from midnight:
iex> series = Explorer.Series.from_list([~T[00:00:00.000000], ~T[23:59:59.999999]])
iex> Explorer.Series.to_iovec(series)
[<<0::signed-64-native, 86399999999000::signed-64-native>>]Datetimes are encoded as s64 representing their precision from the Unix epoch (1970-01-01):
iex> series = Explorer.Series.from_list([~N[0001-01-01 00:00:00], ~N[1970-01-01 00:00:00], ~N[1986-10-13 01:23:45.987654]])
iex> Explorer.Series.to_iovec(series)
[<<-62135596800000000::signed-64-native, 0::signed-64-native, 529550625987654::signed-64-native>>]The operation raises for binaries and strings, as they do not provide a fixed-width binary representation:
iex> s = Explorer.Series.from_list(["a", "b", "c", "b"])
iex> Explorer.Series.to_iovec(s)
** (ArgumentError) cannot convert series of dtype :string into iovecHowever, if appropriate, you can convert them to categorical types, which will then return the index of each category:
iex> series = Explorer.Series.from_list(["a", "b", "c", "b"], dtype: :category)
iex> Explorer.Series.to_iovec(series)
[<<0::unsigned-32-native, 1::unsigned-32-native, 2::unsigned-32-native, 1::unsigned-32-native>>]Converts a series to a list.
Warning
You must avoid converting a series to list, as that requires copying the whole series in memory. Prefer to use the operations in this module rather than the ones in
Enumwhenever possible, as this module is optimized for large series.
Examples
iex> series = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.to_list(series)
[1, 2, 3]@spec to_tensor(series :: t(), tensor_opts :: Keyword.t()) :: Nx.Tensor.t()
Converts a series to a Nx.Tensor.t/0.
Note that Explorer.Series are automatically converted
to tensors when passed to numerical definitions.
The tensor type is given by iotype/1.
Warning
Nxis an optional dependency. You will need to ensure it's installed to use this function.
Options
:backend- the Nx backend to allocate the tensor on
Examples
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.to_tensor(s)
#Nx.Tensor<
s64[3]
[1, 2, 3]
>
iex> s = Explorer.Series.from_list([true, false, true])
iex> Explorer.Series.to_tensor(s)
#Nx.Tensor<
u8[3]
[1, 0, 1]
>Functions: Aggregation
Returns if all the values in a boolean series are true.
Supported dtypes
:boolean
Examples
iex> s = Series.from_list([true, true, true])
iex> Series.all?(s)
true
iex> s = Series.from_list([true, false, true])
iex> Series.all?(s)
false
iex> s = Series.from_list([1, 2, 3])
iex> Series.all?(s)
** (ArgumentError) Explorer.Series.all?/1 not implemented for dtype {:s, 64}. Valid dtype is :booleanAn empty series will always return true:
iex> s = Series.from_list([], dtype: :boolean)
iex> Series.all?(s)
trueOpposite to Elixir but similar to databases, nil values are ignored:
iex> s = Series.from_list([nil, true, true])
iex> Series.all?(s)
true
iex> s = Series.from_list([nil, nil, nil], dtype: :boolean)
iex> Series.all?(s)
trueReturns if any of the values in a boolean series are true.
Supported dtypes
:boolean
Examples
iex> s = Series.from_list([true, true, true])
iex> Series.any?(s)
true
iex> s = Series.from_list([true, false, true])
iex> Series.any?(s)
true
iex> s = Series.from_list([1, 2, 3])
iex> Series.any?(s)
** (ArgumentError) Explorer.Series.any?/1 not implemented for dtype {:s, 64}. Valid dtype is :booleanAn empty series will always return false:
iex> s = Series.from_list([], dtype: :boolean)
iex> Series.any?(s)
falseOpposite to Elixir but similar to databases, nil values are ignored:
iex> s = Series.from_list([nil, true, true])
iex> Series.any?(s)
true
iex> s = Series.from_list([nil, nil, nil], dtype: :boolean)
iex> Series.any?(s)
false@spec argmax(series :: t()) :: number() | non_finite() | nil
Gets the index of the maximum value of the series.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :date:time:datetime:duration
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 3])
iex> Explorer.Series.argmax(s)
3
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 3.0])
iex> Explorer.Series.argmax(s)
3
iex> s = Explorer.Series.from_list([~D[2021-01-01], ~D[1999-12-31]])
iex> Explorer.Series.argmax(s)
0
iex> s = Explorer.Series.from_list([~N[2021-01-01 00:00:00], ~N[1999-12-31 00:00:00]])
iex> Explorer.Series.argmax(s)
0
iex> s = Explorer.Series.from_list([~T[00:02:03.000212], ~T[00:05:04.000456]])
iex> Explorer.Series.argmax(s)
1
iex> s = Explorer.Series.from_list([], dtype: :integer)
iex> Explorer.Series.argmax(s)
nil
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.argmax(s)
** (ArgumentError) Explorer.Series.argmax/1 not implemented for dtype :string. Valid dtypes are :date, :time, {:datetime, :microsecond}, {:datetime, :millisecond}, {:datetime, :nanosecond}, {:duration, :microsecond}, {:duration, :millisecond}, {:duration, :nanosecond}, {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}@spec argmin(series :: t()) :: number() | non_finite() | nil
Gets the index of the minimum value of the series.
Note that nil is ignored. In case an empty list
or a series whose all elements are nil is used,
the result will be nil.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :date:time:datetime:duration
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 3])
iex> Explorer.Series.argmin(s)
0
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 3.0])
iex> Explorer.Series.argmin(s)
0
iex> s = Explorer.Series.from_list([~D[2021-01-01], ~D[1999-12-31]])
iex> Explorer.Series.argmin(s)
1
iex> s = Explorer.Series.from_list([~N[2021-01-01 00:00:00], ~N[1999-12-31 00:00:00]])
iex> Explorer.Series.argmin(s)
1
iex> s = Explorer.Series.from_list([~T[00:02:03.000212], ~T[00:05:04.000456]])
iex> Explorer.Series.argmin(s)
0
iex> s = Explorer.Series.from_list([], dtype: :integer)
iex> Explorer.Series.argmin(s)
nil
iex> s = Explorer.Series.from_list([nil], dtype: :integer)
iex> Explorer.Series.argmin(s)
nil
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.argmin(s)
** (ArgumentError) Explorer.Series.argmin/1 not implemented for dtype :string. Valid dtypes are :date, :time, {:datetime, :microsecond}, {:datetime, :millisecond}, {:datetime, :nanosecond}, {:duration, :microsecond}, {:duration, :millisecond}, {:duration, :nanosecond}, {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}@spec correlation( left :: t() | number(), right :: t() | number(), opts :: Keyword.t() ) :: float() | non_finite() | nil
Compute the correlation between two series.
The parameter ddof refers to the 'delta degrees of freedom' - the divisor
used in the correlation calculation. Defaults to 1.
The parameter :method refers to the correlation method. The following methods are available:
:pearson: Standard correlation coefficient. (default):spearman: Spearman rank correlation.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s1 = Series.from_list([1, 8, 3])
iex> s2 = Series.from_list([4, 5, 2])
iex> Series.correlation(s1, s2)
0.5447047794019219Counts the number of non-nil elements in a series.
See also:
count_nil/1- counts only thenilelements.size/1- counts all elements.
Examples
Without nil:
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.count(s)
3With nil:
iex> s = Explorer.Series.from_list(["a", nil, "c"])
iex> Explorer.Series.count(s)
2With :nan (:nan does not count as nil):
iex> s = Explorer.Series.from_list([1, :nan, 3])
iex> Explorer.Series.count(s)
3@spec covariance( left :: t() | number(), right :: t() | number(), ddof :: non_neg_integer() ) :: float() | non_finite() | nil
Compute the covariance between two series.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s1 = Series.from_list([1, 8, 3])
iex> s2 = Series.from_list([4, 5, 2])
iex> Series.covariance(s1, s2)
3.0Bins values into discrete values.
Given a bins length of N, there will be N+1 categories.
Options
:labels- The labels assigned to the bins. Givenbinsof length N,:labelsmust be of length N+1. Defaults to the bin bounds (e.g.(-inf -1.0],(-1.0, 1.0],(1.0, inf]):break_point_label- The name given to the breakpoint column. Defaults tobreak_point.:category_label- The name given to the category column. Defaults tocategory.
Examples
iex> s = Explorer.Series.from_list([1.0, 2.0, 3.0])
iex> Explorer.Series.cut(s, [1.5, 2.5])
#Explorer.DataFrame<
Polars[3 x 3]
values f64 [1.0, 2.0, 3.0]
break_point f64 [1.5, 2.5, Inf]
category category ["(-inf, 1.5]", "(1.5, 2.5]", "(2.5, inf]"]
>Creates a new dataframe with unique values and the frequencies of each.
Examples
iex> s = Explorer.Series.from_list(["a", "a", "b", "c", "c", "c"])
iex> Explorer.Series.frequencies(s)
#Explorer.DataFrame<
Polars[3 x 2]
values string ["c", "a", "b"]
counts u32 [3, 2, 1]
>@spec max(series :: t()) :: number() | non_finite() | Date.t() | Time.t() | NaiveDateTime.t() | nil
Gets the maximum value of the series.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :date:time:datetime:duration
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 3])
iex> Explorer.Series.max(s)
3
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 3.0])
iex> Explorer.Series.max(s)
3.0
iex> s = Explorer.Series.from_list([~D[2021-01-01], ~D[1999-12-31]])
iex> Explorer.Series.max(s)
~D[2021-01-01]
iex> s = Explorer.Series.from_list([~N[2021-01-01 00:00:00], ~N[1999-12-31 00:00:00]])
iex> Explorer.Series.max(s)
~N[2021-01-01 00:00:00.000000]
iex> s = Explorer.Series.from_list([~T[00:02:03.000212], ~T[00:05:04.000456]])
iex> Explorer.Series.max(s)
~T[00:05:04.000456]
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.max(s)
** (ArgumentError) Explorer.Series.max/1 not implemented for dtype :string. Valid dtypes are :date, :time, {:datetime, :microsecond}, {:datetime, :millisecond}, {:datetime, :nanosecond}, {:duration, :microsecond}, {:duration, :millisecond}, {:duration, :nanosecond}, {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}@spec mean(series :: t()) :: float() | non_finite() | nil
Gets the mean value of the series.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 3])
iex> Explorer.Series.mean(s)
2.0
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 3.0])
iex> Explorer.Series.mean(s)
2.0
iex> s = Explorer.Series.from_list([~D[2021-01-01], ~D[1999-12-31]])
iex> Explorer.Series.mean(s)
** (ArgumentError) Explorer.Series.mean/1 not implemented for dtype :date. Valid dtypes are {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}@spec median(series :: t()) :: float() | non_finite() | nil
Gets the median value of the series.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 3])
iex> Explorer.Series.median(s)
2.0
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 3.0])
iex> Explorer.Series.median(s)
2.0
iex> s = Explorer.Series.from_list([~D[2021-01-01], ~D[1999-12-31]])
iex> Explorer.Series.median(s)
** (ArgumentError) Explorer.Series.median/1 not implemented for dtype :date. Valid dtypes are {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}@spec min(series :: t()) :: number() | non_finite() | Date.t() | Time.t() | NaiveDateTime.t() | nil
Gets the minimum value of the series.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :date:time:datetime:duration
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 3])
iex> Explorer.Series.min(s)
1
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 3.0])
iex> Explorer.Series.min(s)
1.0
iex> s = Explorer.Series.from_list([~D[2021-01-01], ~D[1999-12-31]])
iex> Explorer.Series.min(s)
~D[1999-12-31]
iex> s = Explorer.Series.from_list([~N[2021-01-01 00:00:00], ~N[1999-12-31 00:00:00]])
iex> Explorer.Series.min(s)
~N[1999-12-31 00:00:00.000000]
iex> s = Explorer.Series.from_list([~T[00:02:03.000451], ~T[00:05:04.000134]])
iex> Explorer.Series.min(s)
~T[00:02:03.000451]
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.min(s)
** (ArgumentError) Explorer.Series.min/1 not implemented for dtype :string. Valid dtypes are :date, :time, {:datetime, :microsecond}, {:datetime, :millisecond}, {:datetime, :nanosecond}, {:duration, :microsecond}, {:duration, :millisecond}, {:duration, :nanosecond}, {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}Gets the most common value(s) of the series.
This function will return multiple values when there's a tie.
Supported dtypes
All except :list and :struct.
Examples
iex> s = Explorer.Series.from_list([1, 2, 2, nil])
iex> Explorer.Series.mode(s)
#Explorer.Series<
Polars[1]
s64 [2]
>
iex> s = Explorer.Series.from_list(["a", "b", "b", "c"])
iex> Explorer.Series.mode(s)
#Explorer.Series<
Polars[1]
string ["b"]
>This function can return multiple entries, but the order is not guaranteed. You may sort the series if desired.
iex> s = Explorer.Series.from_list([1.0, 2.0, 2.0, 3.0, 3.0])
iex> Explorer.Series.mode(s) |> Explorer.Series.sort()
#Explorer.Series<
Polars[2]
f64 [2.0, 3.0]
>Returns the number of unique values in the series.
Examples
iex> s = Explorer.Series.from_list(["a", "b", "a", "b"])
iex> Explorer.Series.n_distinct(s)
2Counts the number of nil elements in a series.
When used in a query on grouped data, count_nil/1 is a per-group operation.
See also:
Examples
Without nils:
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.nil_count(s)
0With nils:
iex> s = Explorer.Series.from_list(["a", nil, "c"])
iex> Explorer.Series.nil_count(s)
1With :nans (:nan does not count as nil):
iex> s = Explorer.Series.from_list([1, :nan, 3])
iex> Explorer.Series.nil_count(s)
0@spec product(series :: t()) :: float() | non_finite() | nil
Reduce this Series to the product value.
Note that an empty series is going to result in a
product of 1. Values that are nil are ignored.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.product(s)
6
iex> s = Explorer.Series.from_list([], dtype: :float)
iex> Explorer.Series.product(s)
1.0
iex> s = Explorer.Series.from_list([true, false, true])
iex> Explorer.Series.product(s)
** (ArgumentError) Explorer.Series.product/1 not implemented for dtype :boolean. Valid dtypes are {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}Bins values into discrete values base on their quantiles.
Given a quantiles length of N, there will be N+1 categories. Each
element of quantiles is expected to be between 0.0 and 1.0.
Options
:labels- The labels assigned to the bins. Givenbinsof length N,:labelsmust be of length N+1. Defaults to the bin bounds (e.g.(-inf -1.0],(-1.0, 1.0],(1.0, inf]):break_point_label- The name given to the breakpoint column. Defaults tobreak_point.:category_label- The name given to the category column. Defaults tocategory.
Examples
iex> s = Explorer.Series.from_list([1.0, 2.0, 3.0, 4.0, 5.0])
iex> Explorer.Series.qcut(s, [0.25, 0.75])
#Explorer.DataFrame<
Polars[5 x 3]
values f64 [1.0, 2.0, 3.0, 4.0, 5.0]
break_point f64 [2.0, 2.0, 4.0, 4.0, Inf]
category category ["(-inf, 2]", "(-inf, 2]", "(2, 4]", "(2, 4]", "(4, inf]"]
>Gets the given quantile of the series.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :date:time:datetime:duration
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 3])
iex> Explorer.Series.quantile(s, 0.2)
1
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 3.0])
iex> Explorer.Series.quantile(s, 0.5)
2.0
iex> s = Explorer.Series.from_list([~D[2021-01-01], ~D[1999-12-31]])
iex> Explorer.Series.quantile(s, 0.5)
~D[2021-01-01]
iex> s = Explorer.Series.from_list([~N[2021-01-01 00:00:00], ~N[1999-12-31 00:00:00]])
iex> Explorer.Series.quantile(s, 0.5)
~N[2021-01-01 00:00:00.000000]
iex> s = Explorer.Series.from_list([~T[01:55:00], ~T[15:35:00], ~T[23:00:00]])
iex> Explorer.Series.quantile(s, 0.5)
~T[15:35:00]
iex> s = Explorer.Series.from_list([true, false, true])
iex> Explorer.Series.quantile(s, 0.5)
** (ArgumentError) Explorer.Series.quantile/2 not implemented for dtype :boolean. Valid dtypes are :date, :time, {:datetime, :microsecond}, {:datetime, :millisecond}, {:datetime, :nanosecond}, {:duration, :microsecond}, {:duration, :millisecond}, {:duration, :nanosecond}, {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}@spec skew(series :: t(), opts :: Keyword.t()) :: float() | non_finite() | nil
Compute the sample skewness of a series.
For normally distributed data, the skewness should be about zero.
For unimodal continuous distributions, a skewness value greater than zero means that there is more weight in the right tail of the distribution.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s = Explorer.Series.from_list([1, 2, 3, 4, 5, 23])
iex> Explorer.Series.skew(s)
1.6727687946848508
iex> s = Explorer.Series.from_list([1, 2, 3, 4, 5, 23])
iex> Explorer.Series.skew(s, bias: false)
2.2905330058490514
iex> s = Explorer.Series.from_list([1, 2, 3, nil, 1])
iex> Explorer.Series.skew(s, bias: false)
0.8545630383279712
iex> s = Explorer.Series.from_list([1, 2, 3, nil, 1])
iex> Explorer.Series.skew(s)
0.49338220021815865
iex> s = Explorer.Series.from_list([true, false, true])
iex> Explorer.Series.skew(s, false)
** (ArgumentError) Explorer.Series.skew/2 not implemented for dtype :boolean. Valid dtypes are {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}@spec standard_deviation(series :: t(), ddof :: non_neg_integer()) :: float() | non_finite() | nil
Gets the standard deviation of the series.
By default, this is the sample standard deviation. This function also takes an optional delta degrees of freedom (ddof). Setting this to zero corresponds to the population sample standard deviation.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 3])
iex> Explorer.Series.standard_deviation(s)
1.0
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 3.0])
iex> Explorer.Series.standard_deviation(s)
1.0
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.standard_deviation(s)
** (ArgumentError) Explorer.Series.standard_deviation/1 not implemented for dtype :string. Valid dtypes are {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}@spec sum(series :: t()) :: number() | non_finite() | nil
Gets the sum of the series.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :boolean
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 3])
iex> Explorer.Series.sum(s)
6
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 3.0])
iex> Explorer.Series.sum(s)
6.0
iex> s = Explorer.Series.from_list([true, false, true])
iex> Explorer.Series.sum(s)
2
iex> s = Explorer.Series.from_list([~D[2021-01-01], ~D[1999-12-31]])
iex> Explorer.Series.sum(s)
** (ArgumentError) Explorer.Series.sum/1 not implemented for dtype :date. Valid dtypes are :boolean, {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}@spec variance(series :: t(), ddof :: non_neg_integer()) :: float() | non_finite() | nil
Gets the variance of the series.
By default, this is the sample variance. This function also takes an optional delta degrees of freedom (ddof). Setting this to zero corresponds to the population variance.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 3])
iex> Explorer.Series.variance(s)
1.0
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 3.0])
iex> Explorer.Series.variance(s)
1.0
iex> s = Explorer.Series.from_list([~N[2021-01-01 00:00:00], ~N[1999-12-31 00:00:00]])
iex> Explorer.Series.variance(s)
** (ArgumentError) Explorer.Series.variance/1 not implemented for dtype {:datetime, :microsecond}. Valid dtypes are {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}Functions: Element-wise
Gets the series absolute values.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s = Explorer.Series.from_list([1, 2, -1, -3])
iex> Explorer.Series.abs(s)
#Explorer.Series<
Polars[4]
s64 [1, 2, 1, 3]
>
iex> s = Explorer.Series.from_list([1.0, 2.0, -1.0, -3.0])
iex> Explorer.Series.abs(s)
#Explorer.Series<
Polars[4]
f64 [1.0, 2.0, 1.0, 3.0]
>
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, -3.0])
iex> Explorer.Series.abs(s)
#Explorer.Series<
Polars[4]
f64 [1.0, 2.0, nil, 3.0]
>
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.abs(s)
** (ArgumentError) Explorer.Series.abs/1 not implemented for dtype :string. Valid dtypes are {:f, 32}, {:f, 64}, {:s, 8}, {:s, 16}, {:s, 32}, {:s, 64}, {:u, 8}, {:u, 16}, {:u, 32} and {:u, 64}@spec add( left :: t() | number() | Date.t() | NaiveDateTime.t() | Explorer.Duration.t(), right :: t() | number() | Date.t() | NaiveDateTime.t() | Explorer.Duration.t() ) :: t()
Adds right to left, element-wise.
When mixing floats and integers, the resulting series will have dtype {:f, 64}.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> s2 = Explorer.Series.from_list([4, 5, 6])
iex> Explorer.Series.add(s1, s2)
#Explorer.Series<
Polars[3]
s64 [5, 7, 9]
>You can also use scalar values on both sides:
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.add(s1, 2)
#Explorer.Series<
Polars[3]
s64 [3, 4, 5]
>
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.add(2, s1)
#Explorer.Series<
Polars[3]
s64 [3, 4, 5]
>Checks equality between two entire series.
Examples
iex> s1 = Explorer.Series.from_list(["a", "b"])
iex> s2 = Explorer.Series.from_list(["a", "b"])
iex> Explorer.Series.all_equal(s1, s2)
true
iex> s1 = Explorer.Series.from_list(["a", "b"])
iex> s2 = Explorer.Series.from_list(["a", "c"])
iex> Explorer.Series.all_equal(s1, s2)
false
iex> s1 = Explorer.Series.from_list(["a", "b"])
iex> s2 = Explorer.Series.from_list([1, 2])
iex> Explorer.Series.all_equal(s1, s2)
falseReturns a boolean mask of left and right, element-wise.
Both sizes must be series, the series must have the same size or at last one of them must have size of 1.
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> mask1 = Explorer.Series.greater(s1, 1)
iex> mask2 = Explorer.Series.less(s1, 3)
iex> Explorer.Series.and(mask1, mask2)
#Explorer.Series<
Polars[3]
boolean [false, true, false]
>Cast the series to another type.
Examples
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.cast(s, :string)
#Explorer.Series<
Polars[3]
string ["1", "2", "3"]
>
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.cast(s, {:f, 64})
#Explorer.Series<
Polars[3]
f64 [1.0, 2.0, 3.0]
>
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.cast(s, :date)
#Explorer.Series<
Polars[3]
date [1970-01-02, 1970-01-03, 1970-01-04]
>Note that time is represented as an integer of nanoseconds since midnight.
In Elixir we can't represent nanoseconds, only microseconds. So be aware that
information can be lost if a conversion is needed (e.g. calling to_list/1).
iex> s = Explorer.Series.from_list([1_000, 2_000, 3_000])
iex> Explorer.Series.cast(s, :time)
#Explorer.Series<
Polars[3]
time [00:00:00.000001, 00:00:00.000002, 00:00:00.000003]
>
iex> s = Explorer.Series.from_list([86399 * 1_000 * 1_000 * 1_000])
iex> Explorer.Series.cast(s, :time)
#Explorer.Series<
Polars[1]
time [23:59:59.000000]
>Note that datetime is represented as an integer of microseconds since Unix Epoch (1970-01-01 00:00:00).
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.cast(s, {:datetime, :microsecond})
#Explorer.Series<
Polars[3]
datetime[μs] [1970-01-01 00:00:00.000001, 1970-01-01 00:00:00.000002, 1970-01-01 00:00:00.000003]
>
iex> s = Explorer.Series.from_list([1649883642 * 1_000 * 1_000])
iex> Explorer.Series.cast(s, {:datetime, :microsecond})
#Explorer.Series<
Polars[1]
datetime[μs] [2022-04-13 21:00:42.000000]
>You can also use cast/2 to categorise a string:
iex> s = Explorer.Series.from_list(["apple", "banana", "apple", "lemon"])
iex> Explorer.Series.cast(s, :category)
#Explorer.Series<
Polars[4]
category ["apple", "banana", "apple", "lemon"]
>cast/2 will return the series as a no-op if you try to cast to the same dtype.
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.cast(s, :integer)
#Explorer.Series<
Polars[3]
s64 [1, 2, 3]
>Categorise a series of integers or strings according to categories.
This function receives a series of integers or strings and convert them into the categories specified by the second argument. The second argument can be one of:
a series with dtype
:category. The integers will be indexes into the categories of the given series (returned bycategories/1)a series with dtype
:string. The integers will be indexes into the series itselfa list of strings. The integers will be indexes into the list
This is going to essentially "copy" the source of categories from the left series
to the right. All members from the left that are not present in the right hand-side
are going to be nil.
If you have a series of strings and you want to convert them into categories,
invoke cast(series, :category) instead.
Examples
If a categorical series is given as second argument, we will extract its categories and map the integers into it:
iex> categories = Explorer.Series.from_list(["a", "b", "c", nil, "a"], dtype: :category)
iex> indexes = Explorer.Series.from_list([0, 2, 1, 0, 2])
iex> Explorer.Series.categorise(indexes, categories)
#Explorer.Series<
Polars[5]
category ["a", "c", "b", "a", "c"]
>Otherwise, if a list of strings or a series of strings is given, they are considered to be the categories series itself:
iex> categories = Explorer.Series.from_list(["a", "b", "c"])
iex> indexes = Explorer.Series.from_list([0, 2, 1, 0, 2])
iex> Explorer.Series.categorise(indexes, categories)
#Explorer.Series<
Polars[5]
category ["a", "c", "b", "a", "c"]
>
iex> indexes = Explorer.Series.from_list([0, 2, 1, 0, 2])
iex> Explorer.Series.categorise(indexes, ["a", "b", "c"])
#Explorer.Series<
Polars[5]
category ["a", "c", "b", "a", "c"]
>Elements that are not mapped to a category will become nil:
iex> indexes = Explorer.Series.from_list([0, 2, nil, 0, 2, 7])
iex> Explorer.Series.categorise(indexes, ["a", "b", "c"])
#Explorer.Series<
Polars[6]
category ["a", "c", nil, "a", "c", nil]
>Strings can be used as "indexes" to create a categorical series with the intersection of members:
iex> strings = Explorer.Series.from_list(["a", "c", nil, "c", "b", "d"])
iex> Explorer.Series.categorise(strings, ["a", "b", "c"])
#Explorer.Series<
Polars[6]
category ["a", "c", nil, "c", "b", nil]
>Clip (or clamp) the values in a series.
Values that fall outside of the interval defined by the min and max
bounds are clipped to the bounds.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Clipping other dtypes are possible using select/3.
Examples
iex> s = Explorer.Series.from_list([-50, 5, nil, 50])
iex> Explorer.Series.clip(s, 1, 10)
#Explorer.Series<
Polars[4]
s64 [1, 5, nil, 10]
>
iex> s = Explorer.Series.from_list([-50, 5, nil, 50])
iex> Explorer.Series.clip(s, 1.5, 10.5)
#Explorer.Series<
Polars[4]
f64 [1.5, 5.0, nil, 10.5]
>Finds the first non-missing element at each position.
Examples
iex> s1 = Explorer.Series.from_list([1, 2, nil, nil])
iex> s2 = Explorer.Series.from_list([1, 2, nil, 4])
iex> s3 = Explorer.Series.from_list([nil, nil, 3, 4])
iex> Explorer.Series.coalesce([s1, s2, s3])
#Explorer.Series<
Polars[4]
s64 [1, 2, 3, 4]
>Finds the first non-missing element at each position.
coalesce(s1, s2) is equivalent to coalesce([s1, s2]).
Examples
iex> s1 = Explorer.Series.from_list([1, nil, 3, nil])
iex> s2 = Explorer.Series.from_list([1, 2, nil, 4])
iex> Explorer.Series.coalesce(s1, s2)
#Explorer.Series<
Polars[4]
s64 [1, 2, 3, 4]
>
iex> s1 = Explorer.Series.from_list(["foo", nil, "bar", nil])
iex> s2 = Explorer.Series.from_list([1, 2, nil, 4])
iex> Explorer.Series.coalesce(s1, s2)
** (ArgumentError) cannot invoke Explorer.Series.coalesce/2 with mismatched dtypes: :string and {:s, 64}Divides left by right, element-wise.
The resulting series will have the dtype as {:f, 64}.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s1 = [10, 10, 10] |> Explorer.Series.from_list()
iex> s2 = [2, 2, 2] |> Explorer.Series.from_list()
iex> Explorer.Series.divide(s1, s2)
#Explorer.Series<
Polars[3]
f64 [5.0, 5.0, 5.0]
>
iex> s1 = [10, 10, 10] |> Explorer.Series.from_list()
iex> Explorer.Series.divide(s1, 2)
#Explorer.Series<
Polars[3]
f64 [5.0, 5.0, 5.0]
>
iex> s1 = [10, 52 ,10] |> Explorer.Series.from_list()
iex> Explorer.Series.divide(s1, 2.5)
#Explorer.Series<
Polars[3]
f64 [4.0, 20.8, 4.0]
>
iex> s1 = [10, 10, 10] |> Explorer.Series.from_list()
iex> s2 = [2, 0, 2] |> Explorer.Series.from_list()
iex> Explorer.Series.divide(s1, s2)
#Explorer.Series<
Polars[3]
f64 [5.0, Inf, 5.0]
>@spec equal( left :: t() | number() | Date.t() | NaiveDateTime.t() | boolean() | String.t(), right :: t() | number() | Date.t() | NaiveDateTime.t() | boolean() | String.t() ) :: t()
Returns boolean mask of left == right, element-wise.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> s2 = Explorer.Series.from_list([1, 2, 4])
iex> Explorer.Series.equal(s1, s2)
#Explorer.Series<
Polars[3]
boolean [true, true, false]
>
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.equal(s, 1)
#Explorer.Series<
Polars[3]
boolean [true, false, false]
>
iex> s = Explorer.Series.from_list([true, true, false])
iex> Explorer.Series.equal(s, true)
#Explorer.Series<
Polars[3]
boolean [true, true, false]
>
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.equal(s, "a")
#Explorer.Series<
Polars[3]
boolean [true, false, false]
>
iex> s = Explorer.Series.from_list([~D[2021-01-01], ~D[1999-12-31]])
iex> Explorer.Series.equal(s, ~D[1999-12-31])
#Explorer.Series<
Polars[2]
boolean [false, true]
>
iex> s = Explorer.Series.from_list([~N[2022-01-01 00:00:00], ~N[2022-01-01 23:00:00]])
iex> Explorer.Series.equal(s, ~N[2022-01-01 00:00:00])
#Explorer.Series<
Polars[2]
boolean [true, false]
>
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.equal(s, false)
** (ArgumentError) cannot invoke Explorer.Series.equal/2 with mismatched dtypes: :string and falseCalculates the exponential of all elements.
Picks values based on an Explorer.Query.
The query is compiled and runs efficiently against the series.
The query must return a boolean expression or a list of boolean expressions.
When a list is returned, they are joined as and expressions.
Notice
This is a macro.
- You must
require Explorer.Seriesbefore using it.- You must use the special
_syntax. See the moduledoc for details.
Besides element-wise series operations, you can also use window functions and aggregations inside comparisons.
See filter_with/2 for a callback version of this function without
Explorer.Query.
See mask/2 if you want to filter values based on another series.
Examples
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.filter(s, _ == "b")
#Explorer.Series<
Polars[1]
string ["b"]
>
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.filter(s, remainder(_, 2) == 1)
#Explorer.Series<
Polars[2]
s64 [1, 3]
>Returning a non-boolean expression errors:
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.filter(s, cumulative_max(_))
** (ArgumentError) expecting the function to return a boolean LazySeries, but instead it returned a LazySeries of type {:s, 64}Which can be addressed by converting it to boolean:
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.filter(s, cumulative_max(_) == 1)
#Explorer.Series<
Polars[1]
s64 [1]
>Filters a series with a callback function.
See mask/2 if you want to filter values based on another series.
Examples
iex> series = Explorer.Series.from_list([1, 2, 3])
iex> is_odd = fn s -> s |> Explorer.Series.remainder(2) |> Explorer.Series.equal(1) end
iex> Explorer.Series.filter_with(series, is_odd)
#Explorer.Series<
Polars[2]
s64 [1, 3]
>@spec greater( left :: t() | number() | Date.t() | NaiveDateTime.t(), right :: t() | number() | Date.t() | NaiveDateTime.t() ) :: t()
Returns boolean mask of left > right, element-wise.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :date:time:datetime:duration
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> s2 = Explorer.Series.from_list([1, 2, 4])
iex> Explorer.Series.greater(s1, s2)
#Explorer.Series<
Polars[3]
boolean [false, false, false]
>@spec greater_equal( left :: t() | number() | Date.t() | NaiveDateTime.t(), right :: t() | number() | Date.t() | NaiveDateTime.t() ) :: t()
Returns boolean mask of left >= right, element-wise.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :date:time:datetime:duration
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> s2 = Explorer.Series.from_list([1, 2, 4])
iex> Explorer.Series.greater_equal(s1, s2)
#Explorer.Series<
Polars[3]
boolean [true, true, false]
>Checks if each element of the series in the left exists in the series on the right, returning a boolean mask.
The series sizes do not have to match.
See member?/2 if you want to check if a literal belongs to a list.
Examples
iex> left = Explorer.Series.from_list([1, 2, 3])
iex> right = Explorer.Series.from_list([1, 2])
iex> Series.in(left, right)
#Explorer.Series<
Polars[3]
boolean [true, true, false]
>
iex> left = Explorer.Series.from_list([~D[1970-01-01], ~D[2000-01-01], ~D[2010-04-17]])
iex> right = Explorer.Series.from_list([~D[1970-01-01], ~D[2010-04-17]])
iex> Series.in(left, right)
#Explorer.Series<
Polars[3]
boolean [true, false, true]
>Returns a mask of nil values.
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 4])
iex> Explorer.Series.is_nil(s)
#Explorer.Series<
Polars[4]
boolean [false, false, true, false]
>Returns a mask of not nil values.
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 4])
iex> Explorer.Series.is_not_nil(s)
#Explorer.Series<
Polars[4]
boolean [true, true, false, true]
>@spec less( left :: t() | number() | Date.t() | NaiveDateTime.t(), right :: t() | number() | Date.t() | NaiveDateTime.t() ) :: t()
Returns boolean mask of left < right, element-wise.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :date:time:datetime:duration
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> s2 = Explorer.Series.from_list([1, 2, 4])
iex> Explorer.Series.less(s1, s2)
#Explorer.Series<
Polars[3]
boolean [false, false, true]
>@spec less_equal( left :: t() | number() | Date.t() | NaiveDateTime.t(), right :: t() | number() | Date.t() | NaiveDateTime.t() ) :: t()
Returns boolean mask of left <= right, element-wise.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :date:time:datetime:duration
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> s2 = Explorer.Series.from_list([1, 2, 4])
iex> Explorer.Series.less_equal(s1, s2)
#Explorer.Series<
Polars[3]
boolean [true, true, true]
>Calculates the natural logarithm.
The resultant series is going to be of dtype {:f, 64}.
See log/2 for passing a custom base.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s = Explorer.Series.from_list([1, 2, 3, nil, 4])
iex> Explorer.Series.log(s)
#Explorer.Series<
Polars[5]
f64 [0.0, 0.6931471805599453, 1.0986122886681098, nil, 1.3862943611198906]
>Calculates the logarithm on a given base.
The resultant series is going to be of dtype {:f, 64}.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s = Explorer.Series.from_list([8, 16, 32])
iex> Explorer.Series.log(s, 2)
#Explorer.Series<
Polars[3]
f64 [3.0, 4.0, 5.0]
>Maps values based on an Explorer.Query.
The query is compiled and runs efficiently against the series.
Notice
This is a macro.
- You must
require Explorer.Seriesbefore using it.- You must use the special
_syntax. See the moduledoc for details.
See map_with/2 for a callback version of this function without Explorer.Query.
This function only works with lazy computations.
See transform/2 for a version that works with any Elixir function.
Examples
Basic example:
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.map(s, _ * 2)
#Explorer.Series<
Polars[3]
s64 [2, 4, 6]
>You can also use window functions and aggregations:
iex> s = Explorer.Series.from_list([2, 3, 4])
iex> Explorer.Series.map(s, _ - min(_))
#Explorer.Series<
Polars[3]
s64 [0, 1, 2]
>Maps a series with a callback function.
This function only works with lazy computations.
See transform/2 for a version that works with any Elixir function.
Examples
iex> series = Explorer.Series.from_list([2, 3, 4])
iex> shift_left = fn s -> Explorer.Series.subtract(s, Explorer.Series.min(s)) end
iex> Explorer.Series.map_with(series, shift_left)
#Explorer.Series<
Polars[3]
s64 [0, 1, 2]
>Filters a series with a mask.
Examples
iex> s1 = Explorer.Series.from_list([1,2,3])
iex> s2 = Explorer.Series.from_list([true, false, true])
iex> Explorer.Series.mask(s1, s2)
#Explorer.Series<
Polars[2]
s64 [1, 3]
>@spec multiply( left :: t() | number() | Explorer.Duration.t(), right :: t() | number() | Explorer.Duration.t() ) :: t()
Multiplies left and right, element-wise.
When mixing floats and integers, the resulting series will have dtype {:f, 64}.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s1 = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> s2 = 11..20 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.multiply(s1, s2)
#Explorer.Series<
Polars[10]
s64 [11, 24, 39, 56, 75, 96, 119, 144, 171, 200]
>
iex> s1 = 1..5 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.multiply(s1, 2)
#Explorer.Series<
Polars[5]
s64 [2, 4, 6, 8, 10]
>Negate the elements of a boolean series.
Examples
iex> s1 = Explorer.Series.from_list([true, false, false])
iex> Explorer.Series.not(s1)
#Explorer.Series<
Polars[3]
boolean [false, true, true]
>@spec not_equal( left :: t() | number() | Date.t() | NaiveDateTime.t() | boolean() | String.t(), right :: t() | number() | Date.t() | NaiveDateTime.t() | boolean() | String.t() ) :: t()
Returns boolean mask of left != right, element-wise.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> s2 = Explorer.Series.from_list([1, 2, 4])
iex> Explorer.Series.not_equal(s1, s2)
#Explorer.Series<
Polars[3]
boolean [false, false, true]
>
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.not_equal(s, 1)
#Explorer.Series<
Polars[3]
boolean [false, true, true]
>
iex> s = Explorer.Series.from_list([true, true, false])
iex> Explorer.Series.not_equal(s, true)
#Explorer.Series<
Polars[3]
boolean [false, false, true]
>
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.not_equal(s, "a")
#Explorer.Series<
Polars[3]
boolean [false, true, true]
>
iex> s = Explorer.Series.from_list([~D[2021-01-01], ~D[1999-12-31]])
iex> Explorer.Series.not_equal(s, ~D[1999-12-31])
#Explorer.Series<
Polars[2]
boolean [true, false]
>
iex> s = Explorer.Series.from_list([~N[2022-01-01 00:00:00], ~N[2022-01-01 23:00:00]])
iex> Explorer.Series.not_equal(s, ~N[2022-01-01 00:00:00])
#Explorer.Series<
Polars[2]
boolean [false, true]
>
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.not_equal(s, false)
** (ArgumentError) cannot invoke Explorer.Series.not_equal/2 with mismatched dtypes: :string and falseReturns a boolean mask of left or right, element-wise.
Both sizes must be series, the series must have the same size or at last one of them must have size of 1.
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> mask1 = Explorer.Series.less(s1, 2)
iex> mask2 = Explorer.Series.greater(s1, 2)
iex> Explorer.Series.or(mask1, mask2)
#Explorer.Series<
Polars[3]
boolean [true, false, true]
>Returns a boolean mask with true where the 'peaks' (series max or min, default max) are.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :date:time:datetime:duration
Examples
iex> s = Explorer.Series.from_list([1, 2, 4, 1, 4])
iex> Explorer.Series.peaks(s)
#Explorer.Series<
Polars[5]
boolean [false, false, true, false, true]
>
iex> s = [~T[03:00:02.000000], ~T[13:24:56.000000], ~T[02:04:19.000000]] |> Explorer.Series.from_list()
iex> Explorer.Series.peaks(s)
#Explorer.Series<
Polars[3]
boolean [false, true, false]
>Raises a numeric series to the power of the exponent.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s = [8, 16, 32] |> Explorer.Series.from_list()
iex> Explorer.Series.pow(s, 2.0)
#Explorer.Series<
Polars[3]
f64 [64.0, 256.0, 1024.0]
>
iex> s = [2, 4, 6] |> Explorer.Series.from_list()
iex> Explorer.Series.pow(s, 3)
#Explorer.Series<
Polars[3]
f64 [8.0, 64.0, 216.0]
>
iex> s = [2, 4, 6] |> Explorer.Series.from_list()
iex> Explorer.Series.pow(s, -3.0)
#Explorer.Series<
Polars[3]
f64 [0.125, 0.015625, 0.004629629629629629]
>
iex> s = [1.0, 2.0, 3.0] |> Explorer.Series.from_list()
iex> Explorer.Series.pow(s, 3.0)
#Explorer.Series<
Polars[3]
f64 [1.0, 8.0, 27.0]
>
iex> s = [2.0, 4.0, 6.0] |> Explorer.Series.from_list()
iex> Explorer.Series.pow(s, 2)
#Explorer.Series<
Polars[3]
f64 [4.0, 16.0, 36.0]
>Element-wise integer division.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Supported dtype
- integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Returns nil if there is a zero in the right-hand side.
Examples
iex> s1 = [10, 11, 10] |> Explorer.Series.from_list()
iex> s2 = [2, 2, 2] |> Explorer.Series.from_list()
iex> Explorer.Series.quotient(s1, s2)
#Explorer.Series<
Polars[3]
s64 [5, 5, 5]
>
iex> s1 = [10, 11, 10] |> Explorer.Series.from_list()
iex> s2 = [2, 2, 0] |> Explorer.Series.from_list()
iex> Explorer.Series.quotient(s1, s2)
#Explorer.Series<
Polars[3]
s64 [5, 5, nil]
>
iex> s1 = [10, 12, 15] |> Explorer.Series.from_list()
iex> Explorer.Series.quotient(s1, 3)
#Explorer.Series<
Polars[3]
s64 [3, 4, 5]
>Assign ranks to data with appropriate handling of tied values.
Options
:method- Determine how ranks are assigned to tied elements. The following methods are available::average: Each value receives the average rank that would be assigned to all tied values. (default):min: Tied values are assigned the minimum rank. Also known as "competition" ranking.:max: Tied values are assigned the maximum of their ranks.:dense: Similar to:min, but the rank of the next highest element is assigned the rank immediately after those assigned to the tied elements.:ordinal: Each value is given a distinct rank based on its occurrence in the series.:random: Similar to:ordinal, but the rank for ties is not dependent on the order that the values occur in the Series.
:descending- Rank in descending order.:seed- An integer to be used as a random seed. If nil, a random value between 0 and 2^64 − 1 will be used. (default: nil)
Examples
iex> s = Explorer.Series.from_list([3, 6, 1, 1, 6])
iex> Explorer.Series.rank(s)
#Explorer.Series<
Polars[5]
f64 [3.0, 4.5, 1.5, 1.5, 4.5]
>
iex> s = Explorer.Series.from_list([1.1, 2.4, 3.2])
iex> Explorer.Series.rank(s, method: :ordinal)
#Explorer.Series<
Polars[3]
u32 [1, 2, 3]
>
iex> s = Explorer.Series.from_list([ ~N[2022-07-07 17:44:13.020548], ~N[2022-07-07 17:43:08.473561], ~N[2022-07-07 17:45:00.116337] ])
iex> Explorer.Series.rank(s, method: :average)
#Explorer.Series<
Polars[3]
f64 [2.0, 1.0, 3.0]
>
iex> s = Explorer.Series.from_list([3, 6, 1, 1, 6])
iex> Explorer.Series.rank(s, method: :min)
#Explorer.Series<
Polars[5]
s64 [3, 4, 1, 1, 4]
>
iex> s = Explorer.Series.from_list([3, 6, 1, 1, 6])
iex> Explorer.Series.rank(s, method: :dense)
#Explorer.Series<
Polars[5]
s64 [2, 3, 1, 1, 3]
>
iex> s = Explorer.Series.from_list([3, 6, 1, 1, 6])
iex> Explorer.Series.rank(s, method: :random, seed: 42)
#Explorer.Series<
Polars[5]
s64 [3, 4, 2, 1, 5]
>Computes the remainder of an element-wise integer division.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Supported dtype
- integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Returns nil if there is a zero in the right-hand side.
Examples
iex> s1 = [10, 11, 10] |> Explorer.Series.from_list()
iex> s2 = [2, 2, 2] |> Explorer.Series.from_list()
iex> Explorer.Series.remainder(s1, s2)
#Explorer.Series<
Polars[3]
s64 [0, 1, 0]
>
iex> s1 = [10, 11, 10] |> Explorer.Series.from_list()
iex> s2 = [2, 2, 0] |> Explorer.Series.from_list()
iex> Explorer.Series.remainder(s1, s2)
#Explorer.Series<
Polars[3]
s64 [0, 1, nil]
>
iex> s1 = [10, 11, 9] |> Explorer.Series.from_list()
iex> Explorer.Series.remainder(s1, 3)
#Explorer.Series<
Polars[3]
s64 [1, 2, 0]
>Returns a series of indexes for each item (row) in the series, starting from 0.
Examples
iex> s = Series.from_list([nil, true, true])
iex> Series.row_index(s)
#Explorer.Series<
Polars[3]
u32 [0, 1, 2]
>This function can be used to add a row index as the first column of a dataframe.
The resulting column is a regular column of type :u32.
iex> require Explorer.DataFrame, as: DF
iex> df = DF.new(a: [1, 3, 5], b: [2, 4, 6])
iex> DF.mutate(df, index: row_index(a)) |> DF.relocate("index", before: 0)
#Explorer.DataFrame<
Polars[3 x 3]
index u32 [0, 1, 2]
a s64 [1, 3, 5]
b s64 [2, 4, 6]
>
iex> df = DF.new(a: [1, 3, 5], b: [2, 4, 6])
iex> DF.mutate(df, id: row_index(a) + 1000)
#Explorer.DataFrame<
Polars[3 x 3]
a s64 [1, 3, 5]
b s64 [2, 4, 6]
id s64 [1000, 1001, 1002]
>@spec select( predicate :: t(), on_true :: t() | inferable_scalar(), on_false :: t() | inferable_scalar() ) :: t()
Returns a series from two series, based on a predicate.
The resulting series is built by evaluating each element of
predicate and returning either the corresponding element from
on_true or on_false.
predicate must be a boolean series. on_true and on_false must be
a series of the same size as predicate or a series of size 1.
It is possible to mix numeric series in the on_true and on_false,
and the resultant series will have the dtype of the greater side.
For example, :u8 and :s16 is going to result in :s16 series.
Converts a datetime series to a string series.
For the format string specification, refer to the chrono crate documentation.
Use cast(series, :string) for the default "%Y-%m-%d %H:%M:%S%.6f" format.
Examples
iex> s = Explorer.Series.from_list([~N[2023-01-05 12:34:56], nil])
iex> Explorer.Series.strftime(s, "%Y/%m/%d %H:%M:%S")
#Explorer.Series<
Polars[2]
string ["2023/01/05 12:34:56", nil]
>Converts a string series to a datetime series with a given format_string.
For the format string specification, refer to the chrono crate documentation.
Use cast(series, :datetime) if you prefer the format to be inferred (if possible).
Examples
iex> s = Explorer.Series.from_list(["2023-01-05 12:34:56", "XYZ", nil])
iex> Explorer.Series.strptime(s, "%Y-%m-%d %H:%M:%S")
#Explorer.Series<
Polars[3]
datetime[μs] [2023-01-05 12:34:56.000000, nil, nil]
>@spec subtract( left :: t() | number() | Date.t() | NaiveDateTime.t() | Explorer.Duration.t(), right :: t() | number() | Date.t() | NaiveDateTime.t() | Explorer.Duration.t() ) :: t()
Subtracts right from left, element-wise.
When mixing floats and integers, the resulting series will have dtype {:f, 64}.
In case both series are of unsigned integers, we will try to subtract,
but an exception is raised if overflow occurs.
At least one of the arguments must be a series. If both sizes are series, the series must have the same size or at last one of them must have size of 1.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> s2 = Explorer.Series.from_list([4, 5, 6])
iex> Explorer.Series.subtract(s1, s2)
#Explorer.Series<
Polars[3]
s64 [-3, -3, -3]
>You can also use scalar values on both sides:
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.subtract(s1, 2)
#Explorer.Series<
Polars[3]
s64 [-1, 0, 1]
>
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.subtract(2, s1)
#Explorer.Series<
Polars[3]
s64 [1, 0, -1]
>Returns an Explorer.Series where each element is the result of invoking fun on each
corresponding element of series.
This is an expensive operation meant to enable the use of arbitrary Elixir functions against
any backend. The implementation will vary by backend but in most (all?) cases will require
converting to an Elixir.List, applying Enum.map/2, and then converting back to an
Explorer.Series.
Examples
iex> s = Explorer.Series.from_list(["this ", " is", "great "])
iex> Explorer.Series.transform(s, &String.trim/1)
#Explorer.Series<
Polars[3]
string ["this", "is", "great"]
>
iex> s = Explorer.Series.from_list(["this", "is", "great"])
iex> Explorer.Series.transform(s, &String.length/1)
#Explorer.Series<
Polars[3]
s64 [4, 2, 5]
>Functions: Datetime ops
Returns a day-of-week number starting from Monday = 1. (ISO 8601 weekday number)
Examples
iex> s = Explorer.Series.from_list([~D[2023-01-15], ~D[2023-01-16], ~D[2023-01-20], nil])
iex> Explorer.Series.day_of_week(s)
#Explorer.Series<
Polars[4]
s8 [7, 1, 5, nil]
>It can also be called on a datetime series.
iex> s = Explorer.Series.from_list([~N[2023-01-15 00:00:00], ~N[2023-01-16 23:59:59.999999], ~N[2023-01-20 12:00:00], nil])
iex> Explorer.Series.day_of_week(s)
#Explorer.Series<
Polars[4]
s8 [7, 1, 5, nil]
>Returns the day-of-year number starting from 1.
The return value ranges from 1 to 366 (the last day of year differs by years).
Examples
iex> s = Explorer.Series.from_list([~D[2023-01-01], ~D[2023-01-02], ~D[2023-02-01], nil])
iex> Explorer.Series.day_of_year(s)
#Explorer.Series<
Polars[4]
s16 [1, 2, 32, nil]
>It can also be called on a datetime series.
iex> s = Explorer.Series.from_list(
...> [~N[2023-01-01 00:00:00], ~N[2023-01-02 00:00:00], ~N[2023-02-01 23:59:59], nil]
...> )
iex> Explorer.Series.day_of_year(s)
#Explorer.Series<
Polars[4]
s16 [1, 2, 32, nil]
>Returns the hour number from 0 to 23.
Examples
iex> s = Explorer.Series.from_list([~N[2023-01-15 00:00:00], ~N[2022-02-16 23:59:59.999999], ~N[2021-03-20 12:00:00], nil])
iex> Explorer.Series.hour(s)
#Explorer.Series<
Polars[4]
s8 [0, 23, 12, nil]
>Returns the minute number from 0 to 59.
Examples
iex> s = Explorer.Series.from_list([~N[2023-01-15 00:00:00], ~N[2022-02-16 23:59:59.999999], ~N[2021-03-20 12:00:00], nil])
iex> Explorer.Series.minute(s)
#Explorer.Series<
Polars[4]
s8 [0, 59, 0, nil]
>Returns the month number starting from 1. The return value ranges from 1 to 12.
Examples
iex> s = Explorer.Series.from_list([~D[2023-01-15], ~D[2023-02-16], ~D[2023-03-20], nil])
iex> Explorer.Series.month(s)
#Explorer.Series<
Polars[4]
s8 [1, 2, 3, nil]
>It can also be called on a datetime series.
iex> s = Explorer.Series.from_list([~N[2023-01-15 00:00:00], ~N[2023-02-16 23:59:59.999999], ~N[2023-03-20 12:00:00], nil])
iex> Explorer.Series.month(s)
#Explorer.Series<
Polars[4]
s8 [1, 2, 3, nil]
>Returns the second number from 0 to 59.
Examples
iex> s = Explorer.Series.from_list([~N[2023-01-15 00:00:00], ~N[2022-02-16 23:59:59.999999], ~N[2021-03-20 12:00:00], nil])
iex> Explorer.Series.second(s)
#Explorer.Series<
Polars[4]
s8 [0, 59, 0, nil]
>Returns the week-of-year number.
The return value ranges from 1 to 53 (the last week of year differs by years).
Weeks start on Monday and end on Sunday. If the final week of a year does not end on Sunday, the first days of the following year will have a week number of 52 (or 53 for a leap year).
Examples
iex> s = Explorer.Series.from_list([~D[2023-01-01], ~D[2023-01-02], ~D[2023-02-01], nil])
iex> Explorer.Series.week_of_year(s)
#Explorer.Series<
Polars[4]
s8 [52, 1, 5, nil]
>It can also be called on a datetime series.
iex> s = Explorer.Series.from_list(
...> [~N[2023-01-01 00:00:00], ~N[2023-01-02 00:00:00], ~N[2023-02-01 23:59:59], nil]
...> )
iex> Explorer.Series.week_of_year(s)
#Explorer.Series<
Polars[4]
s8 [52, 1, 5, nil]
>Returns the year number in the calendar date.
Examples
iex> s = Explorer.Series.from_list([~D[2023-01-15], ~D[2022-02-16], ~D[2021-03-20], nil])
iex> Explorer.Series.year(s)
#Explorer.Series<
Polars[4]
s32 [2023, 2022, 2021, nil]
>It can also be called on a datetime series.
iex> s = Explorer.Series.from_list([~N[2023-01-15 00:00:00], ~N[2022-02-16 23:59:59.999999], ~N[2021-03-20 12:00:00], nil])
iex> Explorer.Series.year(s)
#Explorer.Series<
Polars[4]
s32 [2023, 2022, 2021, nil]
>Functions: Float ops
Computes the the arccosine of a number.
The resultant series is going to be of dtype {:f, 64}, in radians, with values between 0 and pi.
Supported dtype
{:f, 32}{:f, 64}
Examples
iex> s = [1.0, 0.0, -1.0, -0.7071067811865475, 0.7071067811865475] |> Explorer.Series.from_list()
iex> Explorer.Series.acos(s)
#Explorer.Series<
Polars[5]
f64 [0.0, 1.5707963267948966, 3.141592653589793, 2.356194490192345, 0.7853981633974484]
>Computes the the arcsine of a number.
The resultant series is going to be of dtype {:f, 64}, in radians, with values between -pi/2 and pi/2.
Supported dtype
{:f, 32}{:f, 64}
Examples
iex> s = [1.0, 0.0, -1.0, -0.7071067811865475, 0.7071067811865475] |> Explorer.Series.from_list()
iex> Explorer.Series.asin(s)
#Explorer.Series<
Polars[5]
f64 [1.5707963267948966, 0.0, -1.5707963267948966, -0.7853981633974482, 0.7853981633974482]
>Computes the the arctangent of a number.
The resultant series is going to be of dtype {:f, 64}, in radians, with values between -pi/2 and pi/2.
Supported dtype
{:f, 32}{:f, 64}
Examples
iex> s = [1.0, 0.0, -1.0, -0.7071067811865475, 0.7071067811865475] |> Explorer.Series.from_list()
iex> Explorer.Series.atan(s)
#Explorer.Series<
Polars[5]
f64 [0.7853981633974483, 0.0, -0.7853981633974483, -0.6154797086703873, 0.6154797086703873]
>Ceil floating point series to highest integers smaller or equal to the float value.
Examples
iex> s = Explorer.Series.from_list([1.124993, 2.555321, 3.995001])
iex> Explorer.Series.ceil(s)
#Explorer.Series<
Polars[3]
f64 [2.0, 3.0, 4.0]
>Computes the the cosine of a number (in radians).
The resultant series is going to be of dtype {:f, 64}, with values between 1 and -1.
Supported dtype
{:f, 32}{:f, 64}
Examples
iex> pi = :math.pi()
iex> s = [-pi * 3/2, -pi, -pi / 2, -pi / 4, 0, pi / 4, pi / 2, pi, pi * 3/2] |> Explorer.Series.from_list()
iex> Explorer.Series.cos(s)
#Explorer.Series<
Polars[9]
f64 [-1.8369701987210297e-16, -1.0, 6.123233995736766e-17, 0.7071067811865476, 1.0, 0.7071067811865476, 6.123233995736766e-17, -1.0, -1.8369701987210297e-16]
>Floor floating point series to lowest integers smaller or equal to the float value.
Examples
iex> s = Explorer.Series.from_list([1.124993, 2.555321, 3.995001])
iex> Explorer.Series.floor(s)
#Explorer.Series<
Polars[3]
f64 [1.0, 2.0, 3.0]
>Returns a mask of finite values.
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 0, nil])
iex> s2 = Explorer.Series.from_list([0, 2, 0, nil])
iex> s3 = Explorer.Series.divide(s1, s2)
iex> Explorer.Series.is_finite(s3)
#Explorer.Series<
Polars[4]
boolean [false, true, false, nil]
>Returns a mask of infinite values.
Examples
iex> s1 = Explorer.Series.from_list([1, -1, 2, 0, nil])
iex> s2 = Explorer.Series.from_list([0, 0, 2, 0, nil])
iex> s3 = Explorer.Series.divide(s1, s2)
iex> Explorer.Series.is_infinite(s3)
#Explorer.Series<
Polars[5]
boolean [true, true, false, false, nil]
>Returns a mask of nan values.
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 0, nil])
iex> s2 = Explorer.Series.from_list([0, 2, 0, nil])
iex> s3 = Explorer.Series.divide(s1, s2)
iex> Explorer.Series.is_nan(s3)
#Explorer.Series<
Polars[4]
boolean [false, false, true, nil]
>@spec round(t(), non_neg_integer()) :: t()
Round floating point series to given decimal places.
Examples
iex> s = Explorer.Series.from_list([1.124993, 2.555321, 3.995001])
iex> Explorer.Series.round(s, 2)
#Explorer.Series<
Polars[3]
f64 [1.12, 2.56, 4.0]
>Computes the the sine of a number (in radians).
The resultant series is going to be of dtype {:f, 64}, with values between 1 and -1.
Supported dtype
{:f, 32}{:f, 64}
Examples
iex> pi = :math.pi()
iex> s = [-pi * 3/2, -pi, -pi / 2, -pi / 4, 0, pi / 4, pi / 2, pi, pi * 3/2] |> Explorer.Series.from_list()
iex> Explorer.Series.sin(s)
#Explorer.Series<
Polars[9]
f64 [1.0, -1.2246467991473532e-16, -1.0, -0.7071067811865475, 0.0, 0.7071067811865475, 1.0, 1.2246467991473532e-16, -1.0]
>Computes the tangent of a number (in radians).
The resultant series is going to be of dtype {:f, 64}.
Supported dtype
{:f, 32}{:f, 64}
Examples
iex> pi = :math.pi()
iex> s = [-pi * 3/2, -pi, -pi / 2, -pi / 4, 0, pi / 4, pi / 2, pi, pi * 3/2] |> Explorer.Series.from_list()
iex> Explorer.Series.tan(s)
#Explorer.Series<
Polars[9]
f64 [-5443746451065123.0, 1.2246467991473532e-16, -1.633123935319537e16, -0.9999999999999999, 0.0, 0.9999999999999999, 1.633123935319537e16, -1.2246467991473532e-16, 5443746451065123.0]
>Functions: String ops
Detects whether a string contains a substring.
Notice
This function detects only literal strings. For regular expressions, see
re_contains/2.
Examples
iex> s = Explorer.Series.from_list(["abc", "def", "bcd"])
iex> Explorer.Series.contains(s, "bc")
#Explorer.Series<
Polars[3]
boolean [true, false, true]
>Count how many times a substring appears in a string.
Notice
This function detects only literal strings. For regular expressions, see
re_count_matches/2.
Examples
iex> s = Explorer.Series.from_list(["abc", "def", "bcd", nil])
iex> Explorer.Series.count_matches(s, "bc")
#Explorer.Series<
Polars[4]
u32 [1, 0, 1, nil]
>Converts all characters to lowercase.
Examples
iex> s = Explorer.Series.from_list(["ABC", "DEF", "BCD"])
iex> Explorer.Series.downcase(s)
#Explorer.Series<
Polars[3]
string ["abc", "def", "bcd"]
>Decodes a string series containing valid JSON according to dtype.
Examples
iex> s = Series.from_list(["1"])
iex> Series.json_decode(s, {:s, 64})
#Explorer.Series<
Polars[1]
s64 [1]
>
iex> s = Series.from_list(["{\"a\":1}"])
iex> Series.json_decode(s, {:struct, [{"a", {:s, 64}}]})
#Explorer.Series<
Polars[1]
struct[1] [%{"a" => 1}]
>If the decoded value does not match the given dtype,
nil is returned for the given entry:
iex> s = Series.from_list(["\"1\""])
iex> Series.json_decode(s, {:s, 64})
#Explorer.Series<
Polars[1]
s64 [nil]
>It raises an exception if the string is invalid JSON.
Extracts a string series using a json_path from a series.
Examples
iex> s = Series.from_list(["{\"a\":1}", "{\"a\":2}"])
iex> Series.json_path_match(s, "$.a")
#Explorer.Series<
Polars[2]
string ["1", "2"]
>If json_path is not found or if the string is invalid JSON or nil,
nil is returned for the given entry:
iex> s = Series.from_list(["{\"a\":1}", nil, "{\"a\":2}"])
iex> Series.json_path_match(s, "$.b")
#Explorer.Series<
Polars[3]
string [nil, nil, nil]
>It raises an exception if the json_path is invalid.
Returns a string series where all leading Unicode whitespaces have been removed.
Examples
iex> s = Explorer.Series.from_list([" abc", "def ", " bcd"])
iex> Explorer.Series.lstrip(s)
#Explorer.Series<
Polars[3]
string ["abc", "def ", "bcd"]
>Returns a string series where all leading examples of the provided string have been removed.
Where multiple characters are provided, all combinations of this set of characters will be stripped
Examples
iex> s = Explorer.Series.from_list(["$1", "$$200$$", "$$$3000$"])
iex> Explorer.Series.lstrip(s, "$")
#Explorer.Series<
Polars[3]
string ["1", "200$$", "3000$"]
>
iex> s = Explorer.Series.from_list(["abc", "adefa", "bcda"])
iex> Explorer.Series.lstrip(s, "ab")
#Explorer.Series<
Polars[3]
string ["c", "defa", "cda"]
>Detects whether a string matches a pattern.
Notice
This function matches against a regular expression. It does not expect an Elixir regex, but a escaped string and you can use the
~Ssigil for escaping it. Since each Explorer backend may have its own regular expression rules, you must consult their underlying engine. For the default backend (Polars), the rules are outlined in the Rust create namedregex.To match literal strings, you can use
contains/2.
Examples
iex> s = Explorer.Series.from_list(["abc", "def", "bcd"])
iex> Explorer.Series.re_contains(s, ~S/(a|e)/)
#Explorer.Series<
Polars[3]
boolean [true, true, false]
>Count how many times a pattern matches a string.
Notice
This function matches against a regular expression. It does not expect an Elixir regex, but a escaped string and you can use the
~Ssigil for escaping it. Since each Explorer backend may have its own regular expression rules, you must consult their underlying engine. For the default backend (Polars), the rules are outlined in the Rust create namedregex.To count matching literal strings, you can use
count_matches/2.
Examples
iex> s = Explorer.Series.from_list(["abc", "def def", "bcd", nil])
iex> Explorer.Series.re_count_matches(s, ~S/(a|e)/)
#Explorer.Series<
Polars[4]
u32 [1, 2, 0, nil]
>Extract all capture groups as a struct for the given regex pattern.
All group names are strings. If your pattern contains unnamed groups, their numerical position is converted to a string, starting from "1".
Notice
This function does not work inside the context of
Explorer.Query. If you are working on a data frame, you first need to extract the series from it. SeeExplorer.DataFrame.pull/2.This function matches against a regular expression. It does not expect an Elixir regex, but a escaped string and you can use the
~Ssigil for escaping it. Since each Explorer backend may have its own regular expression rules, you must consult their underlying engine. For the default backend (Polars), the rules are outlined in the Rust create namedregex.
Examples
iex> s = Explorer.Series.from_list(["abc", "def def", "bcd", nil])
iex> Explorer.Series.re_named_captures(s, ~S/(b|d)/)
#Explorer.Series<
Polars[4]
struct[1] [%{"1" => "b"}, %{"1" => "d"}, %{"1" => "b"}, %{"1" => nil}]
>
iex> s = Explorer.Series.from_list(["alice@service.com", "bob@example.com"])
iex> Explorer.Series.re_named_captures(s, ~S/(?<account>[^@]+)@(?<host>.*)/)
#Explorer.Series<
Polars[2]
struct[2] [%{"account" => "alice", "host" => "service.com"}, %{"account" => "bob", "host" => "example.com"}]
>Replaces all occurences of a pattern with replacement in string series.
Both pattern and replacement must be of type string. The replacement
can refer to groups captures by using the ${x}, where x is a number starting from 1.
It can also refer to named groups using the same syntax.
Notice
This function matches against a regular expression. It does not expect an Elixir regex, but a escaped string and you can use the
~Ssigil for escaping it. Since each Explorer backend may have its own regular expression rules, you must consult their underlying engine. For the default backend (Polars), the rules are outlined in the Rust create namedregex.To replace by literal strings, you can use
replace/3.
Examples
iex> series = Explorer.Series.from_list(["1.200,45", "1.234.567,30", "asdf", nil])
iex> Explorer.Series.re_replace(series, ~S/[,.]/, "")
#Explorer.Series<
Polars[4]
string ["120045", "123456730", "asdf", nil]
>
iex> series = Explorer.Series.from_list(["hat", "hut"])
iex> Explorer.Series.re_replace(series, ~S/h(.)t/, "b${1}d")
#Explorer.Series<
Polars[2]
string ["bad", "bud"]
>
iex> series = Explorer.Series.from_list(["hat", "hut"])
iex> Explorer.Series.re_replace(series, ~S/h(?<vowel>.)t/, "b${vowel}d")
#Explorer.Series<
Polars[2]
string ["bad", "bud"]
>Apply case-insensitive string replacement using the (?i) flag - remember, from the regex Rust crate.
iex> series = Explorer.Series.from_list(["Foggy", "Rainy", "Sunny"])
iex> Explorer.Series.re_replace(series, ~S/(?i)foggy|rainy/, "Sunny")
#Explorer.Series<
Polars[3]
string ["Sunny", "Sunny", "Sunny"]
>With an Elixir regex it causes an error:
iex> series = Explorer.Series.from_list(["hat", "hut"])
iex> Explorer.Series.re_replace(series, ~r/h(.)t/, "b${1}d")
** (ArgumentError) standard regexes cannot be used as pattern because it may be incompatible with the backend. Please use the `~S` sigil or extract the source from the regex with `Regex.source/1`Scan for all matches for the given regex pattern.
Extract each successive non-overlapping regex match in an individual string as a list.
Notice
This function matches against a regular expression. It does not expect an Elixir regex, but a escaped string and you can use the
~Ssigil for escaping it. Since each Explorer backend may have its own regular expression rules, you must consult their underlying engine. For the default backend (Polars), the rules are outlined in the Rust create namedregex.
Examples
iex> s = Explorer.Series.from_list(["abc", "def def", "bcd", nil])
iex> Explorer.Series.re_scan(s, ~S/(b|d)/)
#Explorer.Series<
Polars[4]
list[string] [["b"], ["d", "d"], ["b", "d"], nil]
>Replaces all occurences of a substring with replacement in string series.
Both substring and replacement must be of type string.
Notice
This function replaces only literal strings. For regular expressions, see
re_replace/3.
Examples
iex> series = Explorer.Series.from_list(["1,200", "1,234,567", "asdf", nil])
iex> Explorer.Series.replace(series, ",", "")
#Explorer.Series<
Polars[4]
string ["1200", "1234567", "asdf", nil]
>Returns a string series where all trailing Unicode whitespaces have been removed.
Examples
iex> s = Explorer.Series.from_list([" abc", "def ", " bcd"])
iex> Explorer.Series.rstrip(s)
#Explorer.Series<
Polars[3]
string [" abc", "def", " bcd"]
>Returns a string series where all trailing examples of the provided string have been removed.
Where multiple characters are provided, all combinations of this set of characters will be stripped
Examples
iex> s = Explorer.Series.from_list(["__abc__", "def_", "__bcd_"])
iex> Explorer.Series.rstrip(s, "_")
#Explorer.Series<
Polars[3]
string ["__abc", "def", "__bcd"]
>
iex> s = Explorer.Series.from_list(["abc", "adefa", "bcdabaaa"])
iex> Explorer.Series.rstrip(s, "ab")
#Explorer.Series<
Polars[3]
string ["abc", "adef", "bcd"]
>Split the string by a substring.
Examples
iex> s = Series.from_list(["1", "1|2"])
iex> Series.split(s, "|")
#Explorer.Series<
Polars[2]
list[string] [["1"], ["1", "2"]]
>Split a string Series into a struct of string fields.
The length of the field names list determines how many times the string will be split at most. If the string cannot be split into that many separate strings, null values will be provided for the remaining fields.
Examples
iex> s = Series.from_list(["Smith, John", "Jones, Jane"])
iex> Series.split_into(s, ", ", ["Last Name", "First Name"])
#Explorer.Series<
Polars[2]
struct[2] [%{"First Name" => "John", "Last Name" => "Smith"}, %{"First Name" => "Jane", "Last Name" => "Jones"}]
>Returns a string series where all leading and trailing Unicode whitespaces have been removed.
Examples
iex> s = Explorer.Series.from_list([" abc", "def ", " bcd "])
iex> Explorer.Series.strip(s)
#Explorer.Series<
Polars[3]
string ["abc", "def", "bcd"]
>Returns a string series where all leading and trailing examples of the provided string have been removed.
Where multiple characters are provided, all combinations of this set of characters will be stripped
Examples
iex> s = Explorer.Series.from_list(["£123", "1.00£", "£1.00£"])
iex> Explorer.Series.strip(s, "£")
#Explorer.Series<
Polars[3]
string ["123", "1.00", "1.00"]
>
iex> s = Explorer.Series.from_list(["abc", "adefa", "bcda"])
iex> Explorer.Series.strip(s, "ab")
#Explorer.Series<
Polars[3]
string ["c", "def", "cd"]
>Returns a string sliced from the offset to the end of the string, supporting negative indexing
Examples
iex> s = Explorer.Series.from_list(["earth", "mars", "neptune"])
iex> Explorer.Series.substring(s, -3)
#Explorer.Series<
Polars[3]
string ["rth", "ars", "une"]
>
iex> s = Explorer.Series.from_list(["earth", "mars", "neptune"])
iex> Explorer.Series.substring(s, 1)
#Explorer.Series<
Polars[3]
string ["arth", "ars", "eptune"]
>Returns a string sliced from the offset to the length provided, supporting negative indexing
Examples
iex> s = Explorer.Series.from_list(["earth", "mars", "neptune"])
iex> Explorer.Series.substring(s, -3, 2)
#Explorer.Series<
Polars[3]
string ["rt", "ar", "un"]
>
iex> s = Explorer.Series.from_list(["earth", "mars", "neptune"])
iex> Explorer.Series.substring(s, 1, 5)
#Explorer.Series<
Polars[3]
string ["arth", "ars", "eptun"]
>
iex> d = Explorer.Series.from_list(["こんにちは世界", "مرحبًا", "안녕하세요"])
iex> Explorer.Series.substring(d, 1, 3)
#Explorer.Series<
Polars[3]
string ["んにち", "رحب", "녕하세"]
>Converts all characters to uppercase.
Examples
iex> s = Explorer.Series.from_list(["abc", "def", "bcd"])
iex> Explorer.Series.upcase(s)
#Explorer.Series<
Polars[3]
string ["ABC", "DEF", "BCD"]
>Functions: List ops
Join all string items in a sublist and place a separator between them.
Examples
iex> s = Series.from_list([["1"], ["1", "2"]])
iex> Series.join(s, "|")
#Explorer.Series<
Polars[2]
string ["1", "1|2"]
>Calculates the length of each list in a list series.
Examples
iex> s = Series.from_list([[1], [1, 2]])
iex> Series.lengths(s)
#Explorer.Series<
Polars[2]
u32 [1, 2]
>@spec member?(t(), Explorer.Backend.Series.valid_types()) :: t()
Checks for the presence of a value in a list series.
Examples
iex> s = Series.from_list([[1], [1, 2]])
iex> Series.member?(s, 2)
#Explorer.Series<
Polars[2]
boolean [false, true]
>Functions: Struct ops
Extracts a field from a struct series
The field can be an atom or string.
Examples
iex> s = Series.from_list([%{a: 1}, %{a: 2}])
iex> Series.field(s, :a)
#Explorer.Series<
Polars[2]
s64 [1, 2]
>
iex> s = Series.from_list([%{a: 1}, %{a: 2}])
iex> Series.field(s, "a")
#Explorer.Series<
Polars[2]
s64 [1, 2]
>Functions: Introspection
Return a series with the category names of a categorical series.
Each category has the index equal to its position. No order for the categories is guaranteed.
Examples
iex> s = Explorer.Series.from_list(["a", "b", "c", nil, "a", "c"], dtype: :category)
iex> Explorer.Series.categories(s)
#Explorer.Series<
Polars[3]
string ["a", "b", "c"]
>
iex> s = Explorer.Series.from_list(["c", "a", "b"], dtype: :category)
iex> Explorer.Series.categories(s)
#Explorer.Series<
Polars[3]
string ["c", "a", "b"]
>Returns the data type of the series.
See the moduledoc for all supported dtypes.
Examples
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.dtype(s)
{:s, 64}
iex> s = Explorer.Series.from_list(["a", nil, "b", "c"])
iex> Explorer.Series.dtype(s)
:string@spec iotype(series :: t()) :: {:s | :u | :f, non_neg_integer()} | :none
Returns the type of the underlying fixed-width binary representation.
It returns something in the shape of {atom(), bits_size} or :none.
It is often used in conjunction with to_iovec/1 and to_binary/1.
The possible iotypes are:
:ufor unsigned integers.:sfor signed integers.:ffor floats.
Examples
iex> s = Explorer.Series.from_list([1, 2, 3, 4])
iex> Explorer.Series.iotype(s)
{:s, 64}
iex> s = Explorer.Series.from_list([~D[1999-12-31], ~D[1989-01-01]])
iex> Explorer.Series.iotype(s)
{:s, 32}
iex> s = Explorer.Series.from_list([~T[00:00:00.000000], ~T[23:59:59.999999]])
iex> Explorer.Series.iotype(s)
{:s, 64}
iex> s = Explorer.Series.from_list([1.2, 2.3, 3.5, 4.5])
iex> Explorer.Series.iotype(s)
{:f, 64}
iex> s = Explorer.Series.from_list([true, false, true])
iex> Explorer.Series.iotype(s)
{:u, 8}The operation returns :none for strings and binaries, as they do not
provide a fixed-width binary representation:
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.iotype(s)
:noneHowever, if appropriate, you can convert them to categorical types, which will then return the index of each category:
iex> s = Explorer.Series.from_list(["a", "b", "c"], dtype: :category)
iex> Explorer.Series.iotype(s)
{:u, 32}@spec size(series :: t()) :: non_neg_integer() | lazy_t()
Returns the number of elements in the series.
See also:
count/1- counts only the non-nilelements.nil_count/1- counts allnilelements.
Examples
Basic example:
iex> s = Explorer.Series.from_list([~D[1999-12-31], ~D[1989-01-01]])
iex> Explorer.Series.size(s)
2With lists:
iex> s = Explorer.Series.from_list([[1, 2, 3], [4, 5]])
iex> Explorer.Series.size(s)
2Functions: Shape
Returns the indices that would sort the series.
The resultant series is going to have the {:u, 32} dtype.
Options
:direction-:ascor:desc, meaning "ascending" or "descending", respectively. By default it sorts in ascending order.:nils-:firstor:last. By default it is:lastif direction is:asc, and:firstotherwise.:parallel- boolean. Whether to parallelize the sorting. By default it istrue.:stable- boolean. Determines if the sorting is stable (ties are guaranteed to maintain their order) or not. Unstable sorting may be more performant. By default it isfalse.
Examples
iex> s = Explorer.Series.from_list([9, 3, 7, 1])
iex> Explorer.Series.argsort(s)
#Explorer.Series<
Polars[4]
u32 [3, 1, 2, 0]
>
iex> s = Explorer.Series.from_list([9, 3, 7, 1])
iex> Explorer.Series.argsort(s, direction: :desc)
#Explorer.Series<
Polars[4]
u32 [0, 2, 1, 3]
>Returns the value of the series at the given index.
This function will raise an error in case the index is out of bounds.
Examples
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.at(s, 2)
"c"
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.at(s, 4)
** (ArgumentError) index 4 out of bounds for series of size 3Takes every nth value in this series, returned as a new series.
Examples
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.at_every(s, 2)
#Explorer.Series<
Polars[5]
s64 [1, 3, 5, 7, 9]
>If n is bigger than the size of the series, the result is a new series with only the first value of the supplied series.
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.at_every(s, 20)
#Explorer.Series<
Polars[1]
s64 [1]
>Concatenate one or more series.
Type promotion may happen between numeric series or null series. All other dtypes must match.
Examples
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> s2 = Explorer.Series.from_list([4, 5, 6])
iex> Explorer.Series.concat([s1, s2])
#Explorer.Series<
Polars[6]
s64 [1, 2, 3, 4, 5, 6]
>
iex> s1 = Explorer.Series.from_list([1, 2, 3])
iex> s2 = Explorer.Series.from_list([4.0, 5.0, 6.4])
iex> Explorer.Series.concat([s1, s2])
#Explorer.Series<
Polars[6]
f64 [1.0, 2.0, 3.0, 4.0, 5.0, 6.4]
>Concatenate two series.
concat(s1, s2) is equivalent to concat([s1, s2]).
Returns the unique values of the series.
Examples
iex> s = [1, 1, 2, 2, 3, 3] |> Explorer.Series.from_list()
iex> Explorer.Series.distinct(s)
#Explorer.Series<
Polars[3]
s64 [1, 2, 3]
>Returns the first element of the series.
Examples
iex> s = 1..100 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.first(s)
1Returns a string series with all values concatenated.
Examples
iex> s1 = Explorer.Series.from_list(["a", "b", "c"])
iex> s2 = Explorer.Series.from_list(["d", "e", "f"])
iex> s3 = Explorer.Series.from_list(["g", "h", "i"])
iex> Explorer.Series.format([s1, s2, s3])
#Explorer.Series<
Polars[3]
string ["adg", "beh", "cfi"]
>
iex> s1 = Explorer.Series.from_list(["a", "b", "c", "d"])
iex> s2 = Explorer.Series.from_list([1, 2, 3, 4])
iex> s3 = Explorer.Series.from_list([1.5, :nan, :infinity, :neg_infinity])
iex> Explorer.Series.format([s1, "/", s2, "/", s3])
#Explorer.Series<
Polars[4]
string ["a/1/1.5", "b/2/NaN", "c/3/inf", "d/4/-inf"]
>
iex> s1 = Explorer.Series.from_list([<<1>>, <<239, 191, 19>>], dtype: :binary)
iex> s2 = Explorer.Series.from_list([<<3>>, <<4>>], dtype: :binary)
iex> Explorer.Series.format([s1, s2])
** (RuntimeError) Polars Error: invalid utf8Returns the first N elements of the series.
Examples
iex> s = 1..100 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.head(s)
#Explorer.Series<
Polars[10]
s64 [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>Returns the last element of the series.
Examples
iex> s = 1..100 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.last(s)
100Reverses the series order.
Example
iex> s = [1, 2, 3] |> Explorer.Series.from_list()
iex> Explorer.Series.reverse(s)
#Explorer.Series<
Polars[3]
s64 [3, 2, 1]
>Returns a random sample of the series.
If given an integer as the second argument, it will return N samples. If given a float, it will return that proportion of the series.
Can sample with or without replace.
Options
:replace- If set totrue, each sample will be independent and therefore values may repeat. Required to betrueforngreater then the number of rows in the series orfrac> 1.0. (default:false):seed- An integer to be used as a random seed. If nil, a random value between 0 and 2^64 − 1 will be used. (default: nil):shuffle- In case the sample is equal to the size of the series, shuffle tells if the resultant series should be shuffled or if it should return the same series. (default:false).
Examples
iex> s = 1..100 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.sample(s, 10, seed: 100)
#Explorer.Series<
Polars[10]
s64 [57, 9, 54, 62, 50, 77, 35, 88, 1, 69]
>
iex> s = 1..100 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.sample(s, 0.05, seed: 100)
#Explorer.Series<
Polars[5]
s64 [9, 56, 79, 28, 54]
>
iex> s = 1..5 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.sample(s, 7, seed: 100, replace: true)
#Explorer.Series<
Polars[7]
s64 [4, 1, 3, 4, 3, 4, 2]
>
iex> s = 1..5 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.sample(s, 1.2, seed: 100, replace: true)
#Explorer.Series<
Polars[6]
s64 [4, 1, 3, 4, 3, 4]
>
iex> s = 0..9 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.sample(s, 1.0, seed: 100, shuffle: false)
#Explorer.Series<
Polars[10]
s64 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>
iex> s = 0..9 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.sample(s, 1.0, seed: 100, shuffle: true)
#Explorer.Series<
Polars[10]
s64 [7, 9, 2, 0, 4, 1, 3, 8, 5, 6]
>Shifts series by offset with nil values.
Positive offset shifts from first, negative offset shifts from last.
Examples
iex> s = 1..5 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.shift(s, 2)
#Explorer.Series<
Polars[5]
s64 [nil, nil, 1, 2, 3]
>
iex> s = 1..5 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.shift(s, -2)
#Explorer.Series<
Polars[5]
s64 [3, 4, 5, nil, nil]
>Change the elements order randomly.
Options
:seed- An integer to be used as a random seed. If nil, a random value between 0 and 2^64 − 1 will be used. (default: nil)
Examples
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.shuffle(s, seed: 100)
#Explorer.Series<
Polars[10]
s64 [8, 10, 3, 1, 5, 2, 4, 9, 6, 7]
>Slices the elements at the given indices as a new series.
The indices may be either a list (or series) of indices or a range. A list or series of indices does not support negative numbers. Ranges may be negative on either end, which are then normalized. Note ranges in Elixir are inclusive.
Examples
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.slice(s, [0, 2])
#Explorer.Series<
Polars[2]
string ["a", "c"]
>
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.slice(s, 1..2)
#Explorer.Series<
Polars[2]
string ["b", "c"]
>
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.slice(s, -2..-1)
#Explorer.Series<
Polars[2]
string ["b", "c"]
>
iex> s = Explorer.Series.from_list(["a", "b", "c"])
iex> Explorer.Series.slice(s, 3..2//1)
#Explorer.Series<
Polars[0]
string []
>Returns a slice of the series, with size elements starting at offset.
Examples
iex> s = Explorer.Series.from_list([1, 2, 3, 4, 5])
iex> Explorer.Series.slice(s, 1, 2)
#Explorer.Series<
Polars[2]
s64 [2, 3]
>Negative offsets count from the end of the series:
iex> s = Explorer.Series.from_list([1, 2, 3, 4, 5])
iex> Explorer.Series.slice(s, -3, 2)
#Explorer.Series<
Polars[2]
s64 [3, 4]
>If the offset runs past the end of the series, the series is empty:
iex> s = Explorer.Series.from_list([1, 2, 3, 4, 5])
iex> Explorer.Series.slice(s, 10, 3)
#Explorer.Series<
Polars[0]
s64 []
>If the size runs past the end of the series, the result may be shorter than the size:
iex> s = Explorer.Series.from_list([1, 2, 3, 4, 5])
iex> Explorer.Series.slice(s, -3, 4)
#Explorer.Series<
Polars[3]
s64 [3, 4, 5]
>Sorts the series.
See sort_by/3 for an expression-based sorting function.
See sort_with/3 for a callback-based sorting function.
Options
:direction-:ascor:desc, meaning "ascending" or "descending", respectively. By default it sorts in ascending order.:nils-:firstor:last. By default it is:lastif direction is:asc, and:firstotherwise.:parallel- boolean. Whether to parallelize the sorting. By default it istrue.:stable- boolean. Determines if the sorting is stable (ties are guaranteed to maintain their order) or not. Unstable sorting may be more performant. By default it isfalse.
Examples
iex> s = Explorer.Series.from_list([9, 3, 7, 1])
iex> Explorer.Series.sort(s)
#Explorer.Series<
Polars[4]
s64 [1, 3, 7, 9]
>
iex> s = Explorer.Series.from_list([9, 3, 7, 1])
iex> Explorer.Series.sort(s, direction: :desc)
#Explorer.Series<
Polars[4]
s64 [9, 7, 3, 1]
>Sorts the series based on an expression.
Notice
This is a macro.
- You must
require Explorer.Seriesbefore using it.- You must use the special
_syntax. See the moduledoc for details.
See sort_with/3 for the callback-based version of this function.
Options
:direction-:ascor:desc, meaning "ascending" or "descending", respectively. By default it sorts in ascending order.:nils-:firstor:last. By default it is:lastif direction is:asc, and:firstotherwise.:parallel- boolean. Whether to parallelize the sorting. By default it istrue.Parallel sort isn't available on certain lazy operations. In those situations this option is ignored.
:stable- boolean. Determines if the sorting is stable (ties are guaranteed to maintain their order) or not. Unstable sorting may be more performant. By default it isfalse.
Examples
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.sort_by(s, remainder(_, 3))
#Explorer.Series<
Polars[3]
s64 [3, 1, 2]
>
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.sort_by(s, remainder(_, 3), direction: :desc)
#Explorer.Series<
Polars[3]
s64 [2, 1, 3]
>
iex> s = Explorer.Series.from_list([1, nil, 2, 3])
iex> Explorer.Series.sort_by(s, -2 * _, nils: :first)
#Explorer.Series<
Polars[4]
s64 [nil, 3, 2, 1]
>Sorts the series based on a callback that returns a lazy series.
See sort_by/3 for the expression-based version of this function.
Options
:direction-:ascor:desc, meaning "ascending" or "descending", respectively. By default it sorts in ascending order.:nils-:firstor:last. By default it is:lastif direction is:asc, and:firstotherwise.:parallel- boolean. Whether to parallelize the sorting. By default it istrue.Parallel sort isn't available on certain lazy operations. In those situations this option is ignored.
:stable- boolean. Determines if the sorting is stable (ties are guaranteed to maintain their order) or not. Unstable sorting may be more performant. By default it isfalse.
Examples
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.sort_with(s, &Explorer.Series.remainder(&1, 3))
#Explorer.Series<
Polars[3]
s64 [3, 1, 2]
>
iex> s = Explorer.Series.from_list([1, 2, 3])
iex> Explorer.Series.sort_with(s, &Explorer.Series.remainder(&1, 3), direction: :desc)
#Explorer.Series<
Polars[3]
s64 [2, 1, 3]
>
iex> s = Explorer.Series.from_list([1, nil, 2, 3])
iex> Explorer.Series.sort_with(s, &Explorer.Series.multiply(-2, &1), nils: :first)
#Explorer.Series<
Polars[4]
s64 [nil, 3, 2, 1]
>Returns the last N elements of the series.
Examples
iex> s = 1..100 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.tail(s)
#Explorer.Series<
Polars[10]
s64 [91, 92, 93, 94, 95, 96, 97, 98, 99, 100]
>Returns the unique values of the series, but does not maintain order.
Faster than distinct/1.
Examples
iex> s = [1, 1, 2, 2, 3, 3] |> Explorer.Series.from_list()
iex> Explorer.Series.unordered_distinct(s)Functions: Window
Calculates the cumulative maximum of the series.
Optionally, can accumulate in reverse.
Does not fill nil values. See fill_missing/2.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :date:time:datetime:duration
Examples
iex> s = [1, 2, 3, 4] |> Explorer.Series.from_list()
iex> Explorer.Series.cumulative_max(s)
#Explorer.Series<
Polars[4]
s64 [1, 2, 3, 4]
>
iex> s = [1, 2, nil, 4] |> Explorer.Series.from_list()
iex> Explorer.Series.cumulative_max(s)
#Explorer.Series<
Polars[4]
s64 [1, 2, nil, 4]
>
iex> s = [~T[03:00:02.000000], ~T[02:04:19.000000], nil, ~T[13:24:56.000000]] |> Explorer.Series.from_list()
iex> Explorer.Series.cumulative_max(s)
#Explorer.Series<
Polars[4]
time [03:00:02.000000, 03:00:02.000000, nil, 13:24:56.000000]
>Calculates the cumulative minimum of the series.
Optionally, can accumulate in reverse.
Does not fill nil values. See fill_missing/2.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :date:time:datetime:duration
Examples
iex> s = [1, 2, 3, 4] |> Explorer.Series.from_list()
iex> Explorer.Series.cumulative_min(s)
#Explorer.Series<
Polars[4]
s64 [1, 1, 1, 1]
>
iex> s = [1, 2, nil, 4] |> Explorer.Series.from_list()
iex> Explorer.Series.cumulative_min(s)
#Explorer.Series<
Polars[4]
s64 [1, 1, nil, 1]
>
iex> s = [~T[03:00:02.000000], ~T[02:04:19.000000], nil, ~T[13:24:56.000000]] |> Explorer.Series.from_list()
iex> Explorer.Series.cumulative_min(s)
#Explorer.Series<
Polars[4]
time [03:00:02.000000, 02:04:19.000000, nil, 02:04:19.000000]
>Calculates the cumulative product of the series.
Optionally, can accumulate in reverse.
Does not fill nil values. See fill_missing/2.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64}
Examples
iex> s = [1, 2, 3, 2] |> Explorer.Series.from_list()
iex> Explorer.Series.cumulative_product(s)
#Explorer.Series<
Polars[4]
s64 [1, 2, 6, 12]
>
iex> s = [1, 2, nil, 4] |> Explorer.Series.from_list()
iex> Explorer.Series.cumulative_product(s)
#Explorer.Series<
Polars[4]
s64 [1, 2, nil, 8]
>Calculates the cumulative sum of the series.
Optionally, can accumulate in reverse.
Does not fill nil values. See fill_missing/2.
Supported dtypes
- floats:
{:f, 32}and{:f, 64} - integers:
{:s, 8},{:s, 16},{:s, 32},{:s, 64},{:u, 8},{:u, 16},{:u, 32}and{:u, 64} :boolean
Examples
iex> s = [1, 2, 3, 4] |> Explorer.Series.from_list()
iex> Explorer.Series.cumulative_sum(s)
#Explorer.Series<
Polars[4]
s64 [1, 3, 6, 10]
>
iex> s = [1, 2, nil, 4] |> Explorer.Series.from_list()
iex> Explorer.Series.cumulative_sum(s)
#Explorer.Series<
Polars[4]
s64 [1, 3, nil, 7]
>Calculate the exponentially weighted moving average, given smoothing factor alpha.
Options
:alpha- Optional smoothing factor which specifies the importance given to most recent observations. It is a value such that, 0 < alpha <= 1. Defaults to 0.5.:adjust- If set to true, it corrects the bias introduced by smoothing process. Defaults totrue.:min_periods- The number of values in the window that should be non-nil before computing a result. Defaults to1.:ignore_nils- If set to true, it ignore nulls in the calculation. Defaults totrue.
Examples
iex> s = 1..5 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.ewm_mean(s)
#Explorer.Series<
Polars[5]
f64 [1.0, 1.6666666666666667, 2.4285714285714284, 3.2666666666666666, 4.161290322580645]
>
iex> s = 1..5 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.ewm_mean(s, alpha: 0.1)
#Explorer.Series<
Polars[5]
f64 [1.0, 1.5263157894736843, 2.070110701107011, 2.6312881651642916, 3.2097140484969833]
>Calculate the exponentially weighted moving standard deviation, given smoothing factor alpha.
Options
:alpha- Optional smoothing factor which specifies the importance given to most recent observations. It is a value such that, 0 < alpha <= 1. Defaults to 0.5.:adjust- If set to true, it corrects the bias introduced by smoothing process. Defaults totrue.:bias- If set to false, it corrects the estimate to be statistically unbiased. Defaults tofalse.:min_periods- The number of values in the window that should be non-nil before computing a result. Defaults to1.:ignore_nils- If set to true, it ignore nulls in the calculation. Defaults totrue.
Examples
iex> s = 1..5 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.ewm_standard_deviation(s)
#Explorer.Series<
Polars[5]
f64 [0.0, 0.7071067811865476, 0.9636241116594314, 1.1771636613972951, 1.3452425132127066]
>
iex> s = 1..5 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.ewm_standard_deviation(s, alpha: 0.1)
#Explorer.Series<
Polars[5]
f64 [0.0, 0.7071067811865476, 0.9990770648702808, 1.2879021599718157, 1.5741638698820746]
>Calculate the exponentially weighted moving variance, given smoothing factor alpha.
Options
:alpha- Optional smoothing factor which specifies the importance given to most recent observations. It is a value such that, 0 < alpha <= 1. Defaults to 0.5.:adjust- If set to true, it corrects the bias introduced by smoothing process. Defaults totrue.:bias- If set to false, it corrects the estimate to be statistically unbiased. Defaults tofalse.:min_periods- The number of values in the window that should be non-nil before computing a result. Defaults to1.:ignore_nils- If set to true, it ignore nulls in the calculation. Defaults totrue.
Examples
iex> s = 1..5 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.ewm_variance(s)
#Explorer.Series<
Polars[5]
f64 [0.0, 0.5, 0.9285714285714284, 1.385714285714286, 1.8096774193548393]
>
iex> s = 1..5 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.ewm_variance(s, alpha: 0.1)
#Explorer.Series<
Polars[5]
f64 [0.0, 0.5, 0.9981549815498153, 1.6586919736600685, 2.4779918892421087]
>@spec fill_missing( t(), :forward | :backward | :max | :min | :mean | :nan | :infinity | :neg_infinity | Explorer.Backend.Series.valid_types() ) :: t()
Fill missing values with the given strategy. If a scalar value is provided instead of a strategy
atom, nil will be replaced with that value. It must be of the same dtype as the series.
Strategies
:forward- replace nil with the previous value:backward- replace nil with the next value:max- replace nil with the series maximum:min- replace nil with the series minimum:mean- replace nil with the series mean:nan(float only) - replace nil withNaN
Examples
iex> s = Explorer.Series.from_list([1, 2, nil, 4])
iex> Explorer.Series.fill_missing(s, :forward)
#Explorer.Series<
Polars[4]
s64 [1, 2, 2, 4]
>
iex> s = Explorer.Series.from_list([1, 2, nil, 4])
iex> Explorer.Series.fill_missing(s, :backward)
#Explorer.Series<
Polars[4]
s64 [1, 2, 4, 4]
>
iex> s = Explorer.Series.from_list([1, 2, nil, 4])
iex> Explorer.Series.fill_missing(s, :max)
#Explorer.Series<
Polars[4]
s64 [1, 2, 4, 4]
>
iex> s = Explorer.Series.from_list([1, 2, nil, 4])
iex> Explorer.Series.fill_missing(s, :min)
#Explorer.Series<
Polars[4]
s64 [1, 2, 1, 4]
>
iex> s = Explorer.Series.from_list([1, 2, nil, 4])
iex> Explorer.Series.fill_missing(s, :mean)
#Explorer.Series<
Polars[4]
s64 [1, 2, 2, 4]
>Values that belong to the series itself can also be added as missing:
iex> s = Explorer.Series.from_list([1, 2, nil, 4])
iex> Explorer.Series.fill_missing(s, 3)
#Explorer.Series<
Polars[4]
s64 [1, 2, 3, 4]
>
iex> s = Explorer.Series.from_list(["a", "b", nil, "d"])
iex> Explorer.Series.fill_missing(s, "c")
#Explorer.Series<
Polars[4]
string ["a", "b", "c", "d"]
>Mismatched types will raise:
iex> s = Explorer.Series.from_list([1, 2, nil, 4])
iex> Explorer.Series.fill_missing(s, "foo")
** (ArgumentError) cannot invoke Explorer.Series.fill_missing/2 with mismatched dtypes: {:s, 64} and "foo"Floats in particular accept missing values to be set to NaN, Inf, and -Inf:
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 4.0])
iex> Explorer.Series.fill_missing(s, :nan)
#Explorer.Series<
Polars[4]
f64 [1.0, 2.0, NaN, 4.0]
>
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 4.0])
iex> Explorer.Series.fill_missing(s, :infinity)
#Explorer.Series<
Polars[4]
f64 [1.0, 2.0, Inf, 4.0]
>
iex> s = Explorer.Series.from_list([1.0, 2.0, nil, 4.0])
iex> Explorer.Series.fill_missing(s, :neg_infinity)
#Explorer.Series<
Polars[4]
f64 [1.0, 2.0, -Inf, 4.0]
>Calculate the rolling max, given a window size and optional list of weights.
Options
:weights- An optional list of weights with the same length as the window that will be multiplied elementwise with the values in the window. Defaults tonil.:min_periods- The number of values in the window that should be non-nil before computing a result. Ifnil, it will be set equal to window size. Defaults to1.:center- Set the labels at the center of the window. Defaults tofalse.
Examples
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.window_max(s, 4)
#Explorer.Series<
Polars[10]
s64 [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.window_max(s, 2, weights: [1.0, 2.0])
#Explorer.Series<
Polars[10]
f64 [1.0, 4.0, 6.0, 8.0, 10.0, 12.0, 14.0, 16.0, 18.0, 20.0]
>Calculate the rolling mean, given a window size and optional list of weights.
Options
:weights- An optional list of weights with the same length as the window that will be multiplied elementwise with the values in the window. Defaults tonil.:min_periods- The number of values in the window that should be non-nil before computing a result. Ifnil, it will be set equal to window size. Defaults to1.:center- Set the labels at the center of the window. Defaults tofalse.
Examples
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.window_mean(s, 4)
#Explorer.Series<
Polars[10]
f64 [1.0, 1.5, 2.0, 2.5, 3.5, 4.5, 5.5, 6.5, 7.5, 8.5]
>
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.window_mean(s, 2, weights: [0.25, 0.75])
#Explorer.Series<
Polars[10]
f64 [0.25, 1.75, 2.75, 3.75, 4.75, 5.75, 6.75, 7.75, 8.75, 9.75]
>
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.window_mean(s, 2, weights: [0.25, 0.75], min_periods: nil)
#Explorer.Series<
Polars[10]
f64 [nil, 1.75, 2.75, 3.75, 4.75, 5.75, 6.75, 7.75, 8.75, 9.75]
>Calculate the rolling median, given a window size and optional list of weights.
Options
:weights- An optional list of weights with the same length as the window that will be multiplied elementwise with the values in the window. Defaults tonil.:min_periods- The number of values in the window that should be non-nil before computing a result. Ifnil, it will be set equal to window size. Defaults to1.:center- Set the labels at the center of the window. Defaults tofalse.
Examples
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.window_median(s, 4)
#Explorer.Series<
Polars[10]
f64 [1.0, 1.5, 2.0, 2.5, 3.5, 4.5, 5.5, 6.5, 7.5, 8.5]
>
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.window_median(s, 2, weights: [0.25, 0.75])
#Explorer.Series<
Polars[10]
f64 [1.5, 1.5, 2.5, 3.5, 4.5, 5.5, 6.5, 7.5, 8.5, 9.5]
>
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.window_median(s, 2, weights: [0.25, 0.75], min_periods: nil)
#Explorer.Series<
Polars[10]
f64 [nil, 1.5, 2.5, 3.5, 4.5, 5.5, 6.5, 7.5, 8.5, 9.5]
>Calculate the rolling min, given a window size and optional list of weights.
Options
:weights- An optional list of weights with the same length as the window that will be multiplied elementwise with the values in the window. Defaults tonil.:min_periods- The number of values in the window that should be non-nil before computing a result. Ifnil, it will be set equal to window size. Defaults to1.:center- Set the labels at the center of the window. Defaults tofalse.
Examples
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.window_min(s, 4)
#Explorer.Series<
Polars[10]
s64 [1, 1, 1, 1, 2, 3, 4, 5, 6, 7]
>
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.window_min(s, 2, weights: [1.0, 2.0])
#Explorer.Series<
Polars[10]
f64 [1.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0]
>Calculate the rolling standard deviation, given a window size and optional list of weights.
Options
:weights- An optional list of weights with the same length as the window that will be multiplied elementwise with the values in the window. Defaults tonil.:min_periods- The number of values in the window that should be non-nil before computing a result. Ifnil, it will be set equal to window size. Defaults to1.:center- Set the labels at the center of the window. Defaults tofalse.
Examples
iex> s = Explorer.Series.from_list([1, 2, 3, 4, 1])
iex> Explorer.Series.window_standard_deviation(s, 2)
#Explorer.Series<
Polars[5]
f64 [nil, 0.7071067811865476, 0.7071067811865476, 0.7071067811865476, 2.1213203435596424]
>
iex> s = Explorer.Series.from_list([1, 2, 3, 4, 5, 6])
iex> Explorer.Series.window_standard_deviation(s, 2, weights: [0.25, 0.75])
#Explorer.Series<
Polars[6]
f64 [0.4330127018922193, 0.4330127018922193, 0.4330127018922193, 0.4330127018922193, 0.4330127018922193, 0.4330127018922193]
>Calculate the rolling sum, given a window size and optional list of weights.
Options
:weights- An optional list of weights with the same length as the window that will be multiplied elementwise with the values in the window. Defaults tonil.:min_periods- The number of values in the window that should be non-nil before computing a result. Ifnil, it will be set equal to window size. Defaults to1.:center- Set the labels at the center of the window. Defaults tofalse.
Examples
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.window_sum(s, 4)
#Explorer.Series<
Polars[10]
s64 [1, 3, 6, 10, 14, 18, 22, 26, 30, 34]
>
iex> s = 1..10 |> Enum.to_list() |> Explorer.Series.from_list()
iex> Explorer.Series.window_sum(s, 2, weights: [1.0, 2.0])
#Explorer.Series<
Polars[10]
f64 [1.0, 5.0, 8.0, 11.0, 14.0, 17.0, 20.0, 23.0, 26.0, 29.0]
>Types
@type datetime_dtype() :: {:datetime, time_unit()}
@type dtype() :: :null | :binary | :boolean | :category | :date | :time | :string | datetime_dtype() | duration_dtype() | float_dtype() | list_dtype() | signed_integer_dtype() | struct_dtype() | unsigned_integer_dtype()
@type duration_dtype() :: {:duration, time_unit()}
@type float_dtype() :: {:f, 32} | {:f, 64}
@type float_dtype_alias() :: :float | :f32 | :f64
@type inferable_scalar() :: number() | non_finite() | boolean() | map() | String.t() | Date.t() | Time.t() | NaiveDateTime.t()
@type integer_dtype_alias() ::
:integer | :u8 | :u16 | :u32 | :u64 | :s8 | :s16 | :s32 | :s64
@type lazy_t() :: %Explorer.Series{ data: Explorer.Backend.LazySeries.t(), dtype: dtype(), name: term() }
@type list_dtype() :: {:list, dtype()}
@type non_finite() :: :nan | :infinity | :neg_infinity
@type signed_integer_dtype() :: {:s, 8} | {:s, 16} | {:s, 32} | {:s, 64}
@type t() :: %Explorer.Series{ data: Explorer.Backend.Series.t(), dtype: dtype(), name: term() }
@type time_unit() :: :nanosecond | :microsecond | :millisecond
@type unsigned_integer_dtype() :: {:u, 8} | {:u, 16} | {:u, 32} | {:u, 64}