@spec arrange(
  df :: t(),
  column_or_columns :: column() | columns() | [{:asc | :desc, column()}]
) :: t()

Arranges/sorts rows by columns.

examples

Examples

A single column name will sort ascending by that column:

iex> df = Explorer.DataFrame.new(a: ["b", "c", "a"], b: [1, 2, 3])
iex> Explorer.DataFrame.arrange(df, "a")
#Explorer.DataFrame<
  Polars[3 x 2]
  a string ["a", "b", "c"]
  b integer [3, 1, 2]
>

You can also sort descending:

iex> df = Explorer.DataFrame.new(a: ["b", "c", "a"], b: [1, 2, 3])
iex> Explorer.DataFrame.arrange(df, desc: "a")
#Explorer.DataFrame<
  Polars[3 x 2]
  a string ["c", "b", "a"]
  b integer [2, 1, 3]
>

Sorting by more than one column sorts them in the order they are entered:

iex> df = Explorer.Datasets.fossil_fuels()
iex> Explorer.DataFrame.arrange(df, asc: "total", desc: "country")
#Explorer.DataFrame<
  Polars[1094 x 10]
  year integer [2010, 2010, 2011, 2011, 2012, ...]
  country string ["NIUE", "TUVALU", "TUVALU", "NIUE", "NIUE", ...]
  total integer [1, 2, 2, 2, 2, ...]
  solid_fuel integer [0, 0, 0, 0, 0, ...]
  liquid_fuel integer [1, 2, 2, 2, 2, ...]
  gas_fuel integer [0, 0, 0, 0, 0, ...]
  cement integer [0, 0, 0, 0, 0, ...]
  gas_flaring integer [0, 0, 0, 0, 0, ...]
  per_capita float [0.52, 0.0, 0.0, 1.04, 1.04, ...]
  bunker_fuels integer [0, 0, 0, 0, 0, ...]
>

Alternatively you can pass a callback to sort by the given columns:

iex> df = Explorer.DataFrame.new(a: ["b", "c", "a"], b: [1, 2, 3])
iex> Explorer.DataFrame.arrange(df, &String.starts_with?(&1, "a"))
#Explorer.DataFrame<
  Polars[3 x 2]
  a string ["a", "b", "c"]
  b integer [3, 1, 2]
>

Or a callback to sort the columns of a given type:

iex> df = Explorer.DataFrame.new(a: ["b", "c", "a"], b: [1, 2, 3])
iex> Explorer.DataFrame.arrange(df, fn _name, type -> type == :string end)
#Explorer.DataFrame<
  Polars[3 x 2]
  a string ["a", "b", "c"]
  b integer [3, 1, 2]
>

@spec arrange_with(
  df :: t(),
  (Explorer.Backend.LazyFrame.t() ->
     Explorer.Series.lazy_t()
     | [Explorer.Series.lazy_t()]
     | [{:asc | :desc, Explorer.Series.lazy_t()}])
) :: t()

Arranges/sorts rows by columns using a callback function.

examples

Examples

A single column name will sort ascending by that column:

iex> df = Explorer.DataFrame.new(a: ["b", "c", "a"], b: [1, 2, 3])
iex> Explorer.DataFrame.arrange_with(df, &(&1["a"]))
#Explorer.DataFrame<
  Polars[3 x 2]
  a string ["a", "b", "c"]
  b integer [3, 1, 2]
>

You can also sort descending:

iex> df = Explorer.DataFrame.new(a: ["b", "c", "a"], b: [1, 2, 3])
iex> Explorer.DataFrame.arrange_with(df, &[desc: &1["a"]])
#Explorer.DataFrame<
  Polars[3 x 2]
  a string ["c", "b", "a"]
  b integer [2, 1, 3]
>

Sorting by more than one column sorts them in the order they are entered:

iex> df = Explorer.DataFrame.new(a: [3, 1, 3], b: [2, 1, 3])
iex> Explorer.DataFrame.arrange_with(df, &[desc: &1["a"], asc: &1["b"]])
#Explorer.DataFrame<
  Polars[3 x 2]
  a integer [3, 3, 1]
  b integer [2, 3, 1]
>

@spec collect(df :: t()) :: {:ok, t()} | {:error, term()}

This collects the lazy data frame into an eager one, computing the query.

If already eager, this is a noop.

@spec distinct(df :: t(), columns :: columns(), opts :: Keyword.t()) :: t()

Takes distinct rows by a selection of columns.

Distinct is not affected by groups, although groups are kept in the columns selection if keep_all option is false (the default).

options

Options

• keep_all - If set to true, keep all columns. Default is false.

examples

Examples

By default will return unique values of the requested columns:

iex> df = Explorer.Datasets.fossil_fuels()
iex> Explorer.DataFrame.distinct(df, ["year", "country"])
#Explorer.DataFrame<
  Polars[1094 x 2]
  year integer [2010, 2010, 2010, 2010, 2010, ...]
  country string ["AFGHANISTAN", "ALBANIA", "ALGERIA", "ANDORRA", "ANGOLA", ...]
>

If keep_all is set to true, then the first value of each column not in the requested columns will be returned:

iex> df = Explorer.Datasets.fossil_fuels()
iex> Explorer.DataFrame.distinct(df, ["year", "country"], keep_all: true)
#Explorer.DataFrame<
  Polars[1094 x 10]
  year integer [2010, 2010, 2010, 2010, 2010, ...]
  country string ["AFGHANISTAN", "ALBANIA", "ALGERIA", "ANDORRA", "ANGOLA", ...]
  total integer [2308, 1254, 32500, 141, 7924, ...]
  solid_fuel integer [627, 117, 332, 0, 0, ...]
  liquid_fuel integer [1601, 953, 12381, 141, 3649, ...]
  gas_fuel integer [74, 7, 14565, 0, 374, ...]
  cement integer [5, 177, 2598, 0, 204, ...]
  gas_flaring integer [0, 0, 2623, 0, 3697, ...]
  per_capita float [0.08, 0.43, 0.9, 1.68, 0.37, ...]
  bunker_fuels integer [9, 7, 663, 0, 321, ...]
>

A callback on the dataframe's names can be passed instead of a list (like select/3):

iex> df = Explorer.DataFrame.new(x1: [1, 3, 3], x2: ["a", "c", "c"], y1: [1, 2, 3])
iex> Explorer.DataFrame.distinct(df, &String.starts_with?(&1, "x"))
#Explorer.DataFrame<
  Polars[2 x 2]
  x1 integer [1, 3]
  x2 string ["a", "c"]
>

If the dataframe has groups, then the columns of each group will be added to the distinct columns:

iex> df = Explorer.DataFrame.new(x1: [1, 3, 3], x2: ["a", "c", "c"], y1: [1, 2, 3])
iex> df = Explorer.DataFrame.group_by(df, "x1")
iex> Explorer.DataFrame.distinct(df, ["x2"])
#Explorer.DataFrame<
  Polars[2 x 2]
  Groups: ["x1"]
  x1 integer [1, 3]
  x2 string ["a", "c"]
>

@spec drop_nil(df :: t(), column() | columns()) :: t()

Drop nil values.

Optionally accepts a subset of columns.

examples

Examples

To drop nils on all columns:

iex> df = Explorer.DataFrame.new(a: [1, 2, nil], b: [1, nil, 3])
iex> Explorer.DataFrame.drop_nil(df)
#Explorer.DataFrame<
  Polars[1 x 2]
  a integer [1]
  b integer [1]
>

To select some columns:

iex> df = Explorer.DataFrame.new(a: [1, 2, nil], b: [1, nil, 3], c: [nil, 5, 6])
iex> Explorer.DataFrame.drop_nil(df, [:a, :c])
#Explorer.DataFrame<
  Polars[1 x 3]
  a integer [2]
  b integer [nil]
  c integer [5]
>

To select some columns by range:

iex> df = Explorer.DataFrame.new(a: [1, 2, nil], b: [1, nil, 3], c: [nil, 5, 6])
iex> Explorer.DataFrame.drop_nil(df, 0..1)
#Explorer.DataFrame<
  Polars[1 x 3]
  a integer [1]
  b integer [1]
  c integer [nil]
>

Or to select columns by a callback on the names:

iex> df = Explorer.DataFrame.new(a: [1, 2, nil], b: [1, nil, 3], c: [nil, 5, 6])
iex> Explorer.DataFrame.drop_nil(df, fn name -> name == "a" or name == "b" end)
#Explorer.DataFrame<
  Polars[1 x 3]
  a integer [1]
  b integer [1]
  c integer [nil]
>

Or to select columns by a callback on the names and types:

iex> df = Explorer.DataFrame.new(a: [1, 2, nil], b: [1, nil, 3], c: [nil, 5.0, 6.0])
iex> Explorer.DataFrame.drop_nil(df, fn _name, type -> type == :float end)
#Explorer.DataFrame<
  Polars[2 x 3]
  a integer [2, nil]
  b integer [nil, 3]
  c float [5.0, 6.0]
>

@spec dummies(df :: t(), column() | columns()) :: t()

Turns a set of columns to dummy variables.

examples

Examples

To mark a single column as dummy:

iex> df = Explorer.DataFrame.new(a: ["a", "b", "a", "c"], b: ["b", "a", "b", "d"])
iex> Explorer.DataFrame.dummies(df, "a")
#Explorer.DataFrame<
  Polars[4 x 3]
  a_a integer [1, 0, 1, 0]
  a_b integer [0, 1, 0, 0]
  a_c integer [0, 0, 0, 1]
>

Or multiple columns:

iex> df = Explorer.DataFrame.new(a: ["a", "b", "a", "c"], b: ["b", "a", "b", "d"])
iex> Explorer.DataFrame.dummies(df, ["a", "b"])
#Explorer.DataFrame<
  Polars[4 x 6]
  a_a integer [1, 0, 1, 0]
  a_b integer [0, 1, 0, 0]
  a_c integer [0, 0, 0, 1]
  b_a integer [0, 1, 0, 0]
  b_b integer [1, 0, 1, 0]
  b_d integer [0, 0, 0, 1]
>

Or all string columns:

iex> df = Explorer.DataFrame.new(num: [1, 2, 3, 4], b: ["b", "a", "b", "d"])
iex> Explorer.DataFrame.dummies(df, fn _name, type -> type == :string end)
#Explorer.DataFrame<
  Polars[4 x 3]
  b_a integer [0, 1, 0, 0]
  b_b integer [1, 0, 1, 0]
  b_d integer [0, 0, 0, 1]
>

@spec filter_with(
  df :: t(),
  callback :: (Explorer.Backend.LazyFrame.t() -> Explorer.Series.lazy_t())
) :: t()

Picks rows based on a callback function.

This function is efficient because it uses a representation of the series without pulling them. The only restriction is that you need to use a function that returns boolean.

But you can also use window functions and aggregations inside comparisons.

examples

Examples

iex> df = Explorer.DataFrame.new(col1: ["a", "b", "c"], col2: [1, 2, 3])
iex> Explorer.DataFrame.filter_with(df, &Explorer.Series.greater(&1["col2"], 2))
#Explorer.DataFrame<
  Polars[1 x 2]
  col1 string ["c"]
  col2 integer [3]
>

iex> df = Explorer.DataFrame.new(col1: ["a", "b", "c"], col2: [1, 2, 3])
iex> Explorer.DataFrame.filter_with(df, fn df -> Explorer.Series.equal(df["col1"], "b") end)
#Explorer.DataFrame<
  Polars[1 x 2]
  col1 string ["b"]
  col2 integer [2]
>

iex> df = Explorer.DataFrame.new(col1: ["a", "b", "c"], col2: [1, 2, 3])
iex> Explorer.DataFrame.filter_with(df, fn df -> Explorer.Series.cumulative_max(df["col2"]) end)
** (ArgumentError) expecting the function to return a boolean LazySeries, but instead it returned a LazySeries of type :integer

But it's possible to use a boolean operation based on another function:

iex> df = Explorer.DataFrame.new(col1: ["a", "b", "c"], col2: [1, 2, 3])
iex> Explorer.DataFrame.filter_with(df, fn df -> Explorer.Series.equal(Explorer.Series.cumulative_max(df["col2"]), 1) end)
#Explorer.DataFrame<
  Polars[1 x 2]
  col1 string ["a"]
  col2 integer [1]
>

@spec group_by(df :: t(), groups_or_group :: column_names() | column_name()) :: t()

Group the dataframe by one or more variables.

When the dataframe has grouping variables, operations are performed per group. Explorer.DataFrame.ungroup/2 removes grouping.

examples

Examples

You can group by a single variable:

iex> df = Explorer.Datasets.fossil_fuels()
iex> Explorer.DataFrame.group_by(df, "country")
#Explorer.DataFrame<
  Polars[1094 x 10]
  Groups: ["country"]
  year integer [2010, 2010, 2010, 2010, 2010, ...]
  country string ["AFGHANISTAN", "ALBANIA", "ALGERIA", "ANDORRA", "ANGOLA", ...]
  total integer [2308, 1254, 32500, 141, 7924, ...]
  solid_fuel integer [627, 117, 332, 0, 0, ...]
  liquid_fuel integer [1601, 953, 12381, 141, 3649, ...]
  gas_fuel integer [74, 7, 14565, 0, 374, ...]
  cement integer [5, 177, 2598, 0, 204, ...]
  gas_flaring integer [0, 0, 2623, 0, 3697, ...]
  per_capita float [0.08, 0.43, 0.9, 1.68, 0.37, ...]
  bunker_fuels integer [9, 7, 663, 0, 321, ...]
>

Or you can group by multiple:

iex> df = Explorer.Datasets.fossil_fuels()
iex> Explorer.DataFrame.group_by(df, ["country", "year"])
#Explorer.DataFrame<
  Polars[1094 x 10]
  Groups: ["country", "year"]
  year integer [2010, 2010, 2010, 2010, 2010, ...]
  country string ["AFGHANISTAN", "ALBANIA", "ALGERIA", "ANDORRA", "ANGOLA", ...]
  total integer [2308, 1254, 32500, 141, 7924, ...]
  solid_fuel integer [627, 117, 332, 0, 0, ...]
  liquid_fuel integer [1601, 953, 12381, 141, 3649, ...]
  gas_fuel integer [74, 7, 14565, 0, 374, ...]
  cement integer [5, 177, 2598, 0, 204, ...]
  gas_flaring integer [0, 0, 2623, 0, 3697, ...]
  per_capita float [0.08, 0.43, 0.9, 1.68, 0.37, ...]
  bunker_fuels integer [9, 7, 663, 0, 321, ...]
>

@spec mask(df :: t(), mask :: Explorer.Series.t() | [boolean()]) :: t()

Picks rows based on a list or series of values.

examples

Examples

This function must only be used when you need to select rows based on external values that are not available to the dataframe. For example, you can pass a list:

iex> df = Explorer.DataFrame.new(col1: ["a", "b", "c"], col2: [1, 2, 3])
iex> Explorer.DataFrame.mask(df, [false, true, false])
#Explorer.DataFrame<
  Polars[1 x 2]
  col1 string ["b"]
  col2 integer [2]
>

You must avoid using masks when the masks themselves are computed from other columns. For example, DO NOT do this:

iex> df = Explorer.DataFrame.new(col1: ["a", "b", "c"], col2: [1, 2, 3])
iex> Explorer.DataFrame.mask(df, Explorer.Series.greater(df["col2"], 1))
#Explorer.DataFrame<
  Polars[2 x 2]
  col1 string ["b", "c"]
  col2 integer [2, 3]
>

Instead, do this:

iex> df = Explorer.DataFrame.new(col1: ["a", "b", "c"], col2: [1, 2, 3])
iex> Explorer.DataFrame.filter_with(df, fn df -> Explorer.Series.greater(df["col2"], 1) end)
#Explorer.DataFrame<
  Polars[2 x 2]
  col1 string ["b", "c"]
  col2 integer [2, 3]
>

The filter_with/2 version is much more efficient because it doesn't need to create intermediate series representations to apply the mask.

@spec mutate(df :: t(), columns :: column_pairs(any())) :: t()

Creates and modifies columns.

Columns are added with keyword list or maps. New variables overwrite existing variables of the same name. Column names are coerced from atoms to strings.

examples

Examples

You can pass in a list directly as a new column:

iex> df = Explorer.DataFrame.new(a: ["a", "b", "c"], b: [1, 2, 3])
iex> Explorer.DataFrame.mutate(df, c: [4, 5, 6])
#Explorer.DataFrame<
  Polars[3 x 3]
  a string ["a", "b", "c"]
  b integer [1, 2, 3]
  c integer [4, 5, 6]
>

Or you can pass in a series:

iex> df = Explorer.DataFrame.new(a: ["a", "b", "c"], b: [1, 2, 3])
iex> s = Explorer.Series.from_list([4, 5, 6])
iex> Explorer.DataFrame.mutate(df, c: s)
#Explorer.DataFrame<
  Polars[3 x 3]
  a string ["a", "b", "c"]
  b integer [1, 2, 3]
  c integer [4, 5, 6]
>

You can overwrite existing columns:

iex> df = Explorer.DataFrame.new(a: ["a", "b", "c"], b: [1, 2, 3])
iex> Explorer.DataFrame.mutate(df, a: [4, 5, 6])
#Explorer.DataFrame<
  Polars[3 x 2]
  a integer [4, 5, 6]
  b integer [1, 2, 3]
>

Scalar values are repeated to fill the series:

iex> df = Explorer.DataFrame.new(a: ["a", "b", "c"], b: [1, 2, 3])
iex> Explorer.DataFrame.mutate(df, a: 4)
#Explorer.DataFrame<
  Polars[3 x 2]
  a integer [4, 4, 4]
  b integer [1, 2, 3]
>

Alternatively, all of the above works with a map instead of a keyword list:

iex> df = Explorer.DataFrame.new(a: ["a", "b", "c"], b: [1, 2, 3])
iex> Explorer.DataFrame.mutate(df, %{"c" => [4, 5, 6]})
#Explorer.DataFrame<
  Polars[3 x 3]
  a string ["a", "b", "c"]
  b integer [1, 2, 3]
  c integer [4, 5, 6]
>

@spec mutate_with(
  df :: t(),
  callback ::
    (Explorer.Backend.LazyFrame.t() -> column_pairs(Explorer.Series.lazy_t()))
) :: t()

Creates or modifies columns using a callback function.

This function is similar to mutate/2, but allows complex operations to be performed, since it uses a virtual representation of the dataframe. The only requirement is that a series operation is returned.

New variables overwrite existing variables of the same name. Column names are coerced from atoms to strings.

examples

Examples

iex> df = Explorer.DataFrame.new(a: [4, 5, 6], b: [1, 2, 3])
iex> Explorer.DataFrame.mutate_with(df, &[c: Explorer.Series.add(&1["a"], &1["b"])])
#Explorer.DataFrame<
  Polars[3 x 3]
  a integer [4, 5, 6]
  b integer [1, 2, 3]
  c integer [5, 7, 9]
>

You can overwrite existing columns:

iex> df = Explorer.DataFrame.new(a: ["a", "b", "c"], b: [1, 2, 3])
iex> Explorer.DataFrame.mutate_with(df, &[b: Explorer.Series.pow(&1["b"], 2)])
#Explorer.DataFrame<
  Polars[3 x 2]
  a string ["a", "b", "c"]
  b float [1.0, 4.0, 9.0]
>

Scalar values are repeated to fill the series:

iex> df = Explorer.DataFrame.new(a: ["a", "b", "c"], b: [1, 2, 3])
iex> Explorer.DataFrame.mutate_with(df, &[a: Explorer.Series.max(&1["b"])])
#Explorer.DataFrame<
  Polars[3 x 2]
  a integer [3, 3, 3]
  b integer [1, 2, 3]
>

Alternatively, all of the above works with a map instead of a keyword list:

iex> df = Explorer.DataFrame.new(a: ["a", "b", "c"], b: [1, 2, 3])
iex> Explorer.DataFrame.mutate_with(df, fn df -> %{"c" => Explorer.Series.window_mean(df["b"], 2)} end)
#Explorer.DataFrame<
  Polars[3 x 3]
  a string ["a", "b", "c"]
  b integer [1, 2, 3]
  c float [1.0, 1.5, 2.5]
>

Mutations in grouped dataframes are also possible:

iex> df = Explorer.DataFrame.new(id: ["a", "a", "b"], b: [1, 2, 3])
iex> grouped = Explorer.DataFrame.group_by(df, :id)
iex> Explorer.DataFrame.mutate_with(grouped, &[count: Explorer.Series.count(&1["b"])])
#Explorer.DataFrame<
  Polars[3 x 3]
  Groups: ["id"]
  id string ["a", "a", "b"]
  b integer [1, 2, 3]
  count integer [2, 2, 1]
>

@spec new(
  Table.Reader.t() | series_pairs,
  opts :: Keyword.t()
) :: t()
when series_pairs:
       %{required(column_name()) => Explorer.Series.t()}
       | [{column_name(), Explorer.Series.t()}]

Creates a new dataframe.

Accepts any tabular data adhering to the Table.Reader protocol, as well as a map or a keyword list with series.

options

Options

• backend - The Explorer backend to use. Defaults to the value returned by Explorer.Backend.get/0.

examples

Examples

From series:

iex> Explorer.DataFrame.new(%{floats: Explorer.Series.from_list([1.0, 2.0]), ints: Explorer.Series.from_list([1, nil])})
#Explorer.DataFrame<
  Polars[2 x 2]
  floats float [1.0, 2.0]
  ints integer [1, nil]
>

From columnar data:

iex> Explorer.DataFrame.new(%{floats: [1.0, 2.0], ints: [1, nil]})
#Explorer.DataFrame<
  Polars[2 x 2]
  floats float [1.0, 2.0]
  ints integer [1, nil]
>

iex> Explorer.DataFrame.new(floats: [1.0, 2.0], ints: [1, nil])
#Explorer.DataFrame<
  Polars[2 x 2]
  floats float [1.0, 2.0]
  ints integer [1, nil]
>

iex> Explorer.DataFrame.new(%{floats: [1.0, 2.0], ints: [1, "wrong"]})
** (ArgumentError) cannot create series "ints": the value "wrong" does not match the inferred series dtype :integer

From row data:

iex> rows = [%{id: 1, name: "José"}, %{id: 2, name: "Christopher"}, %{id: 3, name: "Cristine"}]
iex> Explorer.DataFrame.new(rows)
#Explorer.DataFrame<
  Polars[3 x 2]
  id integer [1, 2, 3]
  name string ["José", "Christopher", "Cristine"]
>

iex> rows = [[id: 1, name: "José"], [id: 2, name: "Christopher"], [id: 3, name: "Cristine"]]
iex> Explorer.DataFrame.new(rows)
#Explorer.DataFrame<
  Polars[3 x 2]
  id integer [1, 2, 3]
  name string ["José", "Christopher", "Cristine"]
>

@spec pivot_longer(
  df :: t(),
  columns_to_pivot :: columns(),
  opts :: Keyword.t()
) :: t()

Pivot data from wide to long.

Explorer.DataFrame.pivot_longer/3 "lengthens" data, increasing the number of rows and decreasing the number of columns. The inverse transformation is Explorer.DataFrame.pivot_wider/4.

The second argument (columns_to_pivot) can be either an array of column names to pivot or a filter callback on the dataframe's names. These columns must always have the same data type.

options

Options

• keep - Columns that are not in the list of pivot and should be kept in the dataframe. May be a filter callback on the dataframe's column names. Defaults to all columns except the ones to pivot.

• drop - Columns that are not in the list of pivot and should be dropped from the dataframe. May be a filter callback on the dataframe's column names. This list of columns is going to be subtracted from the list of keep. Defaults to an empty list.

• names_to - A string specifying the name of the column to create from the data stored in the column names of the dataframe. Defaults to "variable".

• values_to - A string specifying the name of the column to create from the data stored in series element values. Defaults to "value".

examples

Examples

iex> df = Explorer.Datasets.fossil_fuels()
iex> Explorer.DataFrame.pivot_longer(df, &String.ends_with?(&1, "fuel"))
#Explorer.DataFrame<
  Polars[3282 x 9]
  year integer [2010, 2010, 2010, 2010, 2010, ...]
  country string ["AFGHANISTAN", "ALBANIA", "ALGERIA", "ANDORRA", "ANGOLA", ...]
  total integer [2308, 1254, 32500, 141, 7924, ...]
  cement integer [5, 177, 2598, 0, 204, ...]
  gas_flaring integer [0, 0, 2623, 0, 3697, ...]
  per_capita float [0.08, 0.43, 0.9, 1.68, 0.37, ...]
  bunker_fuels integer [9, 7, 663, 0, 321, ...]
  variable string ["solid_fuel", "solid_fuel", "solid_fuel", "solid_fuel", "solid_fuel", ...]
  value integer [627, 117, 332, 0, 0, ...]
>

iex> df = Explorer.Datasets.fossil_fuels()
iex> Explorer.DataFrame.pivot_longer(df, &String.ends_with?(&1, "fuel"), keep: ["year", "country"])
#Explorer.DataFrame<
  Polars[3282 x 4]
  year integer [2010, 2010, 2010, 2010, 2010, ...]
  country string ["AFGHANISTAN", "ALBANIA", "ALGERIA", "ANDORRA", "ANGOLA", ...]
  variable string ["solid_fuel", "solid_fuel", "solid_fuel", "solid_fuel", "solid_fuel", ...]
  value integer [627, 117, 332, 0, 0, ...]
>

iex> df = Explorer.Datasets.fossil_fuels()
iex> Explorer.DataFrame.pivot_longer(df, ["total"], keep: ["year", "country"], drop: ["country"])
#Explorer.DataFrame<
  Polars[1094 x 3]
  year integer [2010, 2010, 2010, 2010, 2010, ...]
  variable string ["total", "total", "total", "total", "total", ...]
  value integer [2308, 1254, 32500, 141, 7924, ...]
>

iex> df = Explorer.Datasets.fossil_fuels()
iex> Explorer.DataFrame.pivot_longer(df, ["total"], keep: [], names_to: "my_var", values_to: "my_value")
#Explorer.DataFrame<
  Polars[1094 x 2]
  my_var string ["total", "total", "total", "total", "total", ...]
  my_value integer [2308, 1254, 32500, 141, 7924, ...]
>

@spec pivot_wider(
  df :: t(),
  names_from :: column(),
  values_from :: column(),
  opts :: Keyword.t()
) :: t()

Pivot data from long to wide.

Explorer.DataFrame.pivot_wider/4 "widens" data, increasing the number of columns and decreasing the number of rows. The inverse transformation is Explorer.DataFrame.pivot_longer/3.

Due to a restriction upstream, values_from must be a numeric type.

options

Options

• id_columns - A set of columns that uniquely identifies each observation. Defaults to all columns in data except for the columns specified in names_from and values_from. Typically used when you have redundant variables, i.e. variables whose values are perfectly correlated with existing variables. May accept a filter callback, a list or a range of column names. Default value is 0..-1. If an empty list is passed, or a range that results in a empty list of column names, it raises an error.

  ID columns cannot be of the float type and attempting so will raise an error. If you need to use float columns as IDs, you must carefully consider rounding or truncating the column and converting it to integer, as long as doing so preserves the properties of the column.

• names_prefix - String added to the start of every variable name. This is particularly useful if names_from is a numeric vector and you want to create syntactic variable names.

examples

Examples

iex> df = Explorer.DataFrame.new(id: [1, 1], variable: ["a", "b"], value: [1, 2])
iex> Explorer.DataFrame.pivot_wider(df, "variable", "value")
#Explorer.DataFrame<
  Polars[1 x 3]
  id integer [1]
  a integer [1]
  b integer [2]
>

@spec pull(df :: t(), column :: column()) :: Explorer.Series.t()

Extracts a single column as a series.

examples

Examples

iex> df = Explorer.Datasets.fossil_fuels()
iex> Explorer.DataFrame.pull(df, "total")
#Explorer.Series<
  integer[1094]
  [2308, 1254, 32500, 141, 7924, 41, 143, 51246, 1150, 684, 106589, 18408, 8366, 451, 7981, 16345, 403, 17192, 30222, 147, 1388, 166, 133, 5802, 1278, 114468, 47, 2237, 12030, 535, 58, 1367, 145806, 152, 152, 72, 141, 19703, 2393248, 20773, 44, 540, 19, 2064, 1900, 5501, 10465, 2102, 30428, 18122, ...]
>

iex> df = Explorer.Datasets.fossil_fuels()
iex> Explorer.DataFrame.pull(df, 2)
#Explorer.Series<
  integer[1094]
  [2308, 1254, 32500, 141, 7924, 41, 143, 51246, 1150, 684, 106589, 18408, 8366, 451, 7981, 16345, 403, 17192, 30222, 147, 1388, 166, 133, 5802, 1278, 114468, 47, 2237, 12030, 535, 58, 1367, 145806, 152, 152, 72, 141, 19703, 2393248, 20773, 44, 540, 19, 2064, 1900, 5501, 10465, 2102, 30428, 18122, ...]
>