Elixir bindings for the XGBoost library. EXGBoost provides an implementation of XGBoost that works with Nx tensors.

Xtreme Gradient Boosting (XGBoost) is an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable. It implements machine learning algorithms under the Gradient Boosting framework. XGBoost provides a parallel tree boosting (also known as GBDT, GBM) that solve many data science problems in a fast and accurate way. The same code runs on major distributed environment (Hadoop, SGE, MPI) and can solve problems beyond billions of examples.

Installation

def deps do
[
  {:exgboost, "~> 0.5"}
]
endcopy

API Data Structures

EXGBoost's top-level EXGBoost API works directly and only with Nx.Tensor for data representation and with EXGBoost.Booster structs as an internal representation. Direct manipulation of EXGBoost.Booster structs is discouraged.

Basic Usage

key = Nx.Random.key(42)
{x, key} = Nx.Random.normal(key, 0, 1, shape: {10, 5})
{y, key} = Nx.Random.normal(key, 0, 1, shape: {10})
model = EXGBoost.train(x, y)
EXGBoost.predict(model, x)copy

Training

EXGBoost is designed to feel familiar to users of the Python XGBoost library. EXGBoost.train/2 is the primary entry point for training a model. It accepts an Nx tensor for the features and an Nx tensor for the labels. EXGBoost.train/2 returns a trained EXGBoost.Booster struct that can be used for prediction. EXGBoost.train/2 also accepts a keyword list of options that can be used to configure the training process. See the XGBoost documentation for the full list of options.

EXGBoost.train/2 has the ability for the user to provide a custom training function that will be used to train the model. This is done by passing a function to the :obj option. See EXGBoost.Booster.update/4 for more information on this.

Another feature of EXGBoost.train/2 is the ability to provide a validation set for early stopping. This is done by passing a list of 3-tuples to the :evals option. Each 3-tuple should contain an Nx tensor for the features, an Nx tensor for the labels, and a string label for the validation set name. The validation set will be used to calculate the validation error at each iteration of the training process. If the validation error does not improve for :early_stopping_rounds iterations then the training process will stop. See the XGBoost documentation for a more detailed explanation of early stopping.

Early stopping is achieved through the use of callbacks. EXGBoost.train/2 accepts a list of callbacks that will be called at each iteration of the training process. The callbacks can be used to implement custom logic. For example, the user could implement a callback that will print the validation error at each iteration of the training process or to provide a custom setup function for training. SeeEXGBoost.Training.Callback for more information on callbacks.

Please notes that callbacks are called in the order that they are provided. If you provide multiple callbacks that modify the same parameter then the last callback will trump the previous callbacks. For example, if you provide a callback that sets the :early_stopping_rounds parameter to 10 and then provide a callback that sets the :early_stopping_rounds parameter to 20 then the :early_stopping_rounds parameter will be set to 20.

You are also able to pass parameters to be applied to the Booster model using the :params option. These parameters will be applied to the Booster model before training begins. This allows you to set parameters that are not available as options to EXGBoost.train/2. See the XGBoost documentation for a full list of parameters.

EXGBoost.train(
  x,
  y,
  obj: :multi_softprob,
  evals: [{x_test, y_test, "test"}],
  learning_rates: fn i -> i / 10 end,
  num_boost_round: 10,
  early_stopping_rounds: 3,
  max_depth: 3,
  eval_metric: [:rmse, :logloss]
)copy

Prediction

EXGBoost.predict/2 is the primary entry point for making predictions with a trained model. It accepts an EXGBoost.Booster struct (which is the output of EXGBoost.train/2). EXGBoost.predict/2 returns an Nx tensor containing the predictions and also accepts a keyword list of options that can be used to configure the prediction process.

preds = EXGBoost.train(X, y) |> EXGBoost.predict(X)copy

Serialization

A Booster can be serialized to a file using EXGBoost.write_* and loaded from a file using EXGBoost.read_*. The file format can be specified using the :format option which can be either :json or :ubj. The default is :json. If the file already exists, it will NOT be overwritten by default. Boosters can either be serialized to a file or to a binary string. Boosters can be serialized in three different ways: configuration only, configuration and model, or model only. dump functions will serialize the Booster to a binary string. Functions named with weights will serialize the model's trained parameters only. This is best used when the model is already trained and only inferences/predictions are going to be performed. Functions named with config will serialize the configuration only. Functions that specify model will serialize both the model parameters and the configuration.

Output Formats

• read/write - File.
• load/dump - Binary buffer.

Output Contents

• config - Save the configuration only.
• weights - Save the model parameters only. Use this when you want to save the model to a format that can be ingested by other XGBoost APIs.
• model - Save both the model parameters and the configuration.

Plotting

EXGBoost.plot_tree/2 is the primary entry point for plotting a tree from a trained model. It accepts an EXGBoost.Booster struct (which is the output of EXGBoost.train/2). EXGBoost.plot_tree/2 returns a VegaLite spec that can be rendered in a notebook or saved to a file. EXGBoost.plot_tree/2 also accepts a keyword list of options that can be used to configure the plotting process.

See EXGBoost.Plotting for more detail on plotting.

You can see available styles by running EXGBoost.Plotting.get_styles() or refer to the EXGBoost.Plotting.Styles documentation for a gallery of the styles.

Kino & Livebook Integration

EXGBoost integrates with Kino and Livebook to provide a rich interactive experience for data scientists.

EXGBoost implements the Kino.Render protocol for EXGBoost.Booster structs. This allows you to render a Booster in a Livebook notebook. Under the hood, EXGBoost uses Vega-Lite and Kino Vega-Lite to render the Booster.

See the Plotting in EXGBoost Notebook for an example of how to use EXGBoost with Kino and Livebook.

Examples

See the example Notebooks in the left sidebar (under the Pages tab) for more examples and tutorials on how to use EXGBoost.