XGBoost Learners

Heuristics provides learners for training XGBoost models, which we describe on this page along with a guide to their parameters.

Shared Parameters

All of learners provided by Heuristics for training XGBoost models are XGBoostLearners. In addition to the shared learner parameters, these learners support all parameters of XGBoost under the same names, with some additional remarks:

  • the general learner parameter criterion is used to set the objective parameter in XGBoost
  • the general learner parameter random_seed is used to set the seed parameter in XGBoost, so setting random_seed should be preferred as this will also control the randomness outside XGBoost (e.g. when splitting the data during grid search)
  • the general learner parameter num_threads is used to set the nthread parameter in XGBoost (if not specified, the XGBoost default behavior is used)
  • the XGBoost parameters monotone_constraints and interaction_constraints should be set as specified below

These learners support the following additional parameters to control their behavior.

num_round

num_round accepts a non-negative Integer to control the number of boosting rounds that will be conducted. The default value is 100.

monotone_constraints

monotone_constraints allows you to enforce monotonicity constraints on the XGBoost model. These constraints are passed as a FeatureMapping where the values should be one of 1/:increasing/"increasing" for an increasing constraint, or one of -1/:decreasing/"decreasing" for a decreasing constraint. Below are some examples:

  • Dict(1 => :increasing) applies an increasing constraint on the first feature
  • (A=1, D=-1) applies an increasing constraint to feature A and a decreasing constraint to feature D
  • (:A => "decreasing", Not(:A) => "increasing") applies a decreasing constraint to feature A and increasing constraints on all other features

interaction_constraints

interaction_constraints allows you to enforce feature interaction constraints on the XGBoost model. These constraints are passed as a Vector of FeatureSets, each specifying a group of features that are allowed to interact with each other. Below are some examples:

  • [[1, 2], ["E", "F"]] allows the first and second features to interact with each other, and similarly allows interaction between features E and F
  • [[:A, :B], Not([:A, :B])] permits features A and B to interact with each other but prevents them from interacting with the remaining features

Classification Learners

The XGBoostClassifier is used for training XGBoost models for classification problems. The following values for criterion are permitted:

There are no additional parameters beyond the shared parameters.

Regression Learners

The XGBoostRegressor is used for training XGBoost models for regression problems. The following values for criterion are permitted:

In addition to the shared parameters, these learners also support the shared regression parameters.

Survival Learners

The XGBoostSurvivalLearner is used for training XGBoost models for survival problems. The following values for criterion are permitted:

In addition to the shared parameters, these learners also support the shared survival parameters.