ogboost.GradientBoostingOrdinal

class ogboost.GradientBoostingOrdinal(base_learner=DecisionTreeRegressor(max_depth=3), n_estimators=100, learning_rate=0.1, learning_rate_thresh=0.001, validation_fraction=0.1, n_iter_no_change=None, tol=0.0001, validation_stratify=True, n_class=None, link_function='probit', subsample=1.0, verbose=0, random_state=None, cv_early_stopping_splits=None, validate_link_func=False)[source]

Bases: ClassifierMixin, BaseEstimator

Gradient Boosting for Ordinal Regression.

This estimator implements a gradient boosting framework tailored for ordinal regression tasks. It employs a coordinate-descent algorithm that alternates between updating a latent function g(x) using regression-based base learners, and refining the threshold vector theta to partition the latent space into ordered categories.

Key features include heterogeneous base learners (the base_learner parameter accepts a single estimator, a list, or a generator of estimators, allowing different models to be used in successive boosting iterations), CV-based early stopping (the cv_early_stopping_splits parameter enables K-fold cross-validation, in addition to a conventional holdout-set approach, for a more robust determination of the optimal number of boosting iterations), customizable link functions (‘probit’, ‘logit’, ‘loglog’, ‘cloglog’ and ‘cauchit’), and full scikit-learn compatibility via BaseEstimator and ClassifierMixin.

Parameters:

  • base_learner (estimator or list/generator of estimators, default=DecisionTreeRegressor(max_depth=3)) – The base learner(s) used to update the latent function. Different learners can be used across iterations.

  • n_estimators (int, default=100) – Maximum number of boosting iterations.

  • learning_rate (float, default=0.1) – Learning rate for the latent function updates.

  • learning_rate_thresh (float, default=0.001) – Learning rate for the threshold updates.

  • validation_fraction (float, default=0.1) – Fraction of data to use as a holdout set for early stopping (if CV is not used).

  • n_iter_no_change (int or None, default=None) – Number of iterations with no improvement to wait before stopping early.

  • tol (float, default=1e-4) – Tolerance for measuring improvement in early stopping.

  • validation_stratify (bool, default=True) – Whether to stratify the validation split by ordinal class for both holdout and cross-validation early stopping. When using CV early stopping, if any class has fewer samples than the number of splits, falls back to unstratified splits with a warning.

  • n_class (int or None, default=None) – Number of ordinal classes. If None, inferred from the training data.

  • link_function ({'probit', 'logit', 'loglog', 'cloglog', 'cauchit'}, default='probit') – Link function used to transform latent scores to probabilities.

  • subsample (float, default=1.0) – Fraction of samples used to fit each base learner.

  • verbose (int, default=0) – Verbosity level.

  • random_state (int, RandomState instance, or None, default=None) – Seed or random state for reproducibility.

  • cv_early_stopping_splits (int or None, default=None) – If an integer > 1, uses K-fold cross-validation for early stopping; otherwise, a holdout set is used.

Variables:

  • classes (array-like) – Array of unique ordinal classes.

  • n_estimators (int) – Actual number of boosting iterations performed.

Notes

Key methods are fit, predict, predict_proba, decision_function, score, plot_loss, and the staged variants staged_predict, staged_predict_proba, and staged_decision_function – see each method’s own docstring for details.

Examples

>>> from ogboost import GradientBoostingOrdinal
>>> model = GradientBoostingOrdinal(cv_early_stopping_splits=5, n_iter_no_change=10)
>>> model.fit(X_train, y_train)
>>> preds = model.predict(X_test)
__init__(base_learner=DecisionTreeRegressor(max_depth=3), n_estimators=100, learning_rate=0.1, learning_rate_thresh=0.001, validation_fraction=0.1, n_iter_no_change=None, tol=0.0001, validation_stratify=True, n_class=None, link_function='probit', subsample=1.0, verbose=0, random_state=None, cv_early_stopping_splits=None, validate_link_func=False)[source]

Methods

__init__([base_learner, n_estimators, ...])

decision_function(X)

fit(X, y[, sample_weight])

Fit the GradientBoostingOrdinal model to the given training data.

get_metadata_routing()

Get metadata routing of this object.

get_params([deep])

Get parameters for this estimator.

plot_loss([show, return_fig])

Plot the training and validation loss over boosting iterations, along with the loss improvement for regression function and threshold updates.

predict(X)

predict_proba(X)

score(X, y[, sample_weight, pred_type])

Return accuracy on provided data and labels.

set_fit_request(*[, sample_weight])

Configure whether metadata should be requested to be passed to the fit method.

set_params(**params)

Set the parameters of this estimator.

set_score_request(*[, pred_type, sample_weight])

Configure whether metadata should be requested to be passed to the score method.

staged_decision_function(X)

staged_predict(X)

staged_predict_proba(X)

fit(X, y, sample_weight=None)[source]

Fit the GradientBoostingOrdinal model to the given training data.

This method implements a gradient boosting procedure specialized for ordinal regression. It builds (and stores) an internal list of base learners that iteratively update a latent function g(x) and refine the ordinal threshold vector θ.

Depending on the parameter settings:

  • If cv_early_stopping_splits > 1, the model uses K-fold cross-validation for early stopping. Training stops when no improvement is observed over n_iter_no_change iterations in the validation fold.

  • Otherwise, if n_iter_no_change is not None, a holdout set (fraction of the training data specified by validation_fraction) is used for early stopping.

  • If neither mechanism is enabled, the model runs for all n_estimators boosting iterations.

Parameters:

  • X (array-like of shape (n_samples, n_features)) – Training data features.

  • y (array-like of shape (n_samples,)) – Ordinal target values. Must be integers in a suitable range (0 to n_class-1). If n_class is None, the maximum label in y determines n_class = max(y) + 1.

  • sample_weight (array-like of shape (n_samples,), optional) – Sample weights to apply in the loss function. Must have non-negative values and a positive sum. If None (default), equal weights are assumed.

Returns:

self (GradientBoostingOrdinal) – Fitted estimator.

Raises:

ValueError – Raised when y has invalid ordinal values (e.g., negative or beyond n_class-1), sample_weight is negative, zero-sum, or mismatched in length, link_function is not one of {‘probit’, ‘logit’, ‘cloglog’}, subsample is not in (0, 1], or a mismatch occurs between templates/overrides in heterogeneous learner scenarios.

Notes

  • After calling fit, the model stores the trained base learners and the final thresholds in self._path and self._final.

  • Early stopping details (e.g., holdout or CV) are determined by the constructor parameters n_iter_no_change, validation_fraction, and cv_early_stopping_splits.

  • When using K-fold CV, the internal _fit_cv method is invoked; otherwise, the holdout-based approach is used if n_iter_no_change is specified.

decision_function(X)[source]
staged_decision_function(X)[source]
predict_proba(X)[source]
staged_predict_proba(X)[source]
predict(X)[source]
staged_predict(X)[source]
score(X, y, sample_weight=None, pred_type='latent')[source]

Return accuracy on provided data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters:

  • X (array-like of shape (n_samples, n_features)) – Test samples.

  • y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True labels for X.

  • sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

Returns:

score (float) – Mean accuracy of self.predict(X) w.r.t. y.

plot_loss(show=True, return_fig=False, **kwargs)[source]

Plot the training and validation loss over boosting iterations, along with the loss improvement for regression function and threshold updates.

This method generates two side-by-side plots: - The left plot shows the training loss and, if available, the validation loss. - The right plot shows the loss improvement at each iteration, separately for updates to the regression function and the threshold vector.

Parameters:

  • show (bool, default=True) – Whether to immediately display the plot. If False, the plot is not shown but can be returned.

  • return_fig (bool, default=False) – Whether to return the Matplotlib figure object. This can be useful for further customization or saving to a file.

  • **kwargs (dict, optional) –

    Customization options for plot aesthetics. The following keyword arguments are supported:

    • figsize: tuple, default=(12, 5)

      Figure size in inches (width, height).

    • training_style: dict, optional

      Style options for the training loss curve (e.g., {“color”: “blue”, “linestyle”: “–“}).

    • validation_style: dict, optional

      Style options for the validation loss curve (e.g., {“color”: “red”, “linestyle”: “-.”}).

    • improvement_style: dict, optional

      Style options for the loss improvement curves for regression function and threshold updates (e.g., {“color”: “green”, “linestyle”: “:”}).

    • title_loss: str, default=”Cross-Entropy Loss”

      Title for the training and validation loss plot.

    • xlabel_loss: str, default=”Boosting Iteration”

      Label for the x-axis of the training and validation loss plot.

    • ylabel_loss: str, default=”CE Loss”

      Label for the y-axis of the training and validation loss plot.

    • title_improvement: str, default=”Cross-Entropy Loss Improvement”

      Title for the loss improvement plot.

    • xlabel_improvement: str, default=”Boosting Iteration”

      Label for the x-axis of the loss improvement plot.

    • ylabel_improvement: str, default=”CE Loss Improvement”

      Label for the y-axis of the loss improvement plot.

    • grid_loss: bool, default=True

      Whether to show a grid on the training/validation loss plot.

    • grid_improvement: bool, default=True

      Whether to show a grid on the loss improvement plot.

Returns:

fig (matplotlib.figure.Figure, optional) – The figure object if return_fig=True, otherwise None.

Notes

  • If validation loss is unavailable (None), only the training loss is plotted.

  • If loss improvement data is unavailable, the second plot will indicate that data is missing.

  • The method supports all Matplotlib plot customizations via kwargs.

Examples

>>> model = GradientBoostingOrdinal(n_estimators=100, n_iter_no_change=10)
>>> model.fit(X_train, y_train)
>>> model.plot_loss()

Customized plot:

>>> model.plot_loss(
...     training_style={"color": "blue", "linestyle": "--"},
...     validation_style={"color": "red", "linestyle": "-."},
...     improvement_style={"color": "green", "linestyle": ":"},
...     title_loss="Training and Validation Loss",
...     grid_loss=False,
... )
set_fit_request(*, sample_weight='$UNCHANGED$')

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

  • sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.

  • self (GradientBoostingOrdinal)

Returns:

self (object) – The updated object.

Return type:

GradientBoostingOrdinal

set_score_request(*, pred_type='$UNCHANGED$', sample_weight='$UNCHANGED$')

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

  • pred_type (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for pred_type parameter in score.

  • sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.

  • self (GradientBoostingOrdinal)

Returns:

self (object) – The updated object.

Return type:

GradientBoostingOrdinal