tabullm.GMMFeatureExtractor

class tabullm.GMMFeatureExtractor(n_components=10, covariance_type='full', n_init=3, random_state=None, include_log_density=False, **gmm_kwargs)

Bases: BaseEstimator, TransformerMixin

Extract features using Gaussian Mixture Model log-joint probabilities.

transform() returns an (n_samples, K) array of per-cluster log-joints

ℓ_k(x) = log p(x, c_k) = log π_k + log p(x | c_k)

where log p(x | c_k) is the Gaussian log-likelihood under component k. This is the quantity the GMM maximises for hard assignment (k* = argmax_k ℓ_k), so features are in exact correspondence with the model’s own criterion. Posterior probabilities are a softmax of these features.

Adds transform() method that sklearn’s GaussianMixture lacks, enabling use in feature extraction pipelines.

Parameters:

• n_components (int, default=10) – Number of mixture components.

• covariance_type ({'full', 'tied', 'diag', 'spherical'}, default='full') – Type of covariance parameters.

• n_init (int, default=3) – Number of initializations to perform.

• random_state (int, default=None) – Random seed for reproducibility.

• include_log_density (bool, default=False) – If True, append log p(x) = log Σ_k exp(ℓ_k(x)) as a (K+1)-th column. This is the log marginal likelihood — how well the overall mixture explains x. It is a deterministic function of the K log-joints (log-sum-exp), so it adds no information for expressive nonlinear models but provides an explicit outlier score for linear ones.

• **gmm_kwargs – Additional parameters for GaussianMixture.

Variables:

• gmm_ (GaussianMixture) – Fitted Gaussian Mixture Model.

• means_ (array, shape (n_components, n_features)) – Component means.

• covariances_ (array) – Component covariances.

• labels_ (array, shape (n_samples,)) – Cluster labels for each sample from the training set.

Examples

>>> from tabullm import TextColumnTransformer, GMMFeatureExtractor
>>> from sklearn.pipeline import Pipeline
>>> from sklearn.ensemble import RandomForestClassifier
>>> from langchain_huggingface import HuggingFaceEmbeddings
>>>
>>> embedding_model = HuggingFaceEmbeddings(
...     model_name='sentence-transformers/all-MiniLM-L6-v2'
... )
>>> pipeline = Pipeline([
...     ('embed', TextColumnTransformer(model=embedding_model)),
...     ('gmm', GMMFeatureExtractor(n_components=10)),
...     ('clf', RandomForestClassifier())
... ])

Methods

__init__
assignment_confidence_stats	Compute per-observation assignment confidence statistics.
fit	Fit GMM to data.
fit_transform	Fit to data, then transform it.
get_feature_names_out	Feature names: lj_0 … lj_{K-1}, plus log_density when include_log_density=True.
get_metadata_routing	Get metadata routing of this object.
get_params	Get parameters for this estimator.
predict	Predict cluster labels.
set_output	Set output container.
set_params	Set the parameters of this estimator.
transform	Transform X to per-cluster log-joint features.

Details

fit(X, y=None)

Fit GMM to data.

transform(X)

Transform X to per-cluster log-joint features.

Parameters:

X (array-like of shape (n_samples, n_features))

Returns:

ndarray of shape (n_samples, K) or (n_samples, K+1) – Per-cluster log p(x, c_k) for k = 0 … K-1, plus log p(x) as the final column when include_log_density=True.

get_feature_names_out(input_features=None)

Feature names: lj_0 … lj_{K-1}, plus log_density when include_log_density=True.

predict(X)

Predict cluster labels.

assignment_confidence_stats(X)

Compute per-observation assignment confidence statistics.

Returns four metrics that characterise how confidently and unambiguously the fitted GMM assigns each observation to a cluster, plus the marginal log-density as an outlier score. All four are scalars per observation and can be aggregated to cluster-level summaries via groupby(cluster_labels).mean().

Parameters:

X (array-like of shape (n_samples, n_features)) – Data to score. Must have the same number of features as the training data.

Returns:

pd.DataFrame of shape (n_samples, 4) – Columns:

max_posterior

Maximum posterior probability \(\max_k p(k \mid x)\). Range \((1/K, 1]\). Higher means the model is more certain about the cluster assignment.

entropy

Shannon entropy \(-\sum_k p(k \mid x) \log p(k \mid x)\) of the posterior distribution. Range \([0, \log K]\). Lower means the probability mass is concentrated on fewer clusters.

log_joint_margin

Difference between the top-1 and top-2 per-cluster log joints \(\ell_{k^*}(x) - \max_{j \neq k^*} \ell_j(x)\). Range \([0, \infty)\). Larger means the assigned cluster is more decisively preferred over its nearest rival.

log_density

Log marginal likelihood \(\log p(x) = \log \sum_k e^{\ell_k(x)}\). Range \((-\infty, 0]\). Captures how well the overall GMM explains this observation; large negative values flag outliers. Orthogonal to the three assignment metrics above: two observations can share the same max_posterior yet have very different log_density.

Examples

>>> stats = gmm.assignment_confidence_stats(X_embeddings)
>>> stats.describe()

>>> # Per-cluster rollup
>>> stats.groupby(cluster_labels).mean()

>>> # Feed into ClusterExplainer
>>> explainer.explain(X, cluster_labels, y=y, observation_stats=stats)