cc_tk.feature package

Submodules

cc_tk.feature.correlation module

Scikit-learn like estimators to deal with correlation in variables.

class cc_tk.feature.correlation.ClusteringCorrelation(threshold: float = 0.1, summary_method: Literal['first', 'pca'] = 'first', n_variables_by_cluster: int = 1)[source]

Bases: TransformerMixin, BaseEstimator

Feature selector based on Clustering of correlations.

fit(features: DataFrame, y: Series = None)[source]

Fit the feature selection to features.

Parameters:

  • features (pd.DataFrame) – Features to fit the feature selection to

  • y (pd.Series, optional) – Target, by default None

get_clusters(linkage: ndarray) → List[List[str]][source]

Retrieve the cluster of variables given a specific threshold.

Parameters:

linkage (np.ndarray) – Linkage matrix from scipy.cluster.hierarchy

Returns:

List of lists of variable names according to each cluster

Return type:

List[List[str]]

plot_correlation_matrix(fig=None, ax: Axes = None) → Axes[source]

Plot correlation matrix of the features.

Parameters:

  • fig (plt.Figure, optional) – Figure to plot the correlation matrix on, by default None

  • ax (plt.Axes, optional) – Axis to plot the correlation matrix on, by default None

Returns:

  • plt.Axes

  • Axis with the correlation matrix

plot_dendro(ax: Axes = None) → Dict[str, Any][source]

Plot dendrogram of the correlation matrix.

Parameters:

ax (plt.Axes, optional) – Axis to plot the dendrogram on, by default None

Returns:

Dendrogram object

Return type:

Dict[str, Any]

set_fit_request(*, features: bool | None | str = '$UNCHANGED$') → ClusteringCorrelation

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

featuresstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for features parameter in fit.

selfobject

The updated object.

set_transform_request(*, features: bool | None | str = '$UNCHANGED$') → ClusteringCorrelation

Configure whether metadata should be requested to be passed to the transform method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to transform if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to transform.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

featuresstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for features parameter in transform.

selfobject

The updated object.

transform(features: DataFrame, y: Series = None) → DataFrame[source]

Transform the features with the feature selection.

Parameters:

  • features (pd.DataFrame) – Features

  • y (pd.Series, optional) – Target, by default None

Returns:

Transformed features with feature selection

Return type:

pd.DataFrame

class cc_tk.feature.correlation.CorrelationToTarget(threshold: float = 0.1)[source]

Bases: TransformerMixin, BaseEstimator

Select columns with correlation to target above a threshold.

Parameters:

threshold (float, optional) – The threshold for the correlation to the target. Default is 0.1.

fit(features: ndarray | DataFrame, y: ndarray | Series) → CorrelationToTarget[source]

Fit the transformer to the data.

Parameters:

  • features (ArrayLike2D) – The features.

  • y (ArrayLike1D) – The target.

plot_correlation()[source]

Plot the correlation of each feature to the target.

The selected features are highlighted in green, the others in red. The threshold values are indicated with dashed lines.

set_fit_request(*, features: bool | None | str = '$UNCHANGED$') → CorrelationToTarget

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

featuresstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for features parameter in fit.

selfobject

The updated object.

set_transform_request(*, features: bool | None | str = '$UNCHANGED$') → CorrelationToTarget

Configure whether metadata should be requested to be passed to the transform method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to transform if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to transform.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

featuresstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for features parameter in transform.

selfobject

The updated object.

transform(features: ndarray | DataFrame, y: ndarray | Series = None) → ndarray | DataFrame[source]

Retrieve only the selected columns.

Parameters:

  • features (ArrayLike2D) – The features.

  • y (ArrayLike1D, optional) – The target, by default None

Returns:

The selected features.

Return type:

ArrayLike2D

Raises:

ValueError – If the number of columns in features is different from the number of columns in the training data.

class cc_tk.feature.correlation.PairwiseCorrelationDrop(threshold: float = 0.9)[source]

Bases: TransformerMixin, BaseEstimator

Scikit-learn like estimator to deal with pair-wise correlation.

static compute_drop_indices_from_detailed_steps(res: DataFrame) → ndarray[source]

Compute the indices of variables to drop from the detailed steps.

Parameters:

res (pd.DataFrame) – Detailed steps of the pairwise correlation drop

Returns:

Indices of variables to drop

Return type:

np.ndarray

classmethod compute_mask_selection(features: ndarray, cut: float = 0.9) → ndarray[source]

Compute the mask of variables to keep.

Parameters:

  • features (np.ndarray) – Features

  • cut (float, optional) – Correlation threshold, by default 0.9

Returns:

Mask of variables to keep

Return type:

np.ndarray

fit(features: ndarray | DataFrame, y: ndarray | Series = None) → PairwiseCorrelationDrop[source]

Fit the transformer to the data.

Parameters:

  • features (ArrayLike2D) – Features

  • y (ArrayLike1D, optional) – Target, by default None

Returns:

Fitted transformer

Return type:

PairwiseCorrelationDrop

set_fit_request(*, features: bool | None | str = '$UNCHANGED$') → PairwiseCorrelationDrop

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

featuresstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for features parameter in fit.

selfobject

The updated object.

set_transform_request(*, features: bool | None | str = '$UNCHANGED$') → PairwiseCorrelationDrop

Configure whether metadata should be requested to be passed to the transform method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to transform if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to transform.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

featuresstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for features parameter in transform.

selfobject

The updated object.

transform(features: ndarray | DataFrame, y: ndarray | Series = None) → ndarray | DataFrame[source]

Retrieve only the selected columns.

Parameters:

  • features (ArrayLike2D) – Features

  • y (ArrayLike1D, optional) – Target, by default None

Returns:

Selected features

Return type:

ArrayLike2D

Raises:

ValueError – If the number of columns in features is different from the number of columns in the training data.

Module contents

Module for dealing with features.