Probability calibration with isotonic regression or logistic regression.
This class uses cross-validation to both estimate the parameters of a classifier and subsequently calibrate a classifier. With ensemble=True, for each cv split it fits a copy of the base estimator to the training subset, and calibrates it using the testing subset. For prediction, predicted probabilities are averaged across these individual calibrated classifiers. When ensemble=False, cross-validation is used to obtain unbiased predictions, via cross_val_predict, which are then used for calibration. For prediction, the base estimator, trained using all the data, is used. This is the prediction method implemented when probabilities=True for SVC and NuSVC estimators (see User Guide for details).
Already fitted classifiers can be calibrated by wrapping the model in a FrozenEstimator. In this case all provided data is used for calibration. The user has to take care manually that data for model fitting and calibration are disjoint.
The calibration is based on the decision_function method of the estimator if it exists, else on predict_proba.
Read more in the User Guide. In order to learn more on the CalibratedClassifierCV class, see the following calibration examples: Probability calibration of classifiers, Probability Calibration curves, and Probability Calibration for 3-class classification.
The classifier whose output need to be calibrated to provide more accurate predict_proba outputs. The default classifier is a LinearSVC.
Added in version 1.2.
The method to use for calibration. Can be ‘sigmoid’ which corresponds to Platt’s method (i.e. a logistic regression model) or ‘isotonic’ which is a non-parametric approach. It is not advised to use isotonic calibration with too few calibration samples (<<1000) since it tends to overfit.
Determines the cross-validation splitting strategy. Possible inputs for cv are:
None, to use the default 5-fold cross-validation,
integer, to specify the number of folds.
An iterable yielding (train, test) splits as arrays of indices.
For integer/None inputs, if y is binary or multiclass, StratifiedKFold is used. If y is neither binary nor multiclass, KFold is used.
Refer to the User Guide for the various cross-validation strategies that can be used here.
Changed in version 0.22: cv default value if None changed from 3-fold to 5-fold.
Changed in version 1.6: "prefit" is deprecated. Use FrozenEstimator instead.
Number of jobs to run in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors.
Base estimator clones are fitted in parallel across cross-validation iterations. Therefore parallelism happens only when cv != "prefit".
See Glossary for more details.
Added in version 0.24.
Determines how the calibrator is fitted.
“auto” will use False if the estimator is a FrozenEstimator, and True otherwise.
If True, the estimator is fitted using training data, and calibrated using testing data, for each cv fold. The final estimator is an ensemble of n_cv fitted classifier and calibrator pairs, where n_cv is the number of cross-validation folds. The output is the average predicted probabilities of all pairs.
If False, cv is used to compute unbiased predictions, via cross_val_predict, which are then used for calibration. At prediction time, the classifier used is the estimator trained on all the data. Note that this method is also internally implemented in sklearn.svm estimators with the probabilities=True parameter.
Added in version 0.24.
Changed in version 1.6: "auto" option is added and is the default.
The class labels.
Number of features seen during fit. Only defined if the underlying estimator exposes such an attribute when fit.
Added in version 0.24.
n_features_in_,)
Names of features seen during fit. Only defined if the underlying estimator exposes such an attribute when fit.
Added in version 1.0.
ensemble=False)
The list of classifier and calibrator pairs.
When ensemble=True, n_cv fitted estimator and calibrator pairs. n_cv is the number of cross-validation folds.
When ensemble=False, the estimator, fitted on all the data, and fitted calibrator.
Changed in version 0.24: Single calibrated classifier case when ensemble=False.
See also
calibration_curve
Compute true and predicted probabilities for a calibration curve.
References
[1]Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers, B. Zadrozny & C. Elkan, ICML 2001
[2]Transforming Classifier Scores into Accurate Multiclass Probability Estimates, B. Zadrozny & C. Elkan, (KDD 2002)
[3]Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods, J. Platt, (1999)
[4]Predicting Good Probabilities with Supervised Learning, A. Niculescu-Mizil & R. Caruana, ICML 2005
Examples
>>> from sklearn.datasets import make_classification
>>> from sklearn.naive_bayes import GaussianNB
>>> from sklearn.calibration import CalibratedClassifierCV
>>> X, y = make_classification(n_samples=100, n_features=2,
... n_redundant=0, random_state=42)
>>> base_clf = GaussianNB()
>>> calibrated_clf = CalibratedClassifierCV(base_clf, cv=3)
>>> calibrated_clf.fit(X, y)
CalibratedClassifierCV(...)
>>> len(calibrated_clf.calibrated_classifiers_)
3
>>> calibrated_clf.predict_proba(X)[:5, :]
array([[0.110, 0.889],
[0.072, 0.927],
[0.928, 0.072],
[0.928, 0.072],
[0.072, 0.928]])
>>> from sklearn.model_selection import train_test_split
>>> X, y = make_classification(n_samples=100, n_features=2,
... n_redundant=0, random_state=42)
>>> X_train, X_calib, y_train, y_calib = train_test_split(
... X, y, random_state=42
... )
>>> base_clf = GaussianNB()
>>> base_clf.fit(X_train, y_train)
GaussianNB()
>>> from sklearn.frozen import FrozenEstimator
>>> calibrated_clf = CalibratedClassifierCV(FrozenEstimator(base_clf))
>>> calibrated_clf.fit(X_calib, y_calib)
CalibratedClassifierCV(...)
>>> len(calibrated_clf.calibrated_classifiers_)
1
>>> calibrated_clf.predict_proba([[-0.5, 0.5]])
array([[0.936, 0.063]])
Fit the calibrated model.
Training data.
Target values.
Sample weights. If None, then samples are equally weighted.
Parameters to pass to the fit method of the underlying classifier.
Returns an instance of self.
Get metadata routing of this object.
Please check User Guide on how the routing mechanism works.
A MetadataRouter encapsulating routing information.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Predict the target of new samples.
The predicted class is the class that has the highest probability, and can thus be different from the prediction of the uncalibrated classifier.
The samples, as accepted by estimator.predict.
The predicted class.
Calibrated probabilities of classification.
This function returns calibrated probabilities of classification according to each class on an array of test vectors X.
The samples, as accepted by estimator.predict_proba.
The predicted probas.
Return accuracy on provided data and labels.
In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.
Test samples.
True labels for X.
Sample weights.
Mean accuracy of self.predict(X) w.r.t. y.
Request metadata passed to the fit method.
Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config). Please see User Guide on how the routing mechanism works.
The options for each parameter are:
True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to fit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.
The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.
Added in version 1.3.
Note
This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.
Metadata routing for sample_weight parameter in fit.
The updated object.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Request metadata passed to the score method.
Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config). Please see User Guide on how the routing mechanism works.
The options for each parameter are:
True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.
The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.
Added in version 1.3.
Note
This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.
Metadata routing for sample_weight parameter in score.
The updated object.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4