simpleml.metrics.classification

Module for classification metrics https://en.wikipedia.org/wiki/Confusion_matrix

Includes base class and derived metrics following the nomenclature:

ConstraintValueMetric

Where:

Constraint is the lookup criteria (ex FPR in ROC curve) Value is desired value (ex TPR in ROC curve)

This module is organized by metric and prediction dependencies:

1. Base classes with methods and utilities

2. Aggregate Metrics (single value output)

   2a) Single values computed via Predict method (operating points) 2b) Single values computed via proba method (agg over curve)

3. Curve Metrics (constraint: value)

   3a) Threshold: confusion matrix metrics 3b) confusion matrix metrics: threshold or other metrics

Module Contents

Classes

AccuracyMetric	TODO: Figure out multiclass generalizations
AggregateBinaryClassificationMetric	TODO: Figure out multiclass generalizations
BinaryClassificationMetric	TODO: Figure out multiclass generalizations
ClassificationMetric	TODO: Figure out multiclass generalizations
F1ScoreMetric	TODO: Figure out multiclass generalizations
FdrAccuracyMetric	TODO: Figure out multiclass generalizations
FdrF1ScoreMetric	TODO: Figure out multiclass generalizations
FdrFnrMetric	TODO: Figure out multiclass generalizations
FdrForMetric	TODO: Figure out multiclass generalizations
FdrFprMetric	TODO: Figure out multiclass generalizations
FdrInformednessMetric	TODO: Figure out multiclass generalizations
FdrMarkednessMetric	TODO: Figure out multiclass generalizations
FdrMccMetric	TODO: Figure out multiclass generalizations
FdrNpvMetric	TODO: Figure out multiclass generalizations
FdrPpvMetric	TODO: Figure out multiclass generalizations
FdrPredictedNegativeRateMetric	TODO: Figure out multiclass generalizations
FdrPredictedPositiveRateMetric	TODO: Figure out multiclass generalizations
FdrThresholdMetric	TODO: Figure out multiclass generalizations
FdrTnrMetric	TODO: Figure out multiclass generalizations
FdrTprMetric	TODO: Figure out multiclass generalizations
FnrAccuracyMetric	TODO: Figure out multiclass generalizations
FnrF1ScoreMetric	TODO: Figure out multiclass generalizations
FnrFdrMetric	TODO: Figure out multiclass generalizations
FnrForMetric	TODO: Figure out multiclass generalizations
FnrFprMetric	TODO: Figure out multiclass generalizations
FnrInformednessMetric	TODO: Figure out multiclass generalizations
FnrMarkednessMetric	TODO: Figure out multiclass generalizations
FnrMccMetric	TODO: Figure out multiclass generalizations
FnrNpvMetric	TODO: Figure out multiclass generalizations
FnrPpvMetric	TODO: Figure out multiclass generalizations
FnrPredictedNegativeRateMetric	TODO: Figure out multiclass generalizations
FnrPredictedPositiveRateMetric	TODO: Figure out multiclass generalizations
FnrThresholdMetric	TODO: Figure out multiclass generalizations
FnrTnrMetric	TODO: Figure out multiclass generalizations
FnrTprMetric	TODO: Figure out multiclass generalizations
ForAccuracyMetric	TODO: Figure out multiclass generalizations
ForF1ScoreMetric	TODO: Figure out multiclass generalizations
ForFdrMetric	TODO: Figure out multiclass generalizations
ForFnrMetric	TODO: Figure out multiclass generalizations
ForFprMetric	TODO: Figure out multiclass generalizations
ForInformednessMetric	TODO: Figure out multiclass generalizations
ForMarkednessMetric	TODO: Figure out multiclass generalizations
ForMccMetric	TODO: Figure out multiclass generalizations
ForNpvMetric	TODO: Figure out multiclass generalizations
ForPpvMetric	TODO: Figure out multiclass generalizations
ForPredictedNegativeRateMetric	TODO: Figure out multiclass generalizations
ForPredictedPositiveRateMetric	TODO: Figure out multiclass generalizations
ForThresholdMetric	TODO: Figure out multiclass generalizations
ForTnrMetric	TODO: Figure out multiclass generalizations
ForTprMetric	TODO: Figure out multiclass generalizations
FprAccuracyMetric	TODO: Figure out multiclass generalizations
FprF1ScoreMetric	TODO: Figure out multiclass generalizations
FprFdrMetric	TODO: Figure out multiclass generalizations
FprFnrMetric	TODO: Figure out multiclass generalizations
FprForMetric	TODO: Figure out multiclass generalizations
FprInformednessMetric	TODO: Figure out multiclass generalizations
FprMarkednessMetric	TODO: Figure out multiclass generalizations
FprMccMetric	TODO: Figure out multiclass generalizations
FprMetric	TODO: Figure out multiclass generalizations
FprNpvMetric	TODO: Figure out multiclass generalizations
FprPpvMetric	TODO: Figure out multiclass generalizations
FprPredictedNegativeRateMetric	TODO: Figure out multiclass generalizations
FprPredictedPositiveRateMetric	TODO: Figure out multiclass generalizations
FprThresholdMetric	TODO: Figure out multiclass generalizations
FprTnrMetric	TODO: Figure out multiclass generalizations
FprTprMetric	TODO: Figure out multiclass generalizations
NpvAccuracyMetric	TODO: Figure out multiclass generalizations
NpvF1ScoreMetric	TODO: Figure out multiclass generalizations
NpvFdrMetric	TODO: Figure out multiclass generalizations
NpvFnrMetric	TODO: Figure out multiclass generalizations
NpvForMetric	TODO: Figure out multiclass generalizations
NpvFprMetric	TODO: Figure out multiclass generalizations
NpvInformednessMetric	TODO: Figure out multiclass generalizations
NpvMarkednessMetric	TODO: Figure out multiclass generalizations
NpvMccMetric	TODO: Figure out multiclass generalizations
NpvPpvMetric	TODO: Figure out multiclass generalizations
NpvPredictedNegativeRateMetric	TODO: Figure out multiclass generalizations
NpvPredictedPositiveRateMetric	TODO: Figure out multiclass generalizations
NpvThresholdMetric	TODO: Figure out multiclass generalizations
NpvTnrMetric	TODO: Figure out multiclass generalizations
NpvTprMetric	TODO: Figure out multiclass generalizations
PpvAccuracyMetric	TODO: Figure out multiclass generalizations
PpvF1ScoreMetric	TODO: Figure out multiclass generalizations
PpvFdrMetric	TODO: Figure out multiclass generalizations
PpvFnrMetric	TODO: Figure out multiclass generalizations
PpvForMetric	TODO: Figure out multiclass generalizations
PpvFprMetric	TODO: Figure out multiclass generalizations
PpvInformednessMetric	TODO: Figure out multiclass generalizations
PpvMarkednessMetric	TODO: Figure out multiclass generalizations
PpvMccMetric	TODO: Figure out multiclass generalizations
PpvNpvMetric	TODO: Figure out multiclass generalizations
PpvPredictedNegativeRateMetric	TODO: Figure out multiclass generalizations
PpvPredictedPositiveRateMetric	TODO: Figure out multiclass generalizations
PpvThresholdMetric	TODO: Figure out multiclass generalizations
PpvTnrMetric	TODO: Figure out multiclass generalizations
PpvTprMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateAccuracyMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateF1ScoreMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateFdrMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateFnrMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateForMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateFprMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateInformednessMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateMarkednessMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateMccMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateNpvMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRatePpvMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRatePredictedPositiveRateMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateThresholdMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateTnrMetric	TODO: Figure out multiclass generalizations
PredictedNegativeRateTprMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateAccuracyMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateF1ScoreMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateFdrMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateFnrMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateForMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateFprMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateInformednessMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateMarkednessMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateMccMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateNpvMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRatePpvMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRatePredictedNegativeRateMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateThresholdMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateTnrMetric	TODO: Figure out multiclass generalizations
PredictedPositiveRateTprMetric	TODO: Figure out multiclass generalizations
RocAucMetric	TODO: Figure out multiclass generalizations
ThresholdAccuracyMetric	TODO: Figure out multiclass generalizations
ThresholdF1ScoreMetric	TODO: Figure out multiclass generalizations
ThresholdFdrMetric	TODO: Figure out multiclass generalizations
ThresholdFnrMetric	TODO: Figure out multiclass generalizations
ThresholdForMetric	TODO: Figure out multiclass generalizations
ThresholdFprMetric	TODO: Figure out multiclass generalizations
ThresholdInformednessMetric	TODO: Figure out multiclass generalizations
ThresholdMarkednessMetric	TODO: Figure out multiclass generalizations
ThresholdMccMetric	TODO: Figure out multiclass generalizations
ThresholdNpvMetric	TODO: Figure out multiclass generalizations
ThresholdPpvMetric	TODO: Figure out multiclass generalizations
ThresholdPredictedNegativeRateMetric	TODO: Figure out multiclass generalizations
ThresholdPredictedPositiveRateMetric	TODO: Figure out multiclass generalizations
ThresholdTnrMetric	TODO: Figure out multiclass generalizations
ThresholdTprMetric	TODO: Figure out multiclass generalizations
TnrAccuracyMetric	TODO: Figure out multiclass generalizations
TnrF1ScoreMetric	TODO: Figure out multiclass generalizations
TnrFdrMetric	TODO: Figure out multiclass generalizations
TnrFnrMetric	TODO: Figure out multiclass generalizations
TnrForMetric	TODO: Figure out multiclass generalizations
TnrFprMetric	TODO: Figure out multiclass generalizations
TnrInformednessMetric	TODO: Figure out multiclass generalizations
TnrMarkednessMetric	TODO: Figure out multiclass generalizations
TnrMccMetric	TODO: Figure out multiclass generalizations
TnrNpvMetric	TODO: Figure out multiclass generalizations
TnrPpvMetric	TODO: Figure out multiclass generalizations
TnrPredictedNegativeRateMetric	TODO: Figure out multiclass generalizations
TnrPredictedPositiveRateMetric	TODO: Figure out multiclass generalizations
TnrThresholdMetric	TODO: Figure out multiclass generalizations
TnrTprMetric	TODO: Figure out multiclass generalizations
TprAccuracyMetric	TODO: Figure out multiclass generalizations
TprF1ScoreMetric	TODO: Figure out multiclass generalizations
TprFdrMetric	TODO: Figure out multiclass generalizations
TprFnrMetric	TODO: Figure out multiclass generalizations
TprForMetric	TODO: Figure out multiclass generalizations
TprFprMetric	TODO: Figure out multiclass generalizations
TprInformednessMetric	TODO: Figure out multiclass generalizations
TprMarkednessMetric	TODO: Figure out multiclass generalizations
TprMccMetric	TODO: Figure out multiclass generalizations
TprMetric	TODO: Figure out multiclass generalizations
TprNpvMetric	TODO: Figure out multiclass generalizations
TprPpvMetric	TODO: Figure out multiclass generalizations
TprPredictedNegativeRateMetric	TODO: Figure out multiclass generalizations
TprPredictedPositiveRateMetric	TODO: Figure out multiclass generalizations
TprThresholdMetric	TODO: Figure out multiclass generalizations
TprTnrMetric	TODO: Figure out multiclass generalizations

Attributes

LOGGER
__author__

simpleml.metrics.classification.LOGGER

simpleml.metrics.classification.__author__ = Elisha Yadgaran

class simpleml.metrics.classification.AccuracyMetric(**kwargs)

Bases: AggregateBinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

static _score(predictions, labels)

Each aggregate needs to define a separate private method to actually calculate the aggregate

Separated from the public score method to enable easier testing and extension (values can be passed from non internal properties)

class simpleml.metrics.classification.AggregateBinaryClassificationMetric(dataset_split=None, **kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split (Optional[str]) – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

abstract static _score(predictions, labels)

Each aggregate needs to define a separate private method to actually calculate the aggregate

Separated from the public score method to enable easier testing and extension (values can be passed from non internal properties)

score(self)

Main scoring method. Uses internal values and passes to class level aggregation method

class simpleml.metrics.classification.BinaryClassificationMetric(dataset_split=None, **kwargs)

Bases: ClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split (Optional[str]) – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

static _create_confusion_matrix(thresholds, probabilities, labels)

Independent computation method (easier testing)

property accuracy(self)

Convenience property for the Accuracy Rate (TP+TN/TP+FP+TN+FN)

property confusion_matrix(self)

Property method to return (or generate) dataframe of confusion matrix at each threshold

create_confusion_matrix(self)

Iterate through each threshold and compute confusion matrix

static dedupe_curve(keys, values, maximize=True, round_places=3)

Method to deduplicate multiple values for the same key on a curve (ex multiple thresholds with the same fpr and different tpr for roc)

Parameters

maximize – Boolean, whether to choose the maximum value for each unique key or the minimum

property f1(self)

Convenience property for the F1 Score (2*TP/2*TP+FP+FN)

property false_discovery_rate(self)

Convenience property for the False Discovery Rate (FP/FP+TP)

property false_negative_rate(self)

Convenience property for the False Negative Rate (FN/TP+FN)

property false_omission_rate(self)

Convenience property for the False Omission Rate (FN/TN+FN)

property false_positive_rate(self)

Convenience property for the False Positive Rate (FP/FP+TN)

property informedness(self)

Convenience property for the Informedness (TPR+TNR-1)

property labels(self)

property markedness(self)

Convenience property for the Markedness (PPV+NPV-1)

property matthews_correlation_coefficient(self)

Convenience property for the Matthews Correlation Coefficient (TP*TN-FP*FN/((FP+TP)*(TP+FN)*(TN+FP)*(TN+FN))^0.5)

property negative_predictive_value(self)

Convenience property for the Negative Predictive Value (TN/TN+FN)

property positive_predictive_value(self)

Convenience property for the Positive Predictive Value (TP/FP+TP)

property predicted_negative_rate(self)

Convenience property for the Predicted Negative Rate (TN+FN/TP+FP+TN+FN)

property predicted_positive_rate(self)

Convenience property for the Predicted Positive Rate (TP+FP/TP+FP+TN+FN)

property predictions(self)

property probabilities(self)

property thresholds(self)

Convenience property for the probability thresholds

property true_negative_rate(self)

Convenience property for the True Negative Rate (TN/FP+TN)

property true_positive_rate(self)

Convenience property for the True Positive Rate (TP/TP+FN)

static validate_labels(labels)

class simpleml.metrics.classification.ClassificationMetric(dataset_split=None, **kwargs)

Bases: simpleml.metrics.base_metric.Metric

TODO: Figure out multiclass generalizations

Parameters

dataset_split (Optional[str]) – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

_get_split(self, column)

Parameters

column (str) –

Return type

Any

property labels(self)

Return type

Any

property predictions(self)

Return type

Any

property probabilities(self)

Return type

Any

static validate_predictions(predictions)

Parameters

predictions (Any) –

Return type

None

class simpleml.metrics.classification.F1ScoreMetric(**kwargs)

Bases: AggregateBinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

static _score(predictions, labels)

Each aggregate needs to define a separate private method to actually calculate the aggregate

Separated from the public score method to enable easier testing and extension (values can be passed from non internal properties)

class simpleml.metrics.classification.FdrAccuracyMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrF1ScoreMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrFnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrForMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrFprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrInformednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrMarkednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrMccMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrNpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrPpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrPredictedNegativeRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrPredictedPositiveRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrThresholdMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrTnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FdrTprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrAccuracyMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrF1ScoreMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrFdrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrForMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrFprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrInformednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrMarkednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrMccMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrNpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrPpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrPredictedNegativeRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrPredictedPositiveRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrThresholdMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrTnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FnrTprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForAccuracyMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForF1ScoreMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForFdrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForFnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForFprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForInformednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForMarkednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForMccMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForNpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForPpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForPredictedNegativeRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForPredictedPositiveRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForThresholdMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForTnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ForTprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprAccuracyMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprF1ScoreMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprFdrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprFnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprForMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprInformednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprMarkednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprMccMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprMetric(**kwargs)

Bases: AggregateBinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

static _score(predictions, labels)

Each aggregate needs to define a separate private method to actually calculate the aggregate

Separated from the public score method to enable easier testing and extension (values can be passed from non internal properties)

class simpleml.metrics.classification.FprNpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprPpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprPredictedNegativeRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprPredictedPositiveRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprThresholdMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprTnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.FprTprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvAccuracyMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvF1ScoreMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvFdrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvFnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvForMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvFprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvInformednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvMarkednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvMccMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvPpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvPredictedNegativeRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvPredictedPositiveRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvThresholdMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvTnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.NpvTprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvAccuracyMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvF1ScoreMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvFdrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvFnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvForMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvFprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvInformednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvMarkednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvMccMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvNpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvPredictedNegativeRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvPredictedPositiveRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvThresholdMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvTnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PpvTprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateAccuracyMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateF1ScoreMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateFdrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateFnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateForMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateFprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateInformednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateMarkednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateMccMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateNpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRatePpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRatePredictedPositiveRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateThresholdMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateTnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedNegativeRateTprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateAccuracyMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateF1ScoreMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateFdrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateFnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateForMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateFprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateInformednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateMarkednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateMccMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateNpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRatePpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRatePredictedNegativeRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateThresholdMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateTnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.PredictedPositiveRateTprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.RocAucMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

static _score(probabilities, labels)

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdAccuracyMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdF1ScoreMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdFdrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdFnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdForMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdFprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdInformednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdMarkednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdMccMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdNpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdPpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdPredictedNegativeRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdPredictedPositiveRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdTnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.ThresholdTprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrAccuracyMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrF1ScoreMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrFdrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrFnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrForMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrFprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrInformednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrMarkednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrMccMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrNpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrPpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrPredictedNegativeRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrPredictedPositiveRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrThresholdMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TnrTprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprAccuracyMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprF1ScoreMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprFdrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprFnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprForMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprFprMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprInformednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprMarkednessMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprMccMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprMetric(**kwargs)

Bases: AggregateBinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

static _score(predictions, labels)

Each aggregate needs to define a separate private method to actually calculate the aggregate

Separated from the public score method to enable easier testing and extension (values can be passed from non internal properties)

class simpleml.metrics.classification.TprNpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprPpvMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprPredictedNegativeRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprPredictedPositiveRateMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprThresholdMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values

class simpleml.metrics.classification.TprTnrMetric(**kwargs)

Bases: BinaryClassificationMetric

TODO: Figure out multiclass generalizations

Parameters

dataset_split – string denoting which dataset split to use can be one of: TRAIN, VALIDATION, Other. Other gets no prefix Default is train split to stay consistent with no split mapping to Train in Pipeline

score(self)

Abstract method for each metric to define

Should set self.values