simpleml.metrics.base_metric
Module Contents
Classes
Abstract Base class for all Metric objects |
|
Base class for all Metric objects |
Attributes
- class simpleml.metrics.base_metric.AbstractMetric(name=None, has_external_files=False, author=None, project=None, version_description=None, save_patterns=None, **kwargs)[source]
Bases:
with_metaclass
(MetricRegistry
,Persistable
)Abstract Base class for all Metric objects
name: the metric name values: JSON object with key: value pairs for performance on test dataset
(ex: FPR: TPR to create ROC Curve) Singular value metrics take the form - {‘agg’: value}
- Parameters
- _get_dataset_split(self, **kwargs)[source]
Default accessor for dataset data. REFERS TO RAW DATASETS not the pipelines superimposed. That means that datasets that do not define explicit splits will have no notion of downstream splits (e.g. RandomSplitPipeline)
- Return type
Any
- _get_latest_version(self)[source]
Versions should be autoincrementing for each object (constrained over friendly name and model). Executes a database lookup and increments..
- Return type
- _get_pipeline_split(self, column, split, **kwargs)[source]
For special case where dataset is the same as the model’s dataset, the dataset splits can refer to the pipeline imposed splits, not the inherent dataset’s splits. Use the pipeline split then ex: RandomSplitPipeline on NoSplitDataset evaluating “in_sample” performance
- _hash(self)[source]
- Hash is the combination of the:
Model
Dataset (optional)
Metric
Config
- Return type
- add_dataset(self, dataset)[source]
Setter method for dataset used
- Parameters
dataset (simpleml.datasets.base_dataset.Dataset) –
- Return type
None
- add_model(self, model)[source]
Setter method for model used
- Parameters
model (simpleml.models.base_model.Model) –
- Return type
None
- class simpleml.metrics.base_metric.Metric(name=None, has_external_files=False, author=None, project=None, version_description=None, save_patterns=None, **kwargs)[source]
Bases:
AbstractMetric
Base class for all Metric objects
model_id: foreign key to the model that was used to generate predictions
- TODO: Should join criteria be composite of model and dataset for multiple
duplicate metric objects computed over different test datasets?
- Parameters