simpleml.metrics.base_metric

Module Contents

Classes

Metric

Base class for all Metric objects

Attributes

LOGGER

__author__

simpleml.metrics.base_metric.LOGGER[source]
simpleml.metrics.base_metric.__author__ = Elisha Yadgaran[source]
class simpleml.metrics.base_metric.Metric(dataset_id=None, model_id=None, **kwargs)[source]

Bases: simpleml.persistables.base_persistable.Persistable

Base class for all Metric objects

Parameters

  • dataset_id (Optional[Union[str, uuid.uuid4]]) –

  • model_id (Optional[Union[str, uuid.uuid4]]) –

object_type :str = METRIC[source]
_get_dataset_split(self, **kwargs)[source]

Default accessor for dataset data. REFERS TO RAW DATASETS not the pipelines superimposed. That means that datasets that do not define explicit splits will have no notion of downstream splits (e.g. RandomSplitPipeline)

Return type

Any

_get_latest_version(self)[source]

Versions should be autoincrementing for each object (constrained over friendly name and model). Executes a database lookup and increments..

Return type

int

_get_pipeline_split(self, column, split, **kwargs)[source]

For special case where dataset is the same as the model’s dataset, the dataset splits can refer to the pipeline imposed splits, not the inherent dataset’s splits. Use the pipeline split then ex: RandomSplitPipeline on NoSplitDataset evaluating “in_sample” performance

Parameters

  • column (str) –

  • split (str) –

Return type

Any

_hash(self)[source]
Hash is the combination of the:

  1. Model

  2. Dataset (optional)

  3. Metric

  4. Config

Return type

str

_load_dataset(self)[source]

Helper to fetch the dataset

_load_model(self)[source]

Helper to fetch the model

add_dataset(self, dataset)[source]

Setter method for dataset used

Parameters

dataset (simpleml.datasets.base_dataset.Dataset) –

Return type

None

add_model(self, model)[source]

Setter method for model used

Parameters

model (simpleml.models.base_model.Model) –

Return type

None

property dataset(self)[source]

Use a weakref to bind linked dataset so it doesnt bloat usage returns dataset if still available or tries to fetch otherwise

property model(self)[source]

Use a weakref to bind linked model so it doesnt bloat usage returns model if still available or tries to fetch otherwise

save(self, **kwargs)[source]

Extend parent function with a few additional save routines

Return type

None

abstract score(self, **kwargs)[source]

Abstract method for each metric to define

Should set self.values