simpleml.utils.hash_recalculation

Util to recalculate persistable hashes

Module Contents

Classes

HashRecalculator

Utility class to recalculate hashes for persistables

Functions

recalculate_dataset_hashes(fail_on_error = False, recursively_recalculate_dependent_hashes = False)

Convenience helper to recompute dataset hashes. Optionally recalculates hashes

recalculate_metric_hashes(fail_on_error = False, recursively_recalculate_dependent_hashes = False)

Convenience helper to recompute metric hashes. Optionally recalculates hashes

recalculate_model_hashes(fail_on_error = False, recursively_recalculate_dependent_hashes = False)

Convenience helper to recompute model hashes. Optionally recalculates hashes

recalculate_pipeline_hashes(fail_on_error = False, recursively_recalculate_dependent_hashes = False)

Convenience helper to recompute pipeline hashes. Optionally recalculates hashes

Attributes

LOGGER

__author__

simpleml.utils.hash_recalculation.LOGGER[source]
simpleml.utils.hash_recalculation.__author__ = Elisha Yadgaran[source]
class simpleml.utils.hash_recalculation.HashRecalculator(fail_on_error, recursively_recalculate_dependent_hashes, dataset_ids=None, pipeline_ids=None, model_ids=None, metric_ids=None)[source]

Bases: object

Utility class to recalculate hashes for persistables

Useful for backfilling changes to hash logic and for database migrations that impact fields included in the hash (e.g. config metadata)

Expects to be called as part of an active session

``` HashRecalculator(

fail_on_error=False, recursively_recalculate_dependent_hashes=True

).run() ```

Parameters

  • fail_on_error (bool) –

  • recursively_recalculate_dependent_hashes (bool) –

  • dataset_ids (Optional[List[str]]) –

  • pipeline_ids (Optional[List[str]]) –

  • model_ids (Optional[List[str]]) –

  • metric_ids (Optional[List[str]]) –

ids_to_records(self, persistable_cls, ids)[source]
Parameters
Return type

List[simpleml.persistables.base_persistable.Persistable]

property is_finished(self)[source]
process_queue(self, queue)[source]

Loop one iteration through a queue – adds items back to queues if recursive parameter set

Parameters

queue (queue.SimpleQueue) –

Return type

None

queue_dependent_persistables(self, persistable)[source]

Queries for dependent persistables and queues them into the respective queues

Parameters

persistable (simpleml.persistables.base_persistable.Persistable) –

Return type

None

recalculate_hash(self, record)[source]
run(self)[source]
Return type

None

simpleml.utils.hash_recalculation.recalculate_dataset_hashes(fail_on_error=False, recursively_recalculate_dependent_hashes=False)[source]

Convenience helper to recompute dataset hashes. Optionally recalculates hashes for downstream persistables

Parameters

  • fail_on_error (bool) –

  • recursively_recalculate_dependent_hashes (bool) –

Return type

None

simpleml.utils.hash_recalculation.recalculate_metric_hashes(fail_on_error=False, recursively_recalculate_dependent_hashes=False)[source]

Convenience helper to recompute metric hashes. Optionally recalculates hashes for downstream persistables

Parameters

  • fail_on_error (bool) –

  • recursively_recalculate_dependent_hashes (bool) –

Return type

None

simpleml.utils.hash_recalculation.recalculate_model_hashes(fail_on_error=False, recursively_recalculate_dependent_hashes=False)[source]

Convenience helper to recompute model hashes. Optionally recalculates hashes for downstream persistables

Parameters

  • fail_on_error (bool) –

  • recursively_recalculate_dependent_hashes (bool) –

Return type

None

simpleml.utils.hash_recalculation.recalculate_pipeline_hashes(fail_on_error=False, recursively_recalculate_dependent_hashes=False)[source]

Convenience helper to recompute pipeline hashes. Optionally recalculates hashes for downstream persistables

Parameters

  • fail_on_error (bool) –

  • recursively_recalculate_dependent_hashes (bool) –

Return type

None