simpleml.orm.persistable

Base class for all database tracked records, called “Persistables”

Module Contents

Classes

ORMPersistable

Base class for all SimpleML database objects.

Attributes

LOGGER
__author__

simpleml.orm.persistable.LOGGER

simpleml.orm.persistable.__author__ = Elisha Yadgaran

class simpleml.orm.persistable.ORMPersistable

Bases: simpleml.orm.metadata.SimplemlCoreSqlalchemy

Base class for all SimpleML database objects. dialect can be swapped out for any supported SQLAlchemy backend.

Takes advantage of sqlalchemy-mixins to enable active record operations (TableModel.save(), create(), where(), destroy())

Uses private class attributes for internal artifact registry Does not need to be persisted because it gets populated on import (and can therefore be changed between versions) cls._ARTIFACT_{artifact_name} = {‘save’: save_attribute, ‘restore’: restore_attribute}

id: Random UUID(4). Used over auto incrementing id to minimize collision probability

with distributed trainings and authors (especially if using central server to combine results across different instantiations of SimpleML)

hash_id: Use hash of object to uniquely identify the contents at train time registered_name: class name of object defined when importing

Can be used for the drag and drop GUI - also for prescribing training config

author: creator project: Project objects are associated with. Useful if multiple persistables

relate to the same project and want to be grouped (but have different names) also good for implementing row based security across teams

name: friendly name - primary way of tracking evolution of “same” object over time version: autoincrementing id of “friendly name” version_description: description that explains what is new or different about this version

# Persistence of fitted states has_external_files = boolean field to signify presence of saved files not in (main) db filepaths = JSON object with external file details

The nested notation is because any persistable can implement multiple save options (with arbitrary priority) and arbitrary inputs. Simple serialization could have only a single string location whereas complex artifacts might have a list or map of filepaths

Structure: {

artifact_name: {

‘save_pattern’: filepath_data

}, “example”: {

“disk_pickled”: path to file, relative to base simpleml folder (default ~/.simpleml), “database”: {“schema”: schema, “table”: table_name}, # (for files extractable with select * from) …

}

}

metadata: Generic JSON store for random attributes

__abstract__ = True

author

filepaths

has_external_files

hash_

id

metadata_

name

project

registered_name

version

version_description

_load_class(self)

Wrapper function to call global registry of all imported class names

Return type

simpleml.persistables.base_persistable.Persistable

classmethod get_latest_version(cls, name)

Versions should be autoincrementing for each object (constrained over friendly name). Executes a database lookup and increments..

Parameters

name (str) –

Return type

int

load(self, load_externals=False)

Counter operation for save Needs to load any file and db objects

Class definition is stored by registered_name param and Pickled objects are stored in external_filename param

Parameters

load_externals (bool) – Boolean flag whether to load the external files

Return type

simpleml.persistables.base_persistable.Persistable

useful for relationships that only need class definitions and not data

static load_reference(reference_cls, id)

Parameters

• reference_cls (ORMPersistable) –

• id (str) –

Return type

simpleml.persistables.base_persistable.Persistable

classmethod save_record(cls, id, **kwargs)

save overloads parent method that is called by helper methods for create/update

Parameters

id (str) –

Return type

None

to_dict(self)

Utility method to inspect the orm model and return a dictionary of attributes -> values

Uses the mapped attribute name, not the column name (e.g. hash_ vs hash). excludes relationships (to support lazy loading)