simpleml.save_patterns.base module

Module for base save pattern definition

class simpleml.save_patterns.base.BaseSavePattern

Bases: simpleml.save_patterns.base.SavePatternMixin

Abstract base class for save patterns

load()

The load method invoked

save()

The save method invoked

class simpleml.save_patterns.base.SavePatternMixin

Bases: object

Mixin class with methods for different save operations

static df_to_sql(engine, df: pandas.core.frame.DataFrame, table: str, dtype: Optional[Dict[str, str]] = None, schema: str = 'public', if_exists: str = 'replace', sep: str = '|', encoding: str = 'utf8', index: bool = False) → None

Utility to bulk insert pandas dataframe via copy from

Parameters:

• df – dataframe to insert
• table – destination table
• dtype – column schema of destination table
• schema – destination schema
• if_exists – what to do if destination table exists; valid inputs are:

[replace, append, fail] :param sep: separator key between cells :param encoding: character encoding to use :param index: whether to output index with data

static hickle_object(obj: Any, filepath: str, overwrite: bool = True, root_directory: str = '/home/docs/.simpleml/filestore/HDF5/') → None

Serializes an object to the filesystem in HDF5 format.

Prepends path to SimpleML HDF5 directory before saving. ONLY pass in a relative filepath from that location

Parameters:overwrite – Boolean indicating whether to first check if HDF5 object is already serialized. Defaults to not checking, but can be leverage by implementations that want the same artifact in multiple places

static load_hickled_object(filepath: str, root_directory: str = '/home/docs/.simpleml/filestore/HDF5/') → Any

Loads an object from the filesystem.

Prepends path to SimpleML HDF5 directory before loading. ONLY pass in a relative filepath from that location

static load_keras_object(filepath: str, root_directory: str = '/home/docs/.simpleml/filestore/HDF5/') → Any

Loads a Keras object from the filesystem.

Prepends path to SimpleML HDF5 directory before loading. ONLY pass in a relative filepath from that location

static load_pickled_object(filepath: str, stream: bool = False, root_directory: str = '/home/docs/.simpleml/filestore/pickle/') → Any

Loads an object from a serialized string or filesystem. When stream is True, it tries to load the file directly from the string.

Prepends path to SimpleML Pickle directory before loading. ONLY pass in a relative filepath from that location

static load_sql(query: str, connection, **kwargs) → pandas.core.frame.DataFrame

Helper method to read in sql data

static pickle_object(obj: Any, filepath: Optional[str] = None, overwrite: bool = True, root_directory: str = '/home/docs/.simpleml/filestore/pickle/') → Optional[str]

Pickles an object to a string or to the filesystem. Assumes that a NULL filepath expects a serialized string returned

Prepends path to SimpleML Pickle directory before saving. ONLY pass in a relative filepath from that location

Parameters:overwrite – Boolean indicating whether to first check if pickled object is already serialized. Defaults to not checking, but can be leverage by implementations that want the same artifact in multiple places

static save_keras_object(obj: Any, filepath: str, overwrite: bool = True, root_directory: str = '/home/docs/.simpleml/filestore/HDF5/') → None

Serializes an object to the filesystem in Keras HDF5 format.

Prepends path to SimpleML HDF5 directory before saving. ONLY pass in a relative filepath from that location

Parameters:overwrite – Boolean indicating whether to first check if HDF5 object is already serialized. Defaults to not checking, but can be leverage by implementations that want the same artifact in multiple places