simpleml.persistables.saving module

Module to define the mixins that support different persistence patterns for external objects

  • Dataframe saving (as tables in dedicated schema)
  • Pickled Object saving
    • In database as a binary blob
    • To local filestore
  • HDF5 object saving
    • In database as a binary blob
    • To local filestore
  • Remote filestore saving
    • S3
    • Google Cloud
    • Azure
class simpleml.persistables.saving.AllSaveMixin[source]

Bases: simpleml.persistables.saving.DataframeTableSaveMixin, simpleml.persistables.saving.DatabasePickleSaveMixin, simpleml.persistables.saving.DiskPickleSaveMixin, simpleml.persistables.saving.DiskHDF5SaveMixin, simpleml.persistables.saving.KerasDiskHDF5SaveMixin

class simpleml.persistables.saving.DatabasePickleSaveMixin[source]

Bases: simpleml.persistables.saving.ExternalSaveMixin

Mixin class to save binary objects to a database table

Expects the following available attributes:
  • self._external_file
  • self.id
  • self.object_type
Sets the following attributes:
  • self.filepaths
  • self.unloaded_externals
class simpleml.persistables.saving.DataframeTableSaveMixin[source]

Bases: simpleml.persistables.saving.ExternalSaveMixin

Mixin class to save dataframes to a database table

Expects the following available attributes:
  • self._external_file
  • self._schema
  • self.id
  • self._engine
  • self.dataframe
Sets the following attributes:
  • self.filepaths
static df_to_sql(engine, df, table, dtype=None, schema='public', if_exists='replace', sep='|', encoding='utf8', index=False)[source]

Utility to bulk insert pandas dataframe via copy from

Parameters:
  • df – dataframe to insert
  • table – destination table
  • dtype – column schema of destination table
  • schema – destination schema
  • if_exists – what to do if destination table exists; valid inputs are:

[replace, append, fail] :param sep: separator key between cells :param encoding: character encoding to use :param index: whether to output index with data

class simpleml.persistables.saving.DiskHDF5SaveMixin[source]

Bases: simpleml.persistables.saving.ExternalSaveMixin

Mixin class to save objects to disk in HDF5 format with hickle

Expects the following available attributes:
  • self._external_file
  • self.id
Sets the following attributes:
  • self.filepaths
  • self.unloaded_externals
class simpleml.persistables.saving.DiskPickleSaveMixin[source]

Bases: simpleml.persistables.saving.ExternalSaveMixin

Mixin class to save objects to disk in pickled format

Expects the following available attributes:
  • self._external_file
  • self.id
Sets the following attributes:
  • self.filepaths
  • self.unloaded_externals
class simpleml.persistables.saving.ExternalSaveMixin[source]

Bases: object

class simpleml.persistables.saving.KerasDiskHDF5SaveMixin[source]

Bases: simpleml.persistables.saving.ExternalSaveMixin

Mixin class to save objects to disk in Keras’s HDF5 format Keras’s internal persistence mechanism utilizes HDF5 and implements a custom pattern

Expects the following available attributes:
  • self._external_file
  • self.id
Sets the following attributes:
  • self.filepaths
  • self.unloaded_externals