simpleml.persistables.saving module¶
Module to define the mixins that support different persistence patterns for external objects
- Dataframe saving (as tables in dedicated schema)
- Pickled Object saving
- In database as a binary blob
- To local filestore
- HDF5 object saving
- In database as a binary blob
- To local filestore
- Remote filestore saving
- S3
- Google Cloud
- Azure
-
class
simpleml.persistables.saving.
AllSaveMixin
[source]¶ Bases:
simpleml.persistables.saving.DataframeTableSaveMixin
,simpleml.persistables.saving.DatabasePickleSaveMixin
,simpleml.persistables.saving.DiskPickleSaveMixin
,simpleml.persistables.saving.DiskHDF5SaveMixin
,simpleml.persistables.saving.KerasDiskHDF5SaveMixin
-
class
simpleml.persistables.saving.
DatabasePickleSaveMixin
[source]¶ Bases:
simpleml.persistables.saving.BaseExternalSaveMixin
Mixin class to save binary objects to a database table
- Expects the following available attributes:
- self._external_file
- self.id
- self.object_type
- Sets the following attributes:
- self.filepaths
- self.unloaded_externals
-
class
simpleml.persistables.saving.
DataframeTableSaveMixin
[source]¶ Bases:
simpleml.persistables.saving.BaseExternalSaveMixin
Mixin class to save dataframes to a database table
- Expects the following available attributes:
- self._external_file
- self._schema
- self.id
- self._engine
- self.dataframe
- Sets the following attributes:
- self.filepaths
-
static
df_to_sql
(engine, df, table, dtype=None, schema='public', if_exists='replace', sep='|', encoding='utf8', index=False)[source]¶ Utility to bulk insert pandas dataframe via copy from
Parameters: - df – dataframe to insert
- table – destination table
- dtype – column schema of destination table
- schema – destination schema
- if_exists – what to do if destination table exists; valid inputs are:
[replace, append, fail] :param sep: separator key between cells :param encoding: character encoding to use :param index: whether to output index with data
-
class
simpleml.persistables.saving.
DiskHDF5SaveMixin
[source]¶ Bases:
simpleml.persistables.saving.BaseExternalSaveMixin
Mixin class to save objects to disk in HDF5 format with hickle
- Expects the following available attributes:
- self._external_file
- self.id
- Sets the following attributes:
- self.filepaths
- self.unloaded_externals
-
class
simpleml.persistables.saving.
DiskPickleSaveMixin
[source]¶ Bases:
simpleml.persistables.saving.BaseExternalSaveMixin
Mixin class to save objects to disk in pickled format
- Expects the following available attributes:
- self._external_file
- self.id
- Sets the following attributes:
- self.filepaths
- self.unloaded_externals
-
class
simpleml.persistables.saving.
KerasDiskHDF5SaveMixin
[source]¶ Bases:
simpleml.persistables.saving.BaseExternalSaveMixin
Mixin class to save objects to disk in Keras’s HDF5 format Keras’s internal persistence mechanism utilizes HDF5 and implements a custom pattern
- Expects the following available attributes:
- self._external_file
- self.id
- Sets the following attributes:
- self.filepaths
- self.unloaded_externals