`simpleml.utils.initialization`¶

Util module to initialize SimpleML and configure database management

Module Contents¶

Classes¶

`AlembicDatabase`	Base database class to manage dbs with schema tracking. Includes alembic
`BaseDatabase`	Base Database class to configure db connection
`BinaryStorageDatabase`	Hardcoded database mapped to binary storage metadata
`Database`	SimpleML specific configuration to interact with the database
`DatasetDatabase`	Hardcoded database mapped to dataset storage metadata

simpleml.utils.initialization.DATABASE_CONF[source]¶

simpleml.utils.initialization.DATABASE_DRIVERNAME[source]¶

simpleml.utils.initialization.DATABASE_HOST[source]¶

simpleml.utils.initialization.DATABASE_NAME[source]¶

simpleml.utils.initialization.DATABASE_PASSWORD[source]¶

simpleml.utils.initialization.DATABASE_PORT[source]¶

simpleml.utils.initialization.DATABASE_QUERY[source]¶

simpleml.utils.initialization.DATABASE_URI[source]¶

simpleml.utils.initialization.DATABASE_USERNAME[source]¶

simpleml.utils.initialization.LOGGER[source]¶

simpleml.utils.initialization.__author__ = Elisha Yadgaran[source]¶

class simpleml.utils.initialization.AlembicDatabase(alembic_filepath, script_location='migrations', *args, **kwargs)[source]¶

Bases: simpleml.utils.initialization.BaseDatabase

Base database class to manage dbs with schema tracking. Includes alembic config references

Parameters

use_ssh_tunnel – boolean - default false. Whether to tunnel sqlalchemy connection through an ssh tunnel or not
sshtunnel_params – Dict of ssh params - specify according to sshtunnel project https://github.com/pahaz/sshtunnel/ - direct passthrough

property alembic_config(self)[source]¶

create_tables(self, base, drop_tables=False, ignore_errors=False)[source]¶

Creates database tables (and potentially drops existing ones). Assumes to be running under a sufficiently privileged user

Parameters: drop_tables – Whether or not to drop the existing tables first.
Returns: None

downgrade(self, revision)[source]¶: Proxy Method to invoke alembic downgrade command to specified revision Indirectly runs the alembic env.py code

initialize(self, base_list, upgrade=False, validate=True, **kwargs)[source]¶

Initialization method to set up database connection and inject session manager

Raises a SimpleML error if database schema is not up to date

Parameters

drop_tables – Bool, whether to drop existing tables in database
upgrade – Bool, whether to run an upgrade migration after establishing a connection

Returns

None

upgrade(self, revision='head')[source]¶: Proxy Method to invoke alembic upgrade command to specified revision Indirectly runs the alembic env.py code

validate_schema_version(self, base_list)[source]¶: Check that the newly initialized database is up-to-date Raises an error otherwise (ahead of any table model mismatches later)

class simpleml.utils.initialization.BaseDatabase(config=None, configuration_section=None, uri=None, use_ssh_tunnel=False, sshtunnel_params=None, **credentials)[source]¶

Bases: sqlalchemy.engine.url.URL

Base Database class to configure db connection Does not assume schema tracking or any other validation

Parameters

use_ssh_tunnel – boolean - default false. Whether to tunnel sqlalchemy connection through an ssh tunnel or not
sshtunnel_params – Dict of ssh params - specify according to sshtunnel project https://github.com/pahaz/sshtunnel/ - direct passthrough

_initialize(self, base, create_tables=False, **kwargs)[source]¶

Initialization method to set up database connection and inject session manager

Parameters

create_tables – Bool, whether to create tables in database
drop_tables – Bool, whether to drop existing tables in database

Returns

None

close_tunnel(self)[source]¶

configure_ssh_tunnel(self, credentials, ssh_config)[source]¶

create_tables(self, base, drop_tables=False, ignore_errors=False)[source]¶

Creates database tables (and potentially drops existing ones). Assumes to be running under a sufficiently privileged user

Parameters: drop_tables – Whether or not to drop the existing tables first.
Returns: None

property engine(self)[source]¶

initialize(self, base_list, **kwargs)[source]¶

Initialization method to set up database connection and inject session manager

Raises a SimpleML error if database schema is not up to date

Parameters

drop_tables – Bool, whether to drop existing tables in database
upgrade – Bool, whether to run an upgrade migration after establishing a connection

Returns

None

open_tunnel(self)[source]¶

property ssh_tunnel(self)[source]¶

class simpleml.utils.initialization.BinaryStorageDatabase(config=None, configuration_section=None, uri=None, use_ssh_tunnel=False, sshtunnel_params=None, **credentials)[source]¶

Bases: simpleml.utils.initialization.BaseDatabase

Hardcoded database mapped to binary storage metadata

Parameters

use_ssh_tunnel – boolean - default false. Whether to tunnel sqlalchemy connection through an ssh tunnel or not
sshtunnel_params – Dict of ssh params - specify according to sshtunnel project https://github.com/pahaz/sshtunnel/ - direct passthrough

initialize(self, base_list=None, **kwargs)[source]¶

Initialization method to set up database connection and inject session manager

Raises a SimpleML error if database schema is not up to date

Parameters

drop_tables – Bool, whether to drop existing tables in database
upgrade – Bool, whether to run an upgrade migration after establishing a connection

Returns

None

class simpleml.utils.initialization.Database(configuration_section=None, uri=None, database=None, username=None, password=None, drivername=None, host=None, port=None, query=None, *args, **kwargs)[source]¶

Bases: simpleml.utils.initialization.AlembicDatabase

SimpleML specific configuration to interact with the database Defaults to sqlite db in filestore directory

Parameters

use_ssh_tunnel – boolean - default false. Whether to tunnel sqlalchemy connection through an ssh tunnel or not
sshtunnel_params – Dict of ssh params - specify according to sshtunnel project https://github.com/pahaz/sshtunnel/ - direct passthrough

initialize(self, base_list=None, **kwargs)[source]¶

Initialization method to set up database connection and inject session manager

Raises a SimpleML error if database schema is not up to date

Parameters

drop_tables – Bool, whether to drop existing tables in database
upgrade – Bool, whether to run an upgrade migration after establishing a connection

Returns

None

class simpleml.utils.initialization.DatasetDatabase(config=None, configuration_section=None, uri=None, use_ssh_tunnel=False, sshtunnel_params=None, **credentials)[source]¶

Bases: simpleml.utils.initialization.BaseDatabase

Hardcoded database mapped to dataset storage metadata

Parameters

use_ssh_tunnel – boolean - default false. Whether to tunnel sqlalchemy connection through an ssh tunnel or not
sshtunnel_params – Dict of ssh params - specify according to sshtunnel project https://github.com/pahaz/sshtunnel/ - direct passthrough

initialize(self, base_list=None, **kwargs)[source]¶

Initialization method to set up database connection and inject session manager

Raises a SimpleML error if database schema is not up to date

Parameters

drop_tables – Bool, whether to drop existing tables in database
upgrade – Bool, whether to run an upgrade migration after establishing a connection

Returns

None

simpleml.utils.initialization¶

Module Contents¶

Classes¶

`simpleml.utils.initialization`¶