simpleml._external.joblib

Copied from joblib library – no modification, just avoiding unnecessary dependencies https://github.com/joblib/joblib/blob/master/joblib/hashing.py#L246

Fast cryptographic hash of Python objects, with a special case for fast hashing of numpy arrays.

Module Contents

Classes

Hasher	A subclass of pickler, to do cryptographic hashing, rather than
NumpyHasher	Special case the hasher for when numpy is loaded.
_ConsistentSet	Class used to ensure the hash of Sets is preserved
_MyHash	Class used to hash objects that won't normally pickle

Functions

hash(obj, hash_name='md5', coerce_mmap=False)

Quick calculation of a hash to identify uniquely Python objects

Attributes

PY3_OR_LATER
Pickler
_basestring

simpleml._external.joblib.PY3_OR_LATER

simpleml._external.joblib.Pickler

simpleml._external.joblib._basestring

class simpleml._external.joblib.Hasher(hash_name='md5')

Bases: Pickler

A subclass of pickler, to do cryptographic hashing, rather than pickling.

This takes a binary file for writing a pickle data stream.

The optional protocol argument tells the pickler to use the given protocol; supported protocols are 0, 1, 2, 3 and 4. The default protocol is 3; a backward-incompatible protocol designed for Python 3.

Specifying a negative protocol version selects the highest protocol version supported. The higher the protocol used, the more recent the version of Python needed to read the pickle produced.

The file argument must have a write() method that accepts a single bytes argument. It can thus be a file object opened for binary writing, an io.BytesIO instance, or any other custom object that meets this interface.

If fix_imports is True and protocol is less than 3, pickle will try to map the new Python 3 names to the old module names used in Python 2, so that the pickle data stream is readable with Python 2.

dispatch

_batch_setitems(self, items)

hash(self, obj, return_digest=True)

memoize(self, obj)

Store an object in the memo.

save(self, obj)

save_global(self, obj, name=None, pack=struct.pack)

save_set(self, set_items)

class simpleml._external.joblib.NumpyHasher(hash_name='md5', coerce_mmap=False)

Bases: Hasher

Special case the hasher for when numpy is loaded.

hash_name: string

The hash algorithm to be used

coerce_mmap: boolean

Make no difference between np.memmap and np.ndarray objects.

save(self, obj)

Subclass the save method, to hash ndarray subclass, rather than pickling them. Off course, this is a total abuse of the Pickler class.

class simpleml._external.joblib._ConsistentSet(set_sequence)

Bases: object

Class used to ensure the hash of Sets is preserved whatever the order of its items.

class simpleml._external.joblib._MyHash(*args)

Bases: object

Class used to hash objects that won’t normally pickle

simpleml._external.joblib.hash(obj, hash_name='md5', coerce_mmap=False)

Quick calculation of a hash to identify uniquely Python objects containing numpy arrays. Parameters ———– hash_name: ‘md5’ or ‘sha1’

Hashing algorithm used. sha1 is supposedly safer, but md5 is faster.

coerce_mmap: boolean

Make no difference between np.memmap and np.ndarray