das.npy_dir#

Dict of dicts <-> hierarchy of npy files.

Tools for working with dictionaries of dictionaries. The keys of the first dictionary are mapped to directories. The values in the nested dictionary are saved as npy files with the key as the name.

As an example, this code: `python data = {'first_level': {'song': some_data, 'response': some_other_data}} data.attrs = {'song_name': 'this is the dream...', 'response_name': 'yeah'} das.npy.save('song_responses.npy', data) ` will result in a folder song_responses.npy containing one subfolder first_level with two npy files - song.npy containing some_data and responses.npy containing some_other_data. The attrs attribute can be used to store metadata and will be stored in song_responses.npy/attrs.npy)

data = das.npy.load('song_responses.npy') will restore the data with the metadata.

class das.npy_dir.DictClass(*args, **kwargs)[source]#

Wrap dict in class so we can attach attrs to it.

das.npy_dir.load(location: str, memmap_dirs: Optional[Union[List[str], str]] = None) das.npy_dir.DictClass[source]#

Load hierarchy of npy files into dict of dicts.

Parameters
  • location ([type]) – [description]

  • memmap_dirs (list, optional) – List of dirs to memmap. String ‘all’ will memmmap all dirs. Defaults to [‘train’].

Returns

[description]

Return type

dict

das.npy_dir.save(location: str, data: das.npy_dir.DictClass) None[source]#

Save nested dict in data to location as a directory with npy files dir.

Parameters
  • location ([type]) – [description]

  • data ([type]) – [description]