das.npy_dir#
Dict of dicts <-> hierarchy of npy files.
Tools for working with dictionaries of dictionaries. The keys of the first dictionary are mapped to directories. The values in the nested dictionary are saved as npy files with the key as the name.
As an example, this code:
`python
data = {'first_level': {'song': some_data, 'response': some_other_data}}
data.attrs = {'song_name': 'this is the dream...', 'response_name': 'yeah'}
das.npy.save('song_responses.npy', data)
`
will result in a folder song_responses.npy containing
one subfolder first_level with two npy files - song.npy containing some_data and
responses.npy containing some_other_data. The attrs attribute can be used to store metadata and will be stored in song_responses.npy/attrs.npy)
data = das.npy.load('song_responses.npy') will restore the data with the metadata.
- class das.npy_dir.DictClass(*args, **kwargs)[source]#
Wrap dict in class so we can attach attrs to it.
- das.npy_dir.load(location: str, memmap_dirs: Optional[Union[List[str], str]] = None) das.npy_dir.DictClass[source]#
Load hierarchy of npy files into dict of dicts.
- Parameters
location ([type]) – [description]
memmap_dirs (list, optional) – List of dirs to memmap. String ‘all’ will memmmap all dirs. Defaults to [‘train’].
- Returns
[description]
- Return type
dict
- das.npy_dir.save(location: str, data: das.npy_dir.DictClass) None[source]#
Save nested dict in data to location as a directory with npy files dir.
- Parameters
location ([type]) – [description]
data ([type]) – [description]