Predict

Similar to training, prediction can be done via three interfaces:

  • via python, das.predict.predict

  • via the command line, das predict, with audio data from a wav file.

  • the GUI - see the GUI tutorial

Prediction will:

  • load the audio data and the network

  • run inference to produce confidence scores (class_probabilties)

  • post-process the confidence score to extract the times of events and label segments.

Prediction using python

import numpy as np
from pprint import pprint
import scipy.io.wavfile
import das.predict
help(das.predict.predict)

%%time
samplerate, x = scipy.io.wavfile.read('dat/dmel_song_rt.wav')
print(f"DAS requires [T, channels], but single-channel wave files are loaded with shape [T,] (data shape is {x.shape}).")
x = np.atleast_2d(x).T
events, segments, class_probabilities, class_names = das.predict.predict(x, 
                                                           model_save_name='models/dmel_single_rt/20200430_201821',
                                                           verbose=2,
                                                           segment_minlen=0.02,
                                                           segment_fillgap=0.02)

Outputs of predict

  • class_probabilties: [T, nb_classes] including noise.

  • segments: Labelled segments

    • samplerate_Hz:

    • names: names of all segment types

    • index: indices of all segments types into class_probabiltiies

    • probabilities = class_probabilites[:, index]

    • sequence: sequence of segment names (one entry per detected segment). Excludes noise

    • samples: labelled sample trace (label of the sequence occupying each sample)

    • onsets_seconds, offsets_seconds, durations_seconds: Onsets, offsets, and duration of individual segmeents

  • events: Detected events

    • samplerate_Hz:

    • index: indices of all events types into class_probabiltiies

    • names: names of all event types

    • probabilities: probabilities (confidence scores) for detected events. Value of class_probabilities for the detected event index at each event time.

    • seconds: times (seconds) of detected events

    • sequence: sequence of event names (one per detected event).

import matplotlib.pyplot as plt
plt.style.use('ncb.mplstyle')

t0 = 0
t1 = 30_000 
fs =segments['samplerate_Hz']
time = np.arange(t0, t1) / fs
nb_classes = class_probabilities.shape[1]

plt.figure(figsize=(30, 10))
plt.subplot(411)
plt.plot(time, x[t0:t1], 'k', linewidth=0.5)
plt.title('Song')
plt.xticks([])
plt.ylim(-0.25, 0.25)

plt.subplot(412)
plt.imshow(class_probabilities[t0:t1].T, cmap='Greys')
plt.yticks(np.arange(nb_classes), labels=class_names)
plt.title('Raw confidence scores')
plt.xticks([])

ax = plt.subplot(413)
plt.plot(time, x[t0:t1],'k', linewidth=0.5)
plt.ylim(-0.25, 0.25)
plt.title('Annotations')
plt.xlabel('Time [seconds]')
for onset, offset, segment_name in zip(segments['onsets_seconds'], segments['offsets_seconds'], segments['sequence']):
    if onset >= t0 /fs and offset <= t1 / fs:
        plt.plot([onset, offset], [0.1, 0.1], c='b')
        ax.annotate(segment_name, xy=(onset, 0.11), c='b')

for pulse_time, pulse_name in zip(events['seconds'], events['sequence']):
    if pulse_time >= t0 /fs and pulse_time <= t1 / fs:
        plt.axvline(pulse_time, c='r')
        ax.annotate(pulse_name, xy=(pulse_time, 0.1), c='r', rotation=-90)

Prediction using command-line scripts

Will save the output of das.predict.predict to a h5 file ending in _das.h5 or specified via the --save-filename argument.

See cli for a full list of arguments.

!das predict dat/dmel_song_rt.wav models/dmel_single_rt/20200430_201821

import h5py
with h5py.File('dat/dmel_song_rt_das.h5', mode='r') as f:
    print(list(f.keys()))