Training and inference with colab

Google colab is a free server for running notebooks with GPU/TPU support - this is a great way to use DeepSS if you do not have a computer with a GPU.

This notebook demonstrates

  • how to setup DeepSS

  • how to load your own datasets

  • how to train a network and then use that network to label a new audio recording.

Open and edit this notebook in colab by clicking this badge:

Open In Colab

Let’s make sure we are using tensorflow v2+

%tensorflow_version 2.x

Install DeepSS:

!pip install deepss
Collecting deepss
?25l  Downloading https://files.pythonhosted.org/packages/f1/87/d9f503b45fe5d34a138ced6d9de238b10b3297e2adf467b407400074becf/deepss-0.16.0-py3-none-any.whl (69kB)

     |████▊                           | 10kB 26.3MB/s eta 0:00:01
     |█████████▍                      | 20kB 33.2MB/s eta 0:00:01
     |██████████████                  | 30kB 25.6MB/s eta 0:00:01
     |██████████████████▉             | 40kB 19.2MB/s eta 0:00:01
     |███████████████████████▌        | 51kB 14.3MB/s eta 0:00:01
     |████████████████████████████▏   | 61kB 13.6MB/s eta 0:00:01
     |████████████████████████████████| 71kB 4.4MB/s 
?25hCollecting flammkuchen
  Downloading https://files.pythonhosted.org/packages/e7/a1/302fde5b5a2217f488893d73e3f556f1fc6379beb179faf9f39eecf20e8f/flammkuchen-0.9.2-py2.py3-none-any.whl
Requirement already satisfied: librosa in /usr/local/lib/python3.6/dist-packages (from deepss) (0.6.3)
Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from deepss) (1.18.5)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from deepss) (3.13)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (from deepss) (3.2.2)
Collecting peakutils
  Downloading https://files.pythonhosted.org/packages/0a/11/6416c8aebba4d5f73e23e1f070a419a8944f3ba17eed9efdf9cdc95f0411/PeakUtils-1.3.3-py3-none-any.whl
Collecting matplotlib_scalebar
  Downloading https://files.pythonhosted.org/packages/89/ba/46ae95cc9d2c49f2d13ec6f4322f06a5e7621bee03685fab59c64f321afd/matplotlib_scalebar-0.6.2-py2.py3-none-any.whl
Requirement already satisfied: sklearn in /usr/local/lib/python3.6/dist-packages (from deepss) (0.0)
Collecting defopt
  Downloading https://files.pythonhosted.org/packages/2f/a6/17700394dddbf65f7c84bcee0c99aae6eb581ae3882c6e0eb26cfd23cfaa/defopt-6.0.2.tar.gz
Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from deepss) (1.4.1)
Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from deepss) (1.1.5)
Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from deepss) (2.10.0)
Collecting zarr
?25l  Downloading https://files.pythonhosted.org/packages/60/67/d69a3fcd34254df8dce23e1a4d570509f8a373fc58e40cb8681cf26795f6/zarr-2.6.1-py3-none-any.whl (132kB)
     |████████████████████████████████| 133kB 20.5MB/s 
?25hRequirement already satisfied: tables in /usr/local/lib/python3.6/dist-packages (from flammkuchen->deepss) (3.4.4)
Requirement already satisfied: scikit-learn!=0.19.0,>=0.14.0 in /usr/local/lib/python3.6/dist-packages (from librosa->deepss) (0.22.2.post1)
Requirement already satisfied: joblib>=0.12 in /usr/local/lib/python3.6/dist-packages (from librosa->deepss) (0.17.0)
Requirement already satisfied: resampy>=0.2.0 in /usr/local/lib/python3.6/dist-packages (from librosa->deepss) (0.2.2)
Requirement already satisfied: decorator>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from librosa->deepss) (4.4.2)
Requirement already satisfied: audioread>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from librosa->deepss) (2.1.9)
Requirement already satisfied: six>=1.3 in /usr/local/lib/python3.6/dist-packages (from librosa->deepss) (1.15.0)
Requirement already satisfied: numba>=0.38.0 in /usr/local/lib/python3.6/dist-packages (from librosa->deepss) (0.48.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->deepss) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->deepss) (1.3.1)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->deepss) (2.8.1)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->deepss) (2.4.7)
Requirement already satisfied: docutils>=0.10 in /usr/local/lib/python3.6/dist-packages (from defopt->deepss) (0.16)
Collecting sphinxcontrib-napoleon>=0.7.0
  Downloading https://files.pythonhosted.org/packages/75/f2/6b7627dfe7b4e418e295e254bb15c3a6455f11f8c0ad0d43113f678049c3/sphinxcontrib_napoleon-0.7-py2.py3-none-any.whl
Requirement already satisfied: typing_extensions>=3.7.4 in /usr/local/lib/python3.6/dist-packages (from defopt->deepss) (3.7.4.3)
Collecting typing_inspect>=0.3.1
  Downloading https://files.pythonhosted.org/packages/42/1c/66402db44184904a2f14722d317a4da0b5c8c78acfc3faf74362566635c5/typing_inspect-0.6.0-py3-none-any.whl
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas->deepss) (2018.9)
Collecting numcodecs>=0.6.4
?25l  Downloading https://files.pythonhosted.org/packages/94/dc/0b7ae820c6c5f3a8b79912ec43dd9a1305e33970a05d3c02a1e8203ead9c/numcodecs-0.7.2-cp36-cp36m-manylinux2010_x86_64.whl (5.8MB)
     |████████████████████████████████| 5.8MB 12.5MB/s 
?25hCollecting fasteners
  Downloading https://files.pythonhosted.org/packages/18/bd/55eb2d6397b9c0e263af9d091ebdb756b15756029b3cededf6461481bc63/fasteners-0.15-py2.py3-none-any.whl
Collecting asciitree
  Downloading https://files.pythonhosted.org/packages/2d/6a/885bc91484e1aa8f618f6f0228d76d0e67000b0fdd6090673b777e311913/asciitree-0.3.3.tar.gz
Requirement already satisfied: numexpr>=2.5.2 in /usr/local/lib/python3.6/dist-packages (from tables->flammkuchen->deepss) (2.7.1)
Requirement already satisfied: llvmlite<0.32.0,>=0.31.0dev0 in /usr/local/lib/python3.6/dist-packages (from numba>=0.38.0->librosa->deepss) (0.31.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from numba>=0.38.0->librosa->deepss) (50.3.2)
Collecting pockets>=0.3
  Downloading https://files.pythonhosted.org/packages/e9/2f/a4583c70fbd8cd04910e2884bcc2bdd670e884061f7b4d70bc13e632a993/pockets-0.9.1-py2.py3-none-any.whl
Collecting mypy-extensions>=0.3.0
  Downloading https://files.pythonhosted.org/packages/5c/eb/975c7c080f3223a5cdaff09612f3a5221e4ba534f7039db34c35d95fa6a5/mypy_extensions-0.4.3-py2.py3-none-any.whl
Collecting monotonic>=0.1
  Downloading https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl
Building wheels for collected packages: defopt, asciitree
  Building wheel for defopt (setup.py) ... ?25l?25hdone
  Created wheel for defopt: filename=defopt-6.0.2-cp36-none-any.whl size=13860 sha256=ddf63f16e8608dab87e5276d50f57b4733544b6af5452bc5474816a655db7c93
  Stored in directory: /root/.cache/pip/wheels/c3/fc/97/353b31fe86a0166c07e2ca598f20f377f19bd51828865dc971
  Building wheel for asciitree (setup.py) ... ?25l?25hdone
  Created wheel for asciitree: filename=asciitree-0.3.3-cp36-none-any.whl size=5038 sha256=393ac8e6811ddb295bcdb591e895721cce1b958bddb4c5e1d5edebba92acdec5
  Stored in directory: /root/.cache/pip/wheels/1d/d9/58/9808b306744df0208fccc640d3d9952a5bc7468502d42897d5
Successfully built defopt asciitree
Installing collected packages: flammkuchen, peakutils, matplotlib-scalebar, pockets, sphinxcontrib-napoleon, mypy-extensions, typing-inspect, defopt, numcodecs, monotonic, fasteners, asciitree, zarr, deepss
Successfully installed asciitree-0.3.3 deepss-0.16.0 defopt-6.0.2 fasteners-0.15 flammkuchen-0.9.2 matplotlib-scalebar-0.6.2 monotonic-1.5 mypy-extensions-0.4.3 numcodecs-0.7.2 peakutils-1.3.3 pockets-0.9.1 sphinxcontrib-napoleon-0.7 typing-inspect-0.6.0 zarr-2.6.1

Import all the things:

import dss.train, dss.predict, dss.utils, dss.npy_dir
import matplotlib.pyplot as plt
import flammkuchen
import logging
logging.basicConfig(level=logging.INFO)

Mount google drive so you can access your own dataasets - this will ask for authentication.

from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive

Train the model

Adjust the path to point to your dataset.

path_to_data = '/content/drive/MyDrive/Dmoj.wrigleyi.npy'  # '/content/drive/MyDrive/tutorial_dataset.npy' 

dss.train.train(model_name='tcn',
                data_dir=path_to_data,
                save_dir='res',
                nb_hist=1024,
                kernel_size=32,
                nb_filters=32,
                ignore_boundaries=True,
                verbose=2,
                nb_conv=4,
                learning_rate=0.0005,
                use_separable=[True, True, False, False],
                nb_epoch=1000)
Model: "TCN"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 1024, 16)]   0                                            
__________________________________________________________________________________________________
conv1d (Conv1D)                 (None, 1024, 32)     544         input_1[0][0]                    
__________________________________________________________________________________________________
separable_conv1d (SeparableConv (None, 1024, 32)     8224        conv1d[0][0]                     
__________________________________________________________________________________________________
activation (Activation)         (None, 1024, 32)     0           separable_conv1d[0][0]           
__________________________________________________________________________________________________
lambda (Lambda)                 (None, 1024, 32)     0           activation[0][0]                 
__________________________________________________________________________________________________
spatial_dropout1d (SpatialDropo (None, 1024, 32)     0           lambda[0][0]                     
__________________________________________________________________________________________________
conv1d_1 (Conv1D)               (None, 1024, 32)     1056        spatial_dropout1d[0][0]          
__________________________________________________________________________________________________
add (Add)                       (None, 1024, 32)     0           conv1d[0][0]                     
                                                                 conv1d_1[0][0]                   
__________________________________________________________________________________________________
separable_conv1d_1 (SeparableCo (None, 1024, 32)     8224        add[0][0]                        
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 1024, 32)     0           separable_conv1d_1[0][0]         
__________________________________________________________________________________________________
lambda_1 (Lambda)               (None, 1024, 32)     0           activation_1[0][0]               
__________________________________________________________________________________________________
spatial_dropout1d_1 (SpatialDro (None, 1024, 32)     0           lambda_1[0][0]                   
__________________________________________________________________________________________________
conv1d_2 (Conv1D)               (None, 1024, 32)     1056        spatial_dropout1d_1[0][0]        
__________________________________________________________________________________________________
add_1 (Add)                     (None, 1024, 32)     0           add[0][0]                        
                                                                 conv1d_2[0][0]                   
__________________________________________________________________________________________________
separable_conv1d_2 (SeparableCo (None, 1024, 32)     8224        add_1[0][0]                      
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 1024, 32)     0           separable_conv1d_2[0][0]         
__________________________________________________________________________________________________
lambda_2 (Lambda)               (None, 1024, 32)     0           activation_2[0][0]               
__________________________________________________________________________________________________
spatial_dropout1d_2 (SpatialDro (None, 1024, 32)     0           lambda_2[0][0]                   
__________________________________________________________________________________________________
conv1d_3 (Conv1D)               (None, 1024, 32)     1056        spatial_dropout1d_2[0][0]        
__________________________________________________________________________________________________
add_2 (Add)                     (None, 1024, 32)     0           add_1[0][0]                      
                                                                 conv1d_3[0][0]                   
__________________________________________________________________________________________________
separable_conv1d_3 (SeparableCo (None, 1024, 32)     8224        add_2[0][0]                      
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 1024, 32)     0           separable_conv1d_3[0][0]         
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 1024, 32)     0           activation_3[0][0]               
__________________________________________________________________________________________________
spatial_dropout1d_3 (SpatialDro (None, 1024, 32)     0           lambda_3[0][0]                   
__________________________________________________________________________________________________
conv1d_4 (Conv1D)               (None, 1024, 32)     1056        spatial_dropout1d_3[0][0]        
__________________________________________________________________________________________________
add_3 (Add)                     (None, 1024, 32)     0           add_2[0][0]                      
                                                                 conv1d_4[0][0]                   
__________________________________________________________________________________________________
separable_conv1d_4 (SeparableCo (None, 1024, 32)     8224        add_3[0][0]                      
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 1024, 32)     0           separable_conv1d_4[0][0]         
__________________________________________________________________________________________________
lambda_4 (Lambda)               (None, 1024, 32)     0           activation_4[0][0]               
__________________________________________________________________________________________________
spatial_dropout1d_4 (SpatialDro (None, 1024, 32)     0           lambda_4[0][0]                   
__________________________________________________________________________________________________
conv1d_5 (Conv1D)               (None, 1024, 32)     1056        spatial_dropout1d_4[0][0]        
__________________________________________________________________________________________________
add_4 (Add)                     (None, 1024, 32)     0           add_3[0][0]                      
                                                                 conv1d_5[0][0]                   
__________________________________________________________________________________________________
separable_conv1d_5 (SeparableCo (None, 1024, 32)     8224        add_4[0][0]                      
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 1024, 32)     0           separable_conv1d_5[0][0]         
__________________________________________________________________________________________________
lambda_5 (Lambda)               (None, 1024, 32)     0           activation_5[0][0]               
__________________________________________________________________________________________________
spatial_dropout1d_5 (SpatialDro (None, 1024, 32)     0           lambda_5[0][0]                   
__________________________________________________________________________________________________
conv1d_6 (Conv1D)               (None, 1024, 32)     1056        spatial_dropout1d_5[0][0]        
__________________________________________________________________________________________________
add_5 (Add)                     (None, 1024, 32)     0           add_4[0][0]                      
                                                                 conv1d_6[0][0]                   
__________________________________________________________________________________________________
separable_conv1d_6 (SeparableCo (None, 1024, 32)     8224        add_5[0][0]                      
__________________________________________________________________________________________________
activation_6 (Activation)       (None, 1024, 32)     0           separable_conv1d_6[0][0]         
__________________________________________________________________________________________________
lambda_6 (Lambda)               (None, 1024, 32)     0           activation_6[0][0]               
__________________________________________________________________________________________________
spatial_dropout1d_6 (SpatialDro (None, 1024, 32)     0           lambda_6[0][0]                   
__________________________________________________________________________________________________
conv1d_7 (Conv1D)               (None, 1024, 32)     1056        spatial_dropout1d_6[0][0]        
__________________________________________________________________________________________________
add_6 (Add)                     (None, 1024, 32)     0           add_5[0][0]                      
                                                                 conv1d_7[0][0]                   
__________________________________________________________________________________________________
separable_conv1d_7 (SeparableCo (None, 1024, 32)     8224        add_6[0][0]                      
__________________________________________________________________________________________________
activation_7 (Activation)       (None, 1024, 32)     0           separable_conv1d_7[0][0]         
__________________________________________________________________________________________________
lambda_7 (Lambda)               (None, 1024, 32)     0           activation_7[0][0]               
__________________________________________________________________________________________________
spatial_dropout1d_7 (SpatialDro (None, 1024, 32)     0           lambda_7[0][0]                   
__________________________________________________________________________________________________
conv1d_8 (Conv1D)               (None, 1024, 32)     1056        spatial_dropout1d_7[0][0]        
__________________________________________________________________________________________________
add_7 (Add)                     (None, 1024, 32)     0           add_6[0][0]                      
                                                                 conv1d_8[0][0]                   
__________________________________________________________________________________________________
separable_conv1d_8 (SeparableCo (None, 1024, 32)     8224        add_7[0][0]                      
__________________________________________________________________________________________________
activation_8 (Activation)       (None, 1024, 32)     0           separable_conv1d_8[0][0]         
__________________________________________________________________________________________________
lambda_8 (Lambda)               (None, 1024, 32)     0           activation_8[0][0]               
__________________________________________________________________________________________________
spatial_dropout1d_8 (SpatialDro (None, 1024, 32)     0           lambda_8[0][0]                   
__________________________________________________________________________________________________
conv1d_9 (Conv1D)               (None, 1024, 32)     1056        spatial_dropout1d_8[0][0]        
__________________________________________________________________________________________________
add_8 (Add)                     (None, 1024, 32)     0           add_7[0][0]                      
                                                                 conv1d_9[0][0]                   
__________________________________________________________________________________________________
separable_conv1d_9 (SeparableCo (None, 1024, 32)     8224        add_8[0][0]                      
__________________________________________________________________________________________________
activation_9 (Activation)       (None, 1024, 32)     0           separable_conv1d_9[0][0]         
__________________________________________________________________________________________________
lambda_9 (Lambda)               (None, 1024, 32)     0           activation_9[0][0]               
__________________________________________________________________________________________________
spatial_dropout1d_9 (SpatialDro (None, 1024, 32)     0           lambda_9[0][0]                   
__________________________________________________________________________________________________
conv1d_10 (Conv1D)              (None, 1024, 32)     1056        spatial_dropout1d_9[0][0]        
__________________________________________________________________________________________________
add_9 (Add)                     (None, 1024, 32)     0           add_8[0][0]                      
                                                                 conv1d_10[0][0]                  
__________________________________________________________________________________________________
conv1d_11 (Conv1D)              (None, 1024, 32)     32800       add_9[0][0]                      
__________________________________________________________________________________________________
activation_10 (Activation)      (None, 1024, 32)     0           conv1d_11[0][0]                  
__________________________________________________________________________________________________
lambda_10 (Lambda)              (None, 1024, 32)     0           activation_10[0][0]              
__________________________________________________________________________________________________
spatial_dropout1d_10 (SpatialDr (None, 1024, 32)     0           lambda_10[0][0]                  
__________________________________________________________________________________________________
conv1d_12 (Conv1D)              (None, 1024, 32)     1056        spatial_dropout1d_10[0][0]       
__________________________________________________________________________________________________
add_10 (Add)                    (None, 1024, 32)     0           add_9[0][0]                      
                                                                 conv1d_12[0][0]                  
__________________________________________________________________________________________________
conv1d_13 (Conv1D)              (None, 1024, 32)     32800       add_10[0][0]                     
__________________________________________________________________________________________________
activation_11 (Activation)      (None, 1024, 32)     0           conv1d_13[0][0]                  
__________________________________________________________________________________________________
lambda_11 (Lambda)              (None, 1024, 32)     0           activation_11[0][0]              
__________________________________________________________________________________________________
spatial_dropout1d_11 (SpatialDr (None, 1024, 32)     0           lambda_11[0][0]                  
__________________________________________________________________________________________________
conv1d_14 (Conv1D)              (None, 1024, 32)     1056        spatial_dropout1d_11[0][0]       
__________________________________________________________________________________________________
add_11 (Add)                    (None, 1024, 32)     0           add_10[0][0]                     
                                                                 conv1d_14[0][0]                  
__________________________________________________________________________________________________
conv1d_15 (Conv1D)              (None, 1024, 32)     32800       add_11[0][0]                     
__________________________________________________________________________________________________
activation_12 (Activation)      (None, 1024, 32)     0           conv1d_15[0][0]                  
__________________________________________________________________________________________________
lambda_12 (Lambda)              (None, 1024, 32)     0           activation_12[0][0]              
__________________________________________________________________________________________________
spatial_dropout1d_12 (SpatialDr (None, 1024, 32)     0           lambda_12[0][0]                  
__________________________________________________________________________________________________
conv1d_16 (Conv1D)              (None, 1024, 32)     1056        spatial_dropout1d_12[0][0]       
__________________________________________________________________________________________________
add_12 (Add)                    (None, 1024, 32)     0           add_11[0][0]                     
                                                                 conv1d_16[0][0]                  
__________________________________________________________________________________________________
conv1d_17 (Conv1D)              (None, 1024, 32)     32800       add_12[0][0]                     
__________________________________________________________________________________________________
activation_13 (Activation)      (None, 1024, 32)     0           conv1d_17[0][0]                  
__________________________________________________________________________________________________
lambda_13 (Lambda)              (None, 1024, 32)     0           activation_13[0][0]              
__________________________________________________________________________________________________
spatial_dropout1d_13 (SpatialDr (None, 1024, 32)     0           lambda_13[0][0]                  
__________________________________________________________________________________________________
conv1d_18 (Conv1D)              (None, 1024, 32)     1056        spatial_dropout1d_13[0][0]       
__________________________________________________________________________________________________
add_13 (Add)                    (None, 1024, 32)     0           add_12[0][0]                     
                                                                 conv1d_18[0][0]                  
__________________________________________________________________________________________________
conv1d_19 (Conv1D)              (None, 1024, 32)     32800       add_13[0][0]                     
__________________________________________________________________________________________________
activation_14 (Activation)      (None, 1024, 32)     0           conv1d_19[0][0]                  
__________________________________________________________________________________________________
lambda_14 (Lambda)              (None, 1024, 32)     0           activation_14[0][0]              
__________________________________________________________________________________________________
spatial_dropout1d_14 (SpatialDr (None, 1024, 32)     0           lambda_14[0][0]                  
__________________________________________________________________________________________________
conv1d_20 (Conv1D)              (None, 1024, 32)     1056        spatial_dropout1d_14[0][0]       
__________________________________________________________________________________________________
add_14 (Add)                    (None, 1024, 32)     0           add_13[0][0]                     
                                                                 conv1d_20[0][0]                  
__________________________________________________________________________________________________
conv1d_21 (Conv1D)              (None, 1024, 32)     32800       add_14[0][0]                     
__________________________________________________________________________________________________
activation_15 (Activation)      (None, 1024, 32)     0           conv1d_21[0][0]                  
__________________________________________________________________________________________________
lambda_15 (Lambda)              (None, 1024, 32)     0           activation_15[0][0]              
__________________________________________________________________________________________________
spatial_dropout1d_15 (SpatialDr (None, 1024, 32)     0           lambda_15[0][0]                  
__________________________________________________________________________________________________
conv1d_22 (Conv1D)              (None, 1024, 32)     1056        spatial_dropout1d_15[0][0]       
__________________________________________________________________________________________________
add_15 (Add)                    (None, 1024, 32)     0           add_14[0][0]                     
                                                                 conv1d_22[0][0]                  
__________________________________________________________________________________________________
conv1d_23 (Conv1D)              (None, 1024, 32)     32800       add_15[0][0]                     
__________________________________________________________________________________________________
activation_16 (Activation)      (None, 1024, 32)     0           conv1d_23[0][0]                  
__________________________________________________________________________________________________
lambda_16 (Lambda)              (None, 1024, 32)     0           activation_16[0][0]              
__________________________________________________________________________________________________
spatial_dropout1d_16 (SpatialDr (None, 1024, 32)     0           lambda_16[0][0]                  
__________________________________________________________________________________________________
conv1d_24 (Conv1D)              (None, 1024, 32)     1056        spatial_dropout1d_16[0][0]       
__________________________________________________________________________________________________
add_16 (Add)                    (None, 1024, 32)     0           add_15[0][0]                     
                                                                 conv1d_24[0][0]                  
__________________________________________________________________________________________________
conv1d_25 (Conv1D)              (None, 1024, 32)     32800       add_16[0][0]                     
__________________________________________________________________________________________________
activation_17 (Activation)      (None, 1024, 32)     0           conv1d_25[0][0]                  
__________________________________________________________________________________________________
lambda_17 (Lambda)              (None, 1024, 32)     0           activation_17[0][0]              
__________________________________________________________________________________________________
spatial_dropout1d_17 (SpatialDr (None, 1024, 32)     0           lambda_17[0][0]                  
__________________________________________________________________________________________________
conv1d_26 (Conv1D)              (None, 1024, 32)     1056        spatial_dropout1d_17[0][0]       
__________________________________________________________________________________________________
add_17 (Add)                    (None, 1024, 32)     0           add_16[0][0]                     
                                                                 conv1d_26[0][0]                  
__________________________________________________________________________________________________
conv1d_27 (Conv1D)              (None, 1024, 32)     32800       add_17[0][0]                     
__________________________________________________________________________________________________
activation_18 (Activation)      (None, 1024, 32)     0           conv1d_27[0][0]                  
__________________________________________________________________________________________________
lambda_18 (Lambda)              (None, 1024, 32)     0           activation_18[0][0]              
__________________________________________________________________________________________________
spatial_dropout1d_18 (SpatialDr (None, 1024, 32)     0           lambda_18[0][0]                  
__________________________________________________________________________________________________
conv1d_28 (Conv1D)              (None, 1024, 32)     1056        spatial_dropout1d_18[0][0]       
__________________________________________________________________________________________________
add_18 (Add)                    (None, 1024, 32)     0           add_17[0][0]                     
                                                                 conv1d_28[0][0]                  
__________________________________________________________________________________________________
conv1d_29 (Conv1D)              (None, 1024, 32)     32800       add_18[0][0]                     
__________________________________________________________________________________________________
activation_19 (Activation)      (None, 1024, 32)     0           conv1d_29[0][0]                  
__________________________________________________________________________________________________
lambda_19 (Lambda)              (None, 1024, 32)     0           activation_19[0][0]              
__________________________________________________________________________________________________
spatial_dropout1d_19 (SpatialDr (None, 1024, 32)     0           lambda_19[0][0]                  
__________________________________________________________________________________________________
conv1d_30 (Conv1D)              (None, 1024, 32)     1056        spatial_dropout1d_19[0][0]       
__________________________________________________________________________________________________
add_20 (Add)                    (None, 1024, 32)     0           conv1d_1[0][0]                   
                                                                 conv1d_2[0][0]                   
                                                                 conv1d_3[0][0]                   
                                                                 conv1d_4[0][0]                   
                                                                 conv1d_5[0][0]                   
                                                                 conv1d_6[0][0]                   
                                                                 conv1d_7[0][0]                   
                                                                 conv1d_8[0][0]                   
                                                                 conv1d_9[0][0]                   
                                                                 conv1d_10[0][0]                  
                                                                 conv1d_12[0][0]                  
                                                                 conv1d_14[0][0]                  
                                                                 conv1d_16[0][0]                  
                                                                 conv1d_18[0][0]                  
                                                                 conv1d_20[0][0]                  
                                                                 conv1d_22[0][0]                  
                                                                 conv1d_24[0][0]                  
                                                                 conv1d_26[0][0]                  
                                                                 conv1d_28[0][0]                  
                                                                 conv1d_30[0][0]                  
__________________________________________________________________________________________________
activation_20 (Activation)      (None, 1024, 32)     0           add_20[0][0]                     
__________________________________________________________________________________________________
dense (Dense)                   (None, 1024, 2)      66          activation_20[0][0]              
__________________________________________________________________________________________________
activation_21 (Activation)      (None, 1024, 2)      0           dense[0][0]                      
==================================================================================================
Total params: 431,970
Trainable params: 431,970
Non-trainable params: 0
__________________________________________________________________________________________________
Epoch 1/1000
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training_v1.py:2048: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
Epoch 00001: val_loss improved from inf to 0.04757, saving model to res/20201211_155436_model.h5
47/47 - 11s - loss: 0.0706 - val_loss: 0.0476
Epoch 2/1000

Epoch 00002: val_loss improved from 0.04757 to 0.04254, saving model to res/20201211_155436_model.h5
47/47 - 8s - loss: 0.0366 - val_loss: 0.0425
Epoch 3/1000

Epoch 00003: val_loss improved from 0.04254 to 0.04069, saving model to res/20201211_155436_model.h5
47/47 - 8s - loss: 0.0324 - val_loss: 0.0407
Epoch 4/1000

Epoch 00004: val_loss improved from 0.04069 to 0.04036, saving model to res/20201211_155436_model.h5
47/47 - 8s - loss: 0.0336 - val_loss: 0.0404
Epoch 5/1000

Epoch 00005: val_loss did not improve from 0.04036
47/47 - 8s - loss: 0.0313 - val_loss: 0.0408
Epoch 6/1000

Epoch 00006: val_loss improved from 0.04036 to 0.04030, saving model to res/20201211_155436_model.h5
47/47 - 8s - loss: 0.0294 - val_loss: 0.0403
Epoch 7/1000

Epoch 00007: val_loss did not improve from 0.04030
47/47 - 8s - loss: 0.0307 - val_loss: 0.0406
Epoch 8/1000

Epoch 00008: val_loss did not improve from 0.04030
47/47 - 8s - loss: 0.0294 - val_loss: 0.0404
Epoch 9/1000

Epoch 00009: val_loss did not improve from 0.04030
47/47 - 8s - loss: 0.0296 - val_loss: 0.0417
Epoch 10/1000

Epoch 00010: val_loss did not improve from 0.04030
47/47 - 8s - loss: 0.0288 - val_loss: 0.0407
Epoch 11/1000

Epoch 00011: val_loss did not improve from 0.04030
47/47 - 8s - loss: 0.0282 - val_loss: 0.0418
Epoch 12/1000

Epoch 00012: val_loss did not improve from 0.04030
47/47 - 8s - loss: 0.0300 - val_loss: 0.0415
Epoch 13/1000

Epoch 00013: val_loss did not improve from 0.04030
47/47 - 8s - loss: 0.0276 - val_loss: 0.0425
Epoch 14/1000

Epoch 00014: val_loss did not improve from 0.04030
47/47 - 8s - loss: 0.0285 - val_loss: 0.0413
Epoch 15/1000

Epoch 00015: val_loss improved from 0.04030 to 0.04022, saving model to res/20201211_155436_model.h5
47/47 - 8s - loss: 0.0291 - val_loss: 0.0402
Epoch 16/1000

Epoch 00016: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0283 - val_loss: 0.0415
Epoch 17/1000

Epoch 00017: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0282 - val_loss: 0.0419
Epoch 18/1000

Epoch 00018: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0262 - val_loss: 0.0407
Epoch 19/1000

Epoch 00019: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0264 - val_loss: 0.0416
Epoch 20/1000

Epoch 00020: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0276 - val_loss: 0.0442
Epoch 21/1000

Epoch 00021: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0259 - val_loss: 0.0440
Epoch 22/1000

Epoch 00022: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0279 - val_loss: 0.0411
Epoch 23/1000

Epoch 00023: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0264 - val_loss: 0.0414
Epoch 24/1000

Epoch 00024: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0244 - val_loss: 0.0411
Epoch 25/1000

Epoch 00025: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0261 - val_loss: 0.0422
Epoch 26/1000

Epoch 00026: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0271 - val_loss: 0.0415
Epoch 27/1000

Epoch 00027: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0278 - val_loss: 0.0427
Epoch 28/1000

Epoch 00028: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0270 - val_loss: 0.0416
Epoch 29/1000

Epoch 00029: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0266 - val_loss: 0.0423
Epoch 30/1000

Epoch 00030: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0253 - val_loss: 0.0429
Epoch 31/1000

Epoch 00031: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0248 - val_loss: 0.0428
Epoch 32/1000

Epoch 00032: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0260 - val_loss: 0.0451
Epoch 33/1000

Epoch 00033: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0241 - val_loss: 0.0429
Epoch 34/1000

Epoch 00034: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0278 - val_loss: 0.0437
Epoch 35/1000

Epoch 00035: val_loss did not improve from 0.04022
47/47 - 8s - loss: 0.0240 - val_loss: 0.0425

Adjust the name to point to the results:

res_name = '/content/res/20201211_155436'
res = flammkuchen.load(f'{res_name}_results.h5')

Inspect the history of the training and validation loss

plt.plot(res['fit_hist']['loss'], label='train')
plt.plot(res['fit_hist']['val_loss'], label='val')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend()
plt.show()
../_images/colab_14_0.png

Plot the test results:

# t0, t1 = 1_020_000, 1_040_000  # dmel tutorial dataset
t0, t1 = 40_000, 60_000

plt.figure(figsize=(40, 8))
plt.subplot(311)
plt.plot(res['x_test'][t0:t1], 'k')
plt.ylabel('Audio')

plt.subplot(312)
plt.plot(res['y_test'][t0:t1, 1:])
plt.ylabel('Prediction targets')

plt.subplot(313)
plt.plot(res['y_pred'][t0:t1, 1:])
plt.ylabel('Predictions')

plt.show()
../_images/colab_16_0.png

You can download the model results via the file tab on the left, from /contest/res

Predict on new data

Load a new recording for prediction

model, params = dss.utils.load_model_and_params(res_name)  # load the model and runtime parameters
ds = dss.npy_dir.load('/content/drive/MyDrive/Dmoj.wrigleyi.npy', memmap_dirs=['train','val'])  # load the new data
print(ds)
x = ds['test']['x']
Data:
   test:
      x: (386000, 16)
      y: (386000, 2)
   val:
      y: (386000, 2)
      x: (386000, 16)
   train:
      y: (1158000, 2)
      x: (1158000, 16)

Attributes:
    class_names: ['noise', 'pulse']
    class_types: ['segment', 'event']
    filename_endsample_test: []
    filename_endsample_train: []
    filename_endsample_val: []
    filename_startsample_test: []
    filename_startsample_train: []
    filename_startsample_val: []
    filename_train: []
    filename_val: []
    samplerate_x_Hz: 10000.0
    samplerate_y_Hz: 10000.0
    filename_test: []

Run inference - this will calculate the confidence score and extract segment boundaries and event times.

events, segments, class_probabilities = dss.predict.predict(x, model=model, params=params, verbose=1, prepend_padding=True)
t0, t1 = 40_000, 60_000
plt.figure(figsize=(40, 8))
plt.plot(x[t0:t1], alpha=0.5, c='k', label='Audio')
plt.plot(class_probabilities[t0:t1, 1:], alpha=0.8, label='Prediction')
plt.xlim(0, t1-t0)
plt.legend()
plt.show()
../_images/colab_22_0.png