Explore the Forest

Welcome to a forest in Provence, France. In these IPython notebooks you'll explore classifying the birds around you from audio clips of their songs and calls. The code is available on Github

The data used in this activity were provided during the NIPS4B workshop in Nevada 2013, in the Bird challenge, organized by Bas, Dufour and Glotin. They shall not be used for another purpose. Publication or report using them shall cite:

Bas Y., Dufour O., Glotin H., (2013) 'NIP4B Bird Challenge Overview', in Proc. of 'Neural Information Processing Scaled for Bioacoustics: from Neurons to Big Data- NIP4B', Ed. Glotin et al., joint to NIPS Conf., ISSN 979-10-90821-04-0, http://sabiod.org



In [1]:

    
import pandas as pd
import numpy as np

import bird_data as data

import wav_utils

from IPython.core.display import HTML

# Plot in the notebook
%pylab inline









    



Populating the interactive namespace from numpy and matplotlib






    



/media/extra/kaggle/bird-recognition/venv/local/lib/python2.7/site-packages/pandas/io/excel.py:626: UserWarning: Installed openpyxl is not supported at this time. Use >=1.6.1 and <2.0.0.
  .format(openpyxl_compat.start_ver, openpyxl_compat.stop_ver))

Explore the data

Which calls will we try to identify?

This information is in the call dataframe. Each call has a number, which we'll often use in place of its name



In [2]:

    
data.call_df.head(10)









    Out[2]:






  
    
      
      class number
      class name
      English_name
      Scientific_name
      type
    
  
  
    
      0
       none
             Empty
                        Empty
                noise sample
       noise
    
    
      1
          1
       Aegcau_call
              Long-tailed Tit
         Aegithalos caudatus
        bird
    
    
      2
          2
       Alaarv_song
             Eurasian Skylark
             Alauda arvensis
        bird
    
    
      3
          3
       Anttri_song
                   Tree Pipit
            Anthus trivialis
        bird
    
    
      4
          4
       Butbut_call
               Common Buzzard
                 Buteo buteo
        bird
    
    
      5
          5
       Carcan_call
                Common Linnet
           Linaria cannabina
        bird
    
    
      6
          6
       Carcan_song
                Common Linnet
           Linaria cannabina
        bird
    
    
      7
          7
       Carcar_call
           European Goldfinch
         Carduelis carduelis
        bird
    
    
      8
          8
       Carcar_song
           European Goldfinch
         Carduelis carduelis
        bird
    
    
      9
          9
       Cerbra_call
       Short-toed Treecreeper
       Certhia brachydactyla
        bird

How do we know the calls in a particular training audio clip?

This is in the label dataframe. Each row is a particular training audio clip, and each column is a particular label. A 1 means the call is present, 0 means it is not



In [3]:

    
HTML(data.label_df.head(10).to_html())









    Out[3]:





  
    
      call_ids
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      53
      54
      55
      56
      57
      58
      59
      60
      61
      62
      63
      64
      65
      66
      67
      68
      69
      70
      71
      72
      73
      74
      75
      76
      77
      78
      79
      80
      81
      82
      83
      84
      85
      86
      87
    
    
      filename
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
    
  
  
    
      nips4b_birds_trainfile001.wav
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
    
    
      nips4b_birds_trainfile002.wav
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
    
    
      nips4b_birds_trainfile003.wav
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
    
    
      nips4b_birds_trainfile004.wav
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
    
    
      nips4b_birds_trainfile005.wav
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
    
    
      nips4b_birds_trainfile006.wav
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
    
    
      nips4b_birds_trainfile007.wav
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
    
    
      nips4b_birds_trainfile008.wav
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
    
    
      nips4b_birds_trainfile009.wav
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
    
    
      nips4b_birds_trainfile010.wav
       0
       0
       0
       0
       0
       1
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0
       0

Exercises

Which calls are not made by birds?
What is a filename for an audio clip containing an insect?



In [4]:

    
data.call_df.type.unique()









    Out[4]:





array(['noise', 'bird', 'insect', 'amphibian'], dtype=object)



In [4]:

What about the sound clips?



In [5]:

    
wavs = data.get_wav_dict()

Find all the clips with three calls



In [6]:

    
three_call_clipnames = data.label_df.index[data.label_df.sum(axis=1)==3]

Plot the audio for one of these three-bird clips



In [7]:

    
plot(wavs[three_call_clipnames[1]])









    Out[7]:





[<matplotlib.lines.Line2D at 0x47bd390>]

That wasn't too helpful, maybe a spectrogram will be more illuminating



In [8]:

    
wav_utils.view_clip(three_call_clipnames[4])









    












    Out[8]:






  
    
      
      English_name
      type
    
    
      call_ids
      
      
    
  
  
    
      17
        Carrion Crow
       bird
    
    
      45
            Blue Tit
       bird
    
    
      49
       House Sparrow
       bird

Much better, how about some more clips

Sounds (and looks) like there may be a stream in the background on this one



In [9]:

    
wav_utils.view_clip(three_call_clipnames[1])









    












    Out[9]:






  
    
      
      English_name
      type
    
    
      call_ids
      
      
    
  
  
    
      22
         European Robin
       bird
    
    
      61
                Dunnock
       bird
    
    
      84
       Common Blackbird
       bird

Is that a car in the background?



In [10]:

    
wav_utils.view_clip(data.label_df.index[32])









    












    Out[10]:





    
    
    

    
    
    
    
  
    
      
      English_name
      type
    
    
      call_ids
      
      
    
  
  
    
      29
       Thekla Lark
       bird

Uhhhhh, anyone there?



In [11]:

    
wav_utils.view_clip(data.label_df.index[8])









    












    Out[11]:






  
    
      
      English_name
      type
    
    
      call_ids



In [12]:

    
wav_utils.view_clip(data.label_df.index[31])









    












    Out[12]:






  
    
      
      English_name
      type
    
    
      call_ids

That's an uncommonly interesting call, Firecrest



In [13]:

    
wav_utils.view_clip(data.label_df.index[34])









    












    Out[13]:






  
    
      
      English_name
      type
    
    
      call_ids
      
      
    
  
  
    
      65
       Common Firecrest
       bird

Exercises

Look at the clip you found that contains an insect
Find a clip you expect to be relatively easy to label correctly
Find a clip you expect to be challenging to label correctly
Look at the clips with 6 labels each
How many examples are there of each call?

Bonus Exercises

Find some particularly noisy clips
Try to syllablize the wavs or spectrograms
Where does the syllablization break down?
Can we synthesize new labeled data by combining clips or exploiting invariances?



In [13]:

	class number	class name	English_name	Scientific_name	type
0	none	Empty	Empty	noise sample	noise
1	1	Aegcau_call	Long-tailed Tit	Aegithalos caudatus	bird
2	2	Alaarv_song	Eurasian Skylark	Alauda arvensis	bird
3	3	Anttri_song	Tree Pipit	Anthus trivialis	bird
4	4	Butbut_call	Common Buzzard	Buteo buteo	bird
5	5	Carcan_call	Common Linnet	Linaria cannabina	bird
6	6	Carcan_song	Common Linnet	Linaria cannabina	bird
7	7	Carcar_call	European Goldfinch	Carduelis carduelis	bird
8	8	Carcar_song	European Goldfinch	Carduelis carduelis	bird
9	9	Cerbra_call	Short-toed Treecreeper	Certhia brachydactyla	bird

call_ids	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46	47	48	49	50	51	52	53	54	55	56	57	58	59	60	61	62	63	64	65	66	67	68	69	70	71	72	73	74	75	76	77	78	79	80	81	82	83	84	85	86	87
filename
nips4b_birds_trainfile001.wav	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
nips4b_birds_trainfile002.wav	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0
nips4b_birds_trainfile003.wav	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0
nips4b_birds_trainfile004.wav	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0
nips4b_birds_trainfile005.wav	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0
nips4b_birds_trainfile006.wav	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0
nips4b_birds_trainfile007.wav	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0
nips4b_birds_trainfile008.wav	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
nips4b_birds_trainfile009.wav	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
nips4b_birds_trainfile010.wav	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0

	English_name	type
call_ids
22	European Robin	bird
61	Dunnock	bird
84	Common Blackbird	bird