

In [ ]:

    
import numpy, scipy, matplotlib.pyplot as plt, sklearn, librosa, mir_eval, IPython.display, urllib

Unsupervised Instrument Classification Using K-Means

This lab is loosely based on Lab 3 (2010).

Read Audio

Retrieve an audio file, load it into an array, and listen to it.



In [ ]:

    
urllib.urlretrieve?



In [ ]:

    
librosa.load?



In [ ]:

    
IPython.display.Audio?

Detect Onsets

Detect onsets in the audio signal:



In [ ]:

    
librosa.onset.onset_detect?

Convert the onsets from units of frames to seconds (and samples):



In [ ]:

    
librosa.frames_to_time?



In [ ]:

    
librosa.frames_to_samples?

Listen to detected onsets:



In [ ]:

    
mir_eval.sonify.clicks?



In [ ]:

    
IPython.display.Audio?

Extract Features

Extract a set of features from the audio at each onset. Use any of the features we have learned so far: zero crossing rate, spectral moments, MFCCs, chroma, etc. For more, see the librosa API reference.

First, define which features to extract:



In [ ]:

    
def extract_features(x, fs):
    feature_1 = librosa.zero_crossings(x).sum() # placeholder
    feature_2 = 0 # placeholder
    return [feature_1, feature_2]

For each onset, extract a feature vector from the signal:



In [ ]:

    
# Assumptions:
# x: input audio signal
# fs: sampling frequency
# onset_samples: onsets in units of samples
frame_sz = fs*0.100
features = numpy.array([extract_features(x[i:i+frame_sz], fs) for i in onset_samples])

Scale Features

Use sklearn.preprocessing.MinMaxScaler to scale your features to be within [-1, 1].



In [ ]:

    
sklearn.preprocessing.MinMaxScaler?



In [ ]:

    
sklearn.preprocessing.MinMaxScaler.fit_transform?

Plot Features

Use scatter to plot features on a 2-D plane. (Choose two features at a time.)



In [ ]:

    
plt.scatter?

Cluster Using K-Means

Use KMeans to cluster your features and compute labels.



In [ ]:

    
sklearn.cluster.KMeans?



In [ ]:

    
sklearn.cluster.KMeans.fit_predict?

Plot Features by Class Label

Use scatter, but this time choose a different marker color (or type) for each class.



In [ ]:

    
plt.scatter?

Listen to Click Track

Create a beep for each onset within a class:



In [ ]:

    
beeps = mir_eval.sonify.clicks(onset_times[labels==0], fs, length=len(x))



In [ ]:

    
IPython.display.Audio?

Listen to Clustered Frames

Use the concatenated_segments function from the feature sonification exercise to concatenate frames from the same cluster into one signal. Then listen to the signal.



In [ ]:

    
def concatenate_segments(segments, fs=44100, pad_time=0.300):
    padded_segments = [numpy.concatenate([segment, numpy.zeros(int(pad_time*fs))]) for segment in segments]
    return numpy.concatenate(padded_segments)
concatenated_signal = concatenate_segments(segments, fs)

Compare across separate classes. What do you hear?

For Further Exploration

Use a different number of clusters in KMeans.

Use a different initialization method in KMeans.

Use different features. Compare tonal features against timbral features.



In [ ]:

    
librosa.feature?

Use different audio files.



In [ ]:

    
#filename = '1_bar_funk_groove.mp3'
#filename = '58bpm.wav'
#filename = '125_bounce.wav'
#filename = 'prelude_cmaj_10s.wav'

← Back to Index