In [1]:
import numpy, scipy, matplotlib.pyplot as plt, pandas, librosa
The basic representation of an audio signal is in the time domain.
Sound is air vibrating. An audio signal represents the fluctuation in air pressure caused by the vibration as a function of time.
Let's download and listen to a file:
In [2]:
import urllib
urllib.urlretrieve('http://audio.musicinformationretrieval.com/c_strum.wav')
x, fs = librosa.load('c_strum.wav', sr=44100)
from IPython.display import Audio
Audio(x, rate=fs)
Out[2]:
To plot a signal in the time domain, use librosa.display.waveplot:
In [3]:
librosa.display.waveplot(x, fs, alpha=0.5)
Out[3]:
Let's zoom in:
In [4]:
plt.plot(x[8000:9000])
Out[4]:
Digital computers can only capture this data at discrete moments in time. The rate at which a computer captures audio data is called the sampling frequency (abbreviated fs) or sample rate (abbreviated sr). For this workshop, we will mostly work with a sampling frequency of 44100 Hz.
The Fourier Transform is one of the most fundamental operations in applied mathematics and signal processing.
It transforms our time-domain signal into the frequency domain. Whereas the time domain expresses our signal as a sequence of samples, the frequency domain expresses our signal as a superposition of sinusoids of varying magnitudes, frequencies, and phase offsets.
To compute a Fourier transform in NumPy or SciPy, use scipy.fft:
In [5]:
X = scipy.fft(x)
X_mag = numpy.absolute(X)
plt.plot(X_mag) # magnitude spectrum
Out[5]:
Zoom in:
In [6]:
plt.plot(X_mag[:5000])
Out[6]:
In Essentia, you can also use essentia.standard.Spectrum to compute a magnitude spectrum.
Musical signals are highly non-stationary, i.e., their statistics change over time. It would be rather meaningless to compute a spectrum over an entire 10-minute song.
Instead, we compute a spectrum for small frames of the audio signal. The resulting sequence of spectra is called a spectrogram.
Matplotlib has specgram which computes and displays a spectrogram:
In [7]:
S, freqs, bins, im = plt.specgram(x, NFFT=1024, noverlap=512, Fs=44100)
plt.xlabel('Time (seconds)')
plt.ylabel('Frequency (Hz)')
Out[7]:
librosa has some outstanding spectral representations, including librosa.feature.melspectrogram:
In [8]:
S = librosa.feature.melspectrogram(x, sr=fs, n_fft=1024)
logS = librosa.logamplitude(S)
To display any type of spectrogram in librosa, use librosa.display.specshow:
In [9]:
librosa.display.specshow(logS, sr=fs, x_axis='time', y_axis='mel')
Out[9]:
The autocorrelation of a signal describes the similarity of a signal against a delayed version of itself.
Using numpy.correlate:
In [10]:
# Because the autocorrelation produces a symmetric signal, we only care about the "right half".
r = numpy.correlate(x, x, mode='full')[len(x)-1:]
print x.shape, r.shape
plt.plot(r[:10000])
plt.xlabel('Lag (samples)')
Out[10]:
Using librosa.autocorrelate:
In [11]:
r = librosa.autocorrelate(x, max_size=10000)
plt.plot(r)
plt.xlabel('Lag (samples)')
Out[11]:
In [12]:
from essentia.standard import AutoCorrelation
autocorr = AutoCorrelation()
r = autocorr(x)
plt.plot(r[:10000])
plt.xlabel('Lag (samples)')
Out[12]:
Unlike the Fourier transform, the constant-Q transform uses a logarithmically spaced frequency axis.
To plot a constant-Q spectrogram, will use librosa.cqt:
In [13]:
fmin = librosa.midi_to_hz(36)
C = librosa.cqt(x, sr=fs, fmin=fmin, n_bins=60)
logC = librosa.logamplitude(C)
librosa.display.specshow(logC, sr=fs, x_axis='time', y_axis='cqt_note', fmin=fmin, cmap='coolwarm')
Out[13]: