Setup imports


In [1]:
%matplotlib inline

In [2]:
import numpy as np
import matplotlib.pyplot as plt
from sys import path
path.append('..')
from oasis.functions import gen_data, gen_sinusoidal_data, deconvolve, estimate_parameters
from oasis.plotting import simpleaxis
from oasis.oasis_methods import oasisAR1, oasisAR2

In [3]:
def plot_trace(groundtruth=False):
    plt.figure(figsize=(20,4))
    plt.subplot(211)
    plt.plot(b+c, lw=2, label='denoised')
    if groundtruth:
        plt.plot(true_b+true_c, c='r', label='truth', zorder=-11)
    plt.plot(y, label='data', zorder=-12, c='y')
    plt.legend(ncol=3, frameon=False, loc=(.02,.85))
    simpleaxis(plt.gca())
    plt.subplot(212)
    plt.plot(s, lw=2, label='deconvolved', c='g')
    if groundtruth:
        for k in np.where(true_s)[0]:
            plt.plot([k,k],[-.1,1], c='r', zorder=-11, clip_on=False)
    plt.ylim(0,1.3)
    plt.legend(ncol=3, frameon=False, loc=(.02,.85));
    simpleaxis(plt.gca())
    print("Correlation of deconvolved activity  with ground truth ('spikes') : %.4f" % np.corrcoef(s,true_s)[0,1])
    print("Correlation of denoised fluorescence with ground truth ('calcium'): %.4f" % np.corrcoef(c,true_c)[0,1])

Load raw fluorescence data

This data happend to have a fast rise time and we model it with an AR(1) process, see below for slower rise time and AR(2).


In [4]:
# here we generate some simulated fluorescence data and plot it
true_b = 2
y, true_c, true_s = map(np.squeeze, gen_data(N=1, b=true_b, seed=0))
plt.figure(figsize=(20,4))
plt.plot(y, c='y');


Deconvolve

With $\ell_1$ penalty we obtain the global minimum of the convex problem

If we only have the trace and no further info, simply calling deconvolve tries to estimate it from the data


In [5]:
%time c, s, b, g, lam = deconvolve(y, penalty=1)


CPU times: user 1.78 ms, sys: 2.67 ms, total: 4.44 ms
Wall time: 3.64 ms

Plot results

Because we happen to have ground truth data, we show it too


In [6]:
plot_trace(True)


Correlation of deconvolved activity  with ground truth ('spikes') : 0.8937
Correlation of denoised fluorescence with ground truth ('calcium'): 0.9794

With $\ell_0$ penalty the problem is non-convex, however, we obtain a good local minimum


In [7]:
%time c, s, b, g, lam = deconvolve(y, penalty=0)
plot_trace(True)


CPU times: user 5.49 ms, sys: 45 µs, total: 5.54 ms
Wall time: 4.72 ms
Correlation of deconvolved activity  with ground truth ('spikes') : 0.9233
Correlation of denoised fluorescence with ground truth ('calcium'): 0.9830

If we have a good idea about the fluorescence baseline, its time constant and it size for 1AP, we can provide this information

$g$ is related to the decay time $\tau_d$ of the exponetial Ca response kernel $e^{-t/\tau_d}$ as $g=e^{-\frac{1}{\tau_d r}}$ with decay time $\tau_d$ in seconds and imaging rate r in Hz.


In [8]:
# Here we provide the ground truth values for b and g. 
# The Ca response kernel to 1 AP has maximal amplitude 1 and we pick s_min slightly larger than 1/2.
%time c, s = oasisAR1(y-true_b, g=.95, s_min=.55)
plot_trace(True)


CPU times: user 731 µs, sys: 53 µs, total: 784 µs
Wall time: 767 µs
Correlation of deconvolved activity  with ground truth ('spikes') : 0.9120
Correlation of denoised fluorescence with ground truth ('calcium'): 0.9894

If $g$ is not provided, it is estimated from the autocorrelation. We can improve upon it, in particularly if the spiking signal has some significant autocorrelation.

In function deconvolve, we already multiply the autocorrelation estimate by some fudge_factor that is close to but smaller than 1, which increases robustness


In [9]:
# no optimization of g
y, true_c, true_s = map(np.squeeze, gen_sinusoidal_data(N=1, b=true_b, seed=0))
%time c, s, b, g, lam = deconvolve(y, penalty=0)
# the line below uses the direct autocorrelation estimate without fudge_factor, yielding worse results
# %time c, s, b, g, lam = deconvolve(y, g=estimate_parameters(y, 1)[0], penalty=0) 
plot_trace(True)


CPU times: user 3.07 ms, sys: 2.01 ms, total: 5.07 ms
Wall time: 5.08 ms
Correlation of deconvolved activity  with ground truth ('spikes') : 0.8224
Correlation of denoised fluorescence with ground truth ('calcium'): 0.9923

In [10]:
# optimization of g. optimize_g=5 uses 5 large isolated calcium events to update g.
%time c, s, b, g, lam = deconvolve(y, penalty=0, optimize_g=5)
plot_trace(True)


CPU times: user 4.74 ms, sys: 1.99 ms, total: 6.72 ms
Wall time: 5.68 ms
Correlation of deconvolved activity  with ground truth ('spikes') : 0.8640
Correlation of denoised fluorescence with ground truth ('calcium'): 0.9933

In [ ]:


In [ ]:

Load some other raw fluorescence data

This data happend to have a slow rise time and we model it with an AR(2) process


In [11]:
# here we generate some simulated fluorescence data and plot it
y, true_c, true_s = map(np.squeeze, gen_data([1.7,-.712], sn=.5, N=1, b=true_b, seed=0))
plt.figure(figsize=(20,4))
plt.plot(y, c='y');


Deconvolve

With $\ell_1$ penalty we obtain the global minimum of the convex problem

If we only have the trace and no further info, simply calling deconvolve tries to estimate it from the data


In [12]:
# g=(None,None) because we want to use an AR(2) model, but don't know the parameters
%time c, s, b, g, lam = deconvolve(y, g=(None,None), penalty=1) 
plot_trace(True)


CPU times: user 428 ms, sys: 23 ms, total: 451 ms
Wall time: 43.1 ms
Correlation of deconvolved activity  with ground truth ('spikes') : 0.6566
Correlation of denoised fluorescence with ground truth ('calcium'): 0.9944

With $\ell_0$ penalty the problem is non-convex, however, we obtain a good local minimum


In [13]:
%time c, s, b, g, lam = deconvolve(y, g=(None,None), penalty=0)
plot_trace(True)


CPU times: user 313 ms, sys: 10.3 ms, total: 323 ms
Wall time: 30.4 ms
Correlation of deconvolved activity  with ground truth ('spikes') : 0.6459
Correlation of denoised fluorescence with ground truth ('calcium'): 0.9934

We might further improve upon the initial estimate of $g$


In [14]:
# this is currently slow, we hope to still improve speed as well as robustness
%time c, s, b, g, lam = deconvolve(y, g=(None,None), penalty=0, optimize_g=5, max_iter=5) 
plot_trace(True)


/mnt/home/jfriedrich/OASIS/oasis/functions.py:167: UserWarning: Optimization of AR parameters is already fairly stable for AR(1), but slower and more experimental for AR(2)
  warn("Optimization of AR parameters is already fairly stable for AR(1), "
CPU times: user 8.36 s, sys: 230 ms, total: 8.59 s
Wall time: 726 ms
Correlation of deconvolved activity  with ground truth ('spikes') : 0.7665
Correlation of denoised fluorescence with ground truth ('calcium'): 0.9969

If we have a good idea about the fluorescence baseline, and its time constants we can provide this information

$g=(g_1,g_2)$ is related to the decay time $\tau_d$ and rise time $\tau_r$ (in seconds) of the Ca response kernel $e^{-t/\tau_d}-e^{-t/\tau_r}$ as
$g_1=e^{-\frac{1}{\tau_d r}}+e^{-\frac{1}{\tau_r r}}$ and
$g_2=-e^{-\frac{1}{\tau_d r}}\cdot e^{-\frac{1}{\tau_r r}}$ with imaging rate r in Hz.


In [15]:
# Here we provide the ground truth values for g. 
%time c, s, b, g, lam = deconvolve(y, (1.7,-.712))
plot_trace(True)


CPU times: user 350 ms, sys: 18.9 ms, total: 369 ms
Wall time: 30.7 ms
Correlation of deconvolved activity  with ground truth ('spikes') : 0.7765
Correlation of denoised fluorescence with ground truth ('calcium'): 0.9968

We can also opt for a quick but less accurate greedy method


In [16]:
# Here we provide the ground truth values for b and g and we pick s_min slightly larger than 1/2.
%time c, s = oasisAR2(y-true_b, 1.7,-.712, s_min=.55)
plot_trace(True)


CPU times: user 4.2 ms, sys: 9 µs, total: 4.21 ms
Wall time: 3.79 ms
Correlation of deconvolved activity  with ground truth ('spikes') : 0.7199
Correlation of denoised fluorescence with ground truth ('calcium'): 0.9973

In [ ]: