Multi-spots paper data analysis

This notebook performs data analysis for the paper:

Multi-spot single-molecule FRET: high-throughput analysis of freely diffusing molecules
Ingargiola et al. PLOS ONE (2017) doi:10.1371/journal.pone.0175766

Here you will find the dsDNA experiements analysis. For the realtime kinetics analysis see the notebook:

Introduction

You will find below the list of notebooks executed during the analysis. The text explains the purpose of each notebook. After each run_notebook command, there are links to access the output notebooks (i.e. the executed version which contains all the results).

You will also find links to the eventual CSV files saved.

Quick Figure's Reference

Jump to the section generating a particular figure:

Figure 2. FRET efficiency for 5 dsDNA samples estimated using different methods.

Figure 3. DCR-corrected total count rate as a function of the pixel.

Figure 6. Leakage coefficient for different samples and spots.

Figure 5. Burst statistics versus spot.

Figure 8. µs-ALEX vs multispot burst data distributions.

Figure 7. FRET efficiency versus dye separation for the multispot experiments.

Execution Details


This section gives more details on how the execution is organized. If you only want to explore the results, just skip the next section.

This is "master" notebook that sequentially executes all the other notebooks. By running this notebook you can reproduce all the results presented in the paper.

We save intermediate results in text CSV files that can be easily tracked by git. The files are shown as tables or links after each notebook is executed.

Notebooks with parameters

Some of the μs-ALEX notebooks are executed several times, once for each of the 5 dsDNA measurement. These are called template notebooks. When a template notebook is executed a set of output notebooks is generated, one for each sample. After the execution, the cell will show links to the template notebook (unexecuted, contains no output) and to the output notebooks for each sample. These output notebooks are meant to be read-only and are used to inspect the full analysis results for each sample.

After the links, we show a table containing summary data saved when executing the notebook (a row for each sample and a column for each variable). To find where a specific variable is computed, you can search the variable name in the output notebooks (one for each sample).

Notebook with no parameters

Notebooks not taking any parameter are execute only once. The output cell shows only two links, one for the template notebook and one for the executed notebook.

Initial setup


In [1]:
from notebook_runner import run_notebook, run_notebook_template

Here we keep track of all the libraries versions used for the execution.


In [2]:
%load_ext version_information

In [3]:
%version_information numpy, scipy, matplotlib, tables, pandas, lmfit, seaborn, fretbursts


 - Optimized (cython) burst search loaded.
 - Optimized (cython) photon counting loaded.
--------------------------------------------------------------
 You are running FRETBursts (version 0.5.9).

 If you use this software please cite the following paper:

   FRETBursts: An Open Source Toolkit for Analysis of Freely-Diffusing Single-Molecule FRET
   Ingargiola et al. (2016). http://dx.doi.org/10.1371/journal.pone.0160716 

--------------------------------------------------------------
Out[3]:
SoftwareVersion
Python3.5.2 64bit [GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)]
IPython5.1.0
OSDarwin 14.5.0 x86_64 i386 64bit
numpy1.11.2
scipy0.18.1
matplotlib1.5.3
tables3.2.3.1
pandas0.19.0
lmfit0.9.5
seaborn0.7.1
fretbursts0.5.9
Mon Mar 27 11:34:11 2017 PDT

μs-ALEX Analysis

Raw Proximity Ratio

We start the analyis by processing and fiting the raw PR values for the 5 us-ALEX measurements. The only burst correction here applied is the background correction.

The same analysis is performed for different burst searches, as indicated in the notebook file name suffix:

  • all-ph: burst search on all photons
  • Dex: burst search on photons during Donor excitation periods
  • DexDem: burst search on photons during Donor excitation detected in the Donor channel (donor emission)
  • AND-gate: AND-gate burst search. The bursts are the intersection (in time) of burst from a "Dex" burst search and burst from a "AexAem" burst search.

In [4]:
nb_name = 'usALEX-5samples-PR-raw'
run_notebook_template(nb_name, ph_sel='all-ph')


Source Notebook: usALEX-5samples-PR-raw.ipynb
n_bursts_all n_bursts_do n_bursts_fret E_kde_w E_gauss_w E_gauss_w_sig E_gauss_w_err E_gauss_w_fiterr S_kde S_gauss S_gauss_sig S_gauss_err S_gauss_fiterr E_pr_do_kde E_pr_do_hsm E_pr_do_gauss nt_mean
sample
7d 1371 726 599 0.9340 0.9315 0.0545 0.0022 0.0016 0.5582 0.5616 0.0981 0.0040 0.0024 0.0978 0.0893 0.0998 22.2931
12d 1531 411 1083 0.7612 0.7498 0.0859 0.0026 0.0023 0.5848 0.5709 0.1038 0.0032 0.0035 0.1012 0.1193 0.0988 22.6536
17d 3018 560 2392 0.4914 0.4850 0.1019 0.0021 0.0018 0.5358 0.5584 0.1103 0.0023 0.0018 0.1008 0.0983 0.1031 23.1577
22d 2552 403 2045 0.2766 0.2799 0.0708 0.0016 0.0016 0.5628 0.5642 0.1129 0.0025 0.0019 0.0922 0.0886 0.0911 25.3736
27d 998 204 757 0.1958 0.1951 0.0586 0.0021 0.0010 0.5540 0.5834 0.1121 0.0041 0.0029 0.0820 0.0964 0.0887 19.5704

In [5]:
nb_name = 'usALEX-5samples-PR-raw'
run_notebook_template(nb_name, ph_sel='DexDem')


Source Notebook: usALEX-5samples-PR-raw.ipynb
n_bursts_all n_bursts_do n_bursts_fret E_kde_w E_gauss_w E_gauss_w_sig E_gauss_w_err E_gauss_w_fiterr S_kde S_gauss S_gauss_sig S_gauss_err S_gauss_fiterr E_pr_do_kde E_pr_do_hsm E_pr_do_gauss nt_mean
sample
7d 1686 1396 151 0.8430 0.7954 0.0898 0.0073 0.0078 0.6360 0.6018 0.0932 0.0076 0.0060 0.0954 0.0838 0.0921 28.1482
12d 1915 722 1092 0.7140 0.7068 0.0756 0.0023 0.0026 0.5592 0.5872 0.1020 0.0031 0.0033 0.0916 0.0870 0.0912 27.9703
17d 4501 1045 3276 0.4634 0.4477 0.1023 0.0018 0.0026 0.5702 0.5740 0.1094 0.0019 0.0018 0.0950 0.0956 0.0950 29.1528
22d 3456 580 2750 0.2642 0.2656 0.0671 0.0013 0.0013 0.5634 0.5707 0.1096 0.0021 0.0018 0.0810 0.0846 0.0887 31.9127
27d 1397 401 943 0.1810 0.1832 0.0540 0.0018 0.0007 0.6176 0.6015 0.1081 0.0035 0.0030 0.0798 0.0569 0.0819 23.2971

In [6]:
nb_name = 'usALEX-5samples-PR-raw'
run_notebook_template(nb_name, ph_sel='Dex')


Source Notebook: usALEX-5samples-PR-raw.ipynb
n_bursts_all n_bursts_do n_bursts_fret E_kde_w E_gauss_w E_gauss_w_sig E_gauss_w_err E_gauss_w_fiterr S_kde S_gauss S_gauss_sig S_gauss_err S_gauss_fiterr E_pr_do_kde E_pr_do_hsm E_pr_do_gauss nt_mean
sample
7d 1654 956 641 0.9338 0.9278 0.0539 0.0021 0.0024 0.5750 0.5768 0.1009 0.0040 0.0024 0.1000 0.0900 0.1041 23.0420
12d 1692 486 1157 0.7508 0.7489 0.0839 0.0025 0.0015 0.5974 0.5885 0.1045 0.0031 0.0026 0.0998 0.0981 0.1021 23.5048
17d 3459 766 2588 0.4906 0.4807 0.1018 0.0020 0.0021 0.5740 0.5835 0.1084 0.0021 0.0022 0.1054 0.1200 0.1089 23.0631
22d 2798 501 2181 0.2752 0.2804 0.0679 0.0015 0.0018 0.5768 0.5834 0.1102 0.0024 0.0022 0.0962 0.0742 0.1047 26.6318
27d 1146 316 792 0.1944 0.1947 0.0572 0.0020 0.0010 0.5786 0.6054 0.1087 0.0039 0.0034 0.0904 0.0916 0.0940 19.8821

In [7]:
nb_name = 'usALEX-5samples-PR-raw-AND-gate'
run_notebook_template(nb_name)


n_bursts_all n_bursts_fret E_kde_w E_gauss_w E_gauss_w_sig E_gauss_w_err E_gauss_w_fiterr S_kde S_gauss S_gauss_sig S_gauss_err S_gauss_fiterr nt_mean
sample
7d 667 611 0.9358 0.9296 0.0533 0.0022 0.0021 0.5766 0.5709 0.0981 0.0040 0.0022 23.1869
12d 1130 1104 0.7524 0.7509 0.0810 0.0024 0.0016 0.5920 0.5828 0.1010 0.0030 0.0023 23.4090
17d 2479 2455 0.4912 0.4824 0.0992 0.0020 0.0016 0.5742 0.5759 0.1030 0.0021 0.0017 23.0325
22d 2197 2110 0.2746 0.2796 0.0686 0.0015 0.0020 0.5698 0.5779 0.1085 0.0024 0.0022 26.5659
27d 781 760 0.1940 0.1936 0.0584 0.0021 0.0013 0.5808 0.5998 0.1063 0.0039 0.0027 19.6871

μs-ALEX: Leakage coefficient

Here we estimate the donor leakage coefficient from the Donor-only peak positions in the Raw PR histograms.

The computation is performed by the following notebook:


In [8]:
run_notebook('usALEX - Corrections - Leakage fit')




μs-ALEX: Direct excitation

In this section we extract the direct excitation parameter by fitting the S peak position of the Acceptor-only population.

For comparison 2 burst searches are performed:

  • all-ph: burst search on all photons
  • AexAem: burst search on photons during Acceptor excitation detected by the Acceptor detector (acceptor emission).

The all-ph burst search provides less accurate results and is performed only to show the improvement of the more appropriate AexAem burst search.

Fitting of the S peak


In [9]:
nb_name = 'usALEX-5samples-PR-raw-dir_ex_aa-fit'
run_notebook_template(nb_name, ph_sel='all-ph')


n_bursts_aa dir_ex_S1p dir_ex_S_kde dir_ex_S2p dir_ex_S2pa dir_ex_S2p_w1 dir_ex_S_kde_w1 dir_ex_S_kde_w4 dir_ex_S_kde_w5 dir_ex_S2p_w5 dir_ex_S2p_w5a S_2peaks_w5 S_2peaks_w5_fiterr
sample
7d 393 0.0552 0.0679 0.0716 0.0481 0.0640 0.0679 0.0625 0.0613 0.0551 0.0468 0.0522 0.0012
12d 169 0.0485 0.0730 0.0988 0.0468 0.0920 0.0727 0.0695 0.0677 0.0763 0.0422 0.0709 0.0040
17d 371 0.0488 0.0844 0.1066 0.0603 0.0947 0.0839 0.0725 0.0652 0.0725 0.0463 0.0676 0.0028
22d 331 0.0478 0.0785 0.0900 0.0586 0.0832 0.0778 0.0704 0.0645 0.0736 0.0511 0.0686 0.0028
27d 61 0.0355 0.0404 0.1212 0.0207 0.0755 0.0404 0.0454 0.0326 0.0356 0.0034 0.0344 0.0043

In [10]:
nb_name = 'usALEX-5samples-PR-raw-dir_ex_aa-fit'
run_notebook_template(nb_name, ph_sel='AexAem')


n_bursts_aa dir_ex_S1p dir_ex_S_kde dir_ex_S2p dir_ex_S2pa dir_ex_S2p_w1 dir_ex_S_kde_w1 dir_ex_S_kde_w4 dir_ex_S_kde_w5 dir_ex_S2p_w5 dir_ex_S2p_w5a S_2peaks_w5 S_2peaks_w5_fiterr
sample
7d 630 0.0350 0.0611 0.0687 0.0605 0.0659 0.0609 0.0551 0.0537 0.0554 0.0500 0.0525 0.0023
12d 269 0.0371 0.0604 0.0925 0.0417 0.0736 0.0604 0.0571 0.0549 0.0576 0.0449 0.0544 0.0033
17d 608 0.0347 0.0834 0.1195 0.0704 0.0950 0.0827 0.0774 0.0714 0.0729 0.1744 0.0679 0.0031
22d 433 0.0265 0.0865 0.1111 0.0538 0.0826 0.0855 0.0716 0.0643 0.0619 0.0469 0.0583 0.0030
27d 129 0.0198 0.0509 0.0787 0.0406 0.0802 0.0504 0.0748 0.0434 0.0555 0.0171 0.0526 0.0040

Direct excitation fit

The notebook following notebook computes the direct excitation coefficient performing an average of the values extracted from each single measurement:


In [11]:
run_notebook('usALEX - Corrections - Direct excitation fit')




μs-ALEX: γ-factor

Here we compute the $\gamma$-factor for the μs-ALEX measurements.

First we reprocess the μs-ALEX measurements including background, leakage and direct excitation correction (standard PR). Then, for the fitted $E$ and $S$ peak position for the FRET population we fit the $\gamma$-factor.

Proximity Ratio analysis


In [12]:
nb_name = 'usALEX-5samples-PR-leakage-dir-ex-all-ph'
run_notebook_template(nb_name)


n_bursts_all n_bursts_do n_bursts_fret E_kde_w E_gauss_w E_gauss_w_sig E_gauss_w_err S_kde S_gauss S_gauss_sig S_gauss_err E_pr_do_kde nt_mean
sample
7d 1172 586 543 0.9304 0.9258 0.0595 0.0026 0.5512 0.5506 0.0995 0.0043 0.0020 22.2278
12d 1307 329 948 0.7440 0.7315 0.0919 0.0030 0.5748 0.5575 0.1051 0.0034 0.0160 22.0024
17d 2489 463 1964 0.4320 0.4271 0.1137 0.0026 0.5434 0.5371 0.1129 0.0025 0.0128 21.1932
22d 2054 319 1673 0.1804 0.1842 0.0790 0.0019 0.5414 0.5429 0.1150 0.0028 0.0000 23.0165
27d 791 160 590 0.0844 0.0854 0.0701 0.0029 0.5160 0.5561 0.1152 0.0047 -0.0088 16.8301

γ-factor fit

The notebook usALEX - Corrections - Gamma factor fit fits the γ-factor from $E$ and $S$ values computed in the previous section.


In [13]:
run_notebook('usALEX - Corrections - Gamma factor fit')




μs-ALEX: Corrected E

Using all the correction coefficients here we computed the corrected $E$ histograms:


In [14]:
nb_name = 'usALEX-5samples-E-corrected-all-ph'
run_notebook_template(nb_name)


n_bursts_all n_bursts_do n_bursts_fret E_kde_w E_gauss_w E_gauss_w_sig E_gauss_w_err E_gauss_w_fiterr S_kde S_gauss S_gauss_sig S_gauss_err S_gauss_fiterr E_pr_do_kde nt_mean
sample
7d 1172 587 542 0.9290 0.9243 0.0601 0.0026 0.0018 0.5520 0.5507 0.0992 0.0043 0.0025 0.0022 22.2278
12d 1307 329 948 0.7398 0.7270 0.0921 0.0030 0.0020 0.5758 0.5587 0.1054 0.0034 0.0038 0.0154 22.0024
17d 2489 464 1964 0.4268 0.4226 0.1130 0.0025 0.0020 0.5462 0.5396 0.1125 0.0025 0.0022 0.0124 21.1932
22d 2054 320 1672 0.1774 0.1811 0.0772 0.0019 0.0015 0.5458 0.5470 0.1150 0.0028 0.0027 0.0000 23.0165
27d 791 160 589 0.0826 0.0840 0.0693 0.0029 0.0016 0.5208 0.5613 0.1145 0.0047 0.0037 -0.0088 16.8301

Direct excitation: physical parameters

Here we compute the "physical" direct direct excitation coefficient:

$$d_{T} = \frac{\sigma_{D_{ex}}^A}{\sigma_{D_{ex}}^D} $$

This coefficient depends only on the fluorophores absorption cross-section at donor excitation wavelength. This coefficient can also be used to correct non-ALEX measurements.

Additionally, the next notebook summarizes the estimated FRET efficiencies obtained with different methods and corrections. A table with fit errors and number of bursts is also included.


In [15]:
run_notebook('usALEX - Corrections - Direct excitation physical parameter')




The previous notebook saves:

The last two files contain, respectively, the corrected E values fitted with various methods (Gauss, KDE, SNA, etc.) and the fit errors.

μs-ALEX: Corrected E figure

The following notebook uses the data usALEX-5samples-E-all-methods.csv to create the final figure for the paper (for details and fit errors see the previous notebook usALEX - Corrections - Direct excitation physical parameter):


In [16]:
run_notebook('Figure - usALEX 5 dsDNA samples FRET')




Multi-spot Analysis

DCR and Background

Plot background fits:


In [17]:
run_notebook('Multi-spot 5-Samples analysis - Background')




Compute detectors DCR:


In [18]:
run_notebook('8-pixel DCR')


Source Notebook: 8-pixel DCR.ipynb

The previous notebook saves:

Signal vs spot


In [19]:
run_notebook('Multi-spot 5-Samples analysis - Signal vs spot')




Figure 2. DCR-corrected total count rate as a function of the pixel.

Leakage coefficient

First step of the analysis of the multi-spot data is fitting the leakage coefficient. The next notebook fits the D-only peak:


In [20]:
run_notebook('Multi-spot 5-Samples analysis - Leakage coefficient fit')




This saves the following tables for leakage coefficients and number of bursts:

Next, we take sample-wise or channel-wise averages and plot summaries:


In [21]:
run_notebook('Multi-spot 5-Samples analysis - Leakage coefficient - Summary')




Figure 5. Leakage coefficient for different samples and spots.

Figure provenance

All data comes from the numerical tables linked above.

Burst statistics vs spot


In [22]:
run_notebook('Multi-spot 5-Samples analysis - Burst Statistics')




Figure 4. Burst statistics versus spot.

Figure provenance

File name (recording processing parameters):

  • 8spot 5-dsDNA samples burst-stats vs ch F=6, R=25kcps lk=3.3 na=[-12,12] DexDem nt1=50 w1=1.0ms p1=310kcps.png

Burst statistics vs usALEX


In [23]:
run_notebook('dsDNA - Burst Statistics - MultiSpot-vs-SingleSpot')




Figure 7. µs-ALEX vs multispot burst data distributions.

PR and FRET analysis

Then we perform the complete burst analysis by running:


In [24]:
run_notebook('Multi-spot 5-Samples analysis - PR-raw')




The previous notebook saves the tables:

Gamma fitting for the multispot is perfomed here:


In [25]:
run_notebook('Multi-spot Gamma Fitting')




FRET vs distance figure

This notebooks created the FRET vs distance figure:


In [26]:
run_notebook('Figure - FRET vs distance')




Figure 6. FRET efficiency versus dye separation for the multispot experiments.

Figure provenance

Data:

Multispot vs usALEX FRET histograms

NOTE: Due to different concentrations and background levels, the comparison of FRET histograms is only qualitative. In particular, even though the duration of the multispot measurements are exactly 1/8 the duration of the μs-ALEX measurements, the number of bursts cannot be directly compared due to different burst rates.