XPCS&XSVS Pipeline for Setup of (Gi)-SAXS

"This notebook corresponds to version {{ version }} of the pipeline tool: https://github.com/NSLS-II/pipelines"

This notebook begins with a raw time-series of images and ends with $g_2(t)$ for a range of $q$, fit to an exponential or stretched exponential, and a two-time correlation functoin.

Overview

Setup: load packages/setup path
Load Metadata & Image Data
Apply Mask
Compress Data
Define Q-ROI, e.g. qr for SAXS, (qr, qz) for gi-saxs

The important scientific code is imported from the chxanalys and scikit-beam project. Refer to chxanalys and scikit-beam for additional documentation and citation information.

CHX Olog NoteBook

CHX Olog (https://logbook.nsls2.bnl.gov/11-ID/)

Load Packages

Import packages for I/O, visualization, and analysis.



In [1]:

    
from pyCHX.chx_packages import *
%matplotlib notebook
plt.rcParams.update({'figure.max_open_warning': 0})
plt.rcParams.update({ 'image.origin': 'lower'   })
plt.rcParams.update({ 'image.interpolation': 'none'   })
import pickle as cpk
from pyCHX.chx_xpcs_xsvs_jupyter_V1 import *



In [2]:

    
%matplotlib notebook



In [ ]:



In [7]:

    
cd sample_data









    



[Errno 2] No such file or directory: 'sample_data'
/CENTRAL/xf11id1/analysis/2018_2/commisioning/Compress_Work/sample_data



In [4]:

    
%run ~/Repos/chx_xpcs/chx_xpcs/io/multifile/multifile.py



In [13]:

    
fp = 'sample_compress.bin'



In [23]:

    
f = MultifileBNL(fp )









    



Indexing file...
Done. Took 0.030787944793701172 secs for 1000 frames



In [37]:

    
%%timeit
f.rdframe(0)
#f.rdframe(1000  )
f.rdframe(1000 -1 )
f.rdframe(500 -1 )









    



14.1 ms ± 2.7 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)



In [35]:

    
from pyCHX.chx_packages import *



In [44]:

    
FD = Multifile(fp )









    



----------------------------------------------------------------
NameError                      Traceback (most recent call last)
<ipython-input-44-f7b6447b92ad> in <module>()
----> 1 FD = MultifileL(fp )

NameError: name 'MultifileL' is not defined



In [36]:

    
%%timeit
FD.rdframe(0)
#f.rdframe(1000  )
FD.rdframe(1000 -1 )
FD.rdframe(500 -1 )









    



2.55 s ± 199 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)



In [ ]:



In [ ]:



In [41]:

    
fp = '/home/yuzhang/analysis/Compressed_Data/uid_fefc3fac-af61-4248-be62-04cdfb231688.cmp'



In [42]:

    
f = MultifileBNL(fp )









    



Indexing file...
Done. Took 77.2925763130188 secs for 7200 frames



In [43]:

    
%%timeit
f.rdframe(0)
#f.rdframe(1000  )
f.rdframe(7200 -1 )
f.rdframe(3600 -1 )









    



11.3 ms ± 518 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)



In [46]:

    
FD = Multifile(fp, 0, 7200 )



In [47]:

    
%%timeit
FD.rdframe(0)
#f.rdframe(1000  )
FD.rdframe(7200 -1 )
FD.rdframe(3600 -1 )









    



3.24 s ± 35.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)



In [48]:

    
3240//11









    Out[48]:





294



In [49]:

    
FD.rdframe(1000)









    Out[49]:





array([[0., 1., 1., ..., 0., 1., 0.],
       [2., 1., 0., ..., 2., 1., 0.],
       [1., 2., 2., ..., 1., 2., 0.],
       ...,
       [1., 1., 1., ..., 1., 1., 1.],
       [1., 1., 1., ..., 1., 1., 1.],
       [0., 2., 0., ..., 1., 2., 0.]])



In [54]:

    
#(f.rdframe(7000).T - FD.rdframe(7000)).sum()

For compress data



In [55]:

    
uid = 'fefc3f'



In [56]:

    
sud = get_sid_filenames(db[uid])









    



filepath : /XF11ID/data/2018/02/15/016f0386-b6e2-4c0a-800d
got images_per_file



In [59]:

    
fp = sud[2][0]
print(fp)









    



/XF11ID/data/2018/02/15/016f0386-b6e2-4c0a-800d_2701_master.h5



In [63]:

    
!ls /XF11ID/data/2018/02/15/016f0386-b6e2-4c0a-800d_2701* |wc









    



     73      73    4959



In [65]:

    
#!ls /XF11ID/data/2018/02/15/016f0386-b6e2-4c0a-800d_2701*



In [66]:

    
pwd









    Out[66]:





'/CENTRAL/xf11id1/analysis/2018_2/commisioning/Compress_Work/sample_data'



In [ ]:



In [71]:

    
%run ~/Repos/chx_xpcs/chx_xpcs/io/eiger/compress_file.py



In [72]:

    
%%time 
compress_file(fp, dset_min=1, dset_max=72, dset_root="/entry/data",
        dset_pref="data_", outfile="uid=%s.bin"%uid)









    



reading dataset 1
reading dataset 2
reading dataset 3
reading dataset 4
reading dataset 5
reading dataset 6
reading dataset 7
reading dataset 8
reading dataset 9
reading dataset 10
reading dataset 11
reading dataset 12
reading dataset 13
reading dataset 14
reading dataset 15
reading dataset 16
reading dataset 17
reading dataset 18
reading dataset 19
reading dataset 20
reading dataset 21
reading dataset 22
reading dataset 23
reading dataset 24
reading dataset 25
reading dataset 26
reading dataset 27
reading dataset 28
reading dataset 29
reading dataset 30
reading dataset 31
reading dataset 32
reading dataset 33
reading dataset 34
reading dataset 35
reading dataset 36
reading dataset 37
reading dataset 38
reading dataset 39
reading dataset 40
reading dataset 41
reading dataset 42
reading dataset 43
reading dataset 44
reading dataset 45
reading dataset 46
reading dataset 47
reading dataset 48
reading dataset 49
reading dataset 50
reading dataset 51
reading dataset 52
reading dataset 53
reading dataset 54
reading dataset 55
reading dataset 56
reading dataset 57
reading dataset 58
reading dataset 59
reading dataset 60






    



----------------------------------------------------------------
OSError                        Traceback (most recent call last)
<timed eval> in <module>()

~/Repos/chx_xpcs/chx_xpcs/io/eiger/compress_file.py in compress_file(filename, dset_min, dset_max, dset_root, dset_pref, outfile)
     89             key = dset_root + "/" + dset_pref + "{:06d}".format(i)
     90             dset = f[key]
---> 91             dset.read_direct(arr, np.s_[j,:,:])
     92 
     93             #test = np.array(f[key][j])

~/chx_compress/lib/python3.6/site-packages/h5py/_hl/dataset.py in read_direct(self, dest, source_sel, dest_sel)
    655 
    656             for mspace in dest_sel.broadcast(source_sel.mshape):
--> 657                 self.id.read(mspace, fspace, dest, dxpl=self._dxpl)
    658 
    659     def write_direct(self, source, source_sel=None, dest_sel=None):

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5d.pyx in h5py.h5d.DatasetID.read()

h5py/_proxy.pyx in h5py._proxy.dset_rw()

h5py/_proxy.pyx in h5py._proxy.H5PY_H5Dread()

OSError: Can't read data (file read failed: time = Fri Jun  8 12:04:32 2018
, filename = '/XF11ID/data/2018/02/15/016f0386-b6e2-4c0a-800d_2701_data_000060.h5', file descriptor = 56, errno = 116, error message = 'Stale file handle', buf = 0x564fbfcb1d08, total read size = 1048576, bytes this sub-read = 1048576, bytes actually read = 18446744073709551615, offset = 242384336)



In [ ]:



In [ ]: