MCMC Step Size

01 - Exploratory

Author: Michael Gully-Santiago

Thursday, January 21, 2016

We want to know more about optimal step sizes.


In [1]:
import h5py

f = h5py.File('../data/mc.hdf5', mode='r')

list(f.keys())

In [7]:
d = f['samples']

In [10]:
d.value.shape


Out[10]:
(50000, 6)

It's a $50000 \times 6$ dataset.


In [11]:
import matplotlib.pyplot as plt
% matplotlib inline
% config InlineBackend.figure_format = 'retina'
import seaborn as sns

In [33]:
from IPython.display import Image

In [41]:
!chain.py --files ../data/mc.hdf5 --chain --burn=10000


/Users/gully/GitHub/Starfish/Starfish/__init__.py:16: UserWarning: Using the default config.yaml file located at /Users/gully/GitHub/Starfish/Starfish/config.yaml. This is likely NOT what you want. Please create a similar 'config.yaml' file in your current working directory.
  warnings.warn("Using the default config.yaml file located at {0}. This is likely NOT what you want. Please create a similar 'config.yaml' file in your current working directory.".format(default), UserWarning)
../data/mc.hdf5 burning by 10000 and thinning by 1
Using a total of 1 flatchains

In [42]:
Image(filename='walkers.png')


Out[42]:

In [30]:
!chain.py --files ../data/mc.hdf5 --triangle --burn=10000


/Users/gully/GitHub/Starfish/Starfish/__init__.py:16: UserWarning: Using the default config.yaml file located at /Users/gully/GitHub/Starfish/Starfish/config.yaml. This is likely NOT what you want. Please create a similar 'config.yaml' file in your current working directory.
  warnings.warn("Using the default config.yaml file located at {0}. This is likely NOT what you want. Please create a similar 'config.yaml' file in your current working directory.".format(default), UserWarning)
../data/mc.hdf5 burning by 10000 and thinning by 1
Using a total of 1 flatchains
WARNING:root:Deprecation Warning: 'triangle' has been renamed to 'corner'. This shim should continue to work but you should use 'import corner' in new code. https://github.com/dfm/corner.py
/anaconda/lib/python3.4/site-packages/matplotlib/collections.py:650: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  if self._edgecolors_original != str('face'):
/anaconda/lib/python3.4/site-packages/matplotlib/collections.py:590: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  if self._edgecolors == str('face'):

In [43]:
Image(filename='triangle.png')


Out[43]:

Not bad! There is little constraint on $\log{g}, Z$, but we sort of expected that.


In [49]:
!chain.py --files ../data/mc.hdf5 --cov


/Users/gully/GitHub/Starfish/Starfish/__init__.py:16: UserWarning: Using the default config.yaml file located at /Users/gully/GitHub/Starfish/Starfish/config.yaml. This is likely NOT what you want. Please create a similar 'config.yaml' file in your current working directory.
  warnings.warn("Using the default config.yaml file located at {0}. This is likely NOT what you want. Please create a similar 'config.yaml' file in your current working directory.".format(default), UserWarning)
../data/mc.hdf5 burning by 0 and thinning by 1
Using a total of 1 flatchains
Correlation coefficient
[[ 1.         -0.08529985  0.34770029  0.24251558 -0.25777186  0.32638421]
 [-0.08529985  1.         -0.09842929  0.10580519 -0.26001077  0.11548361]
 [ 0.34770029 -0.09842929  1.          0.55571884 -0.06748688  0.0138828 ]
 [ 0.24251558  0.10580519  0.55571884  1.         -0.17789596  0.08313011]
 [-0.25777186 -0.26001077 -0.06748688 -0.17789596  1.         -0.20955271]
 [ 0.32638421  0.11548361  0.0138828   0.08313011 -0.20955271  1.        ]]
'Optimal' jumps with covariance (units squared)
[[  5.50937103e+03  -4.90227089e-01   2.50570251e+00   1.71496805e+01
   -1.95884759e+01   2.79686873e-02]
 [ -4.90227089e-01   5.99509529e-03  -7.39938880e-04   7.80495506e-03
   -2.06112113e-02   1.03231044e-05]
 [  2.50570251e+00  -7.39938880e-04   9.42642234e-03   5.14036482e-02
   -6.70821433e-03   1.55611790e-06]
 [  1.71496805e+01   7.80495506e-03   5.14036482e-02   9.07675771e-01
   -1.73518419e-01   9.14356777e-05]
 [ -1.95884759e+01  -2.06112113e-02  -6.70821433e-03  -1.73518419e-01
    1.04816162e+00  -2.47684802e-04]
 [  2.79686873e-02   1.03231044e-05   1.55611790e-06   9.14356777e-05
   -2.47684802e-04   1.33285799e-06]]
Standard deviation
[  7.63923163e+01   7.96886947e-02   9.99245302e-02   9.80537121e-01
   1.05368985e+00   1.18820290e-03]
'Optimal' jumps
[  7.42251374e+01   7.74280007e-02   9.70897643e-02   9.52720196e-01
   1.02379765e+00   1.15449469e-03]

In [51]:
Image(filename='cor_coefficient.png')


Out[51]:

In [52]:
import numpy as np

In [54]:
ojs = np.load('opt_jump.npy')

In [55]:
ojs.shape


Out[55]:
(6, 6)

The end.