MCMC from scratch

Here we will write a simple python program that will perform the Metropolis algorithm. In order to sample the posterior of the probability function given supernovae data.

Loglike computation

First we need to be able to compute the likelihood of the data given some parameters. We will actually compute the logarithm of the likelihood as it is easier to handle for the computer.

First let's import some basic libraries:


In [1]:
import numpy as np
import scipy.integrate as integrate

Now let's use a scipy integrator in order to obtain the luminosity distance


In [11]:
def E(z,OmDE):
    """
    This function computes the integrand for the computation of the luminosity distance for a flat universe
    z -> float
    OmDE -> float
    gives
    E -> float
    """
    return 1/np.sqrt((1-OmDE)*(1+z)**3+OmDE)

def dl(z,OmDE,h=0.7):
    """
    This function computes the luminosity distance
    z -> float
    OmDE -> float
    h ->float
    returns
    dl -> float
    """
    inte=integrate.quad(E,0,z,args=(OmDE))
    # Velocidad del sonido en km/s
    c = 299792.458
    # Factor de Hubble
    Ho = 100*h
    return c*(1+z)/Ho * inte[0]

We now have to load the supernovae data in order to compare it with the theoretical data


In [3]:
zandmu = np.loadtxt('../data/SCPUnion2.1_mu_vs_z.txt', skiprows=5,usecols=(1,2))
covariance = np.loadtxt('../data/SCPUnion2.1_covmat_sys.txt')

In [4]:
type(zandmu)


Out[4]:
numpy.ndarray

In [5]:
np.shape(zandmu[1])


Out[5]:
(2,)

How does this data look like?


In [2]:
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sb

The supernovae


In [7]:
plt.plot(zandmu[:,0],zandmu[:,1],'o')


Out[7]:
[<matplotlib.lines.Line2D at 0x7f9541f3c4d0>]

In [8]:
heat = np.log(abs(covariance))

In [9]:
sb.set()
sb.heatmap(heat)


Out[9]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f9541dbae90>

In [12]:
yerr = np.loadtxt('../data/SCPUnion2.1_mu_vs_z.txt', skiprows=5,usecols=[3])
plt.errorbar(zandmu[:,0],zandmu[:,1], yerr=yerr, fmt='o')


Out[12]:
<Container object of 3 artists>

Now we can compute the log like.


In [14]:
dl = np.vectorize(dl)
def loglike(params,h=0.7):
    """
    This function computes the logarithm of the likelihood. It recieves a vector
    params-> vector with one component (Omega Dark Energy)
    """
    OmDE = params[0]
# Ahora quiero calcular la diferencia entre el valor reportado y el calculado
    delta = 5.*np.log10(dl(zandmu[:,0],OmDE,h))+25-zandmu[:,1]
    chisquare=np.dot(delta,np.dot(np.linalg.inv(covariance),delta))
    return -chisquare/2

In [23]:
table_omega = np.arange(0.,1.,0.01)
tableprob=[loglike([hola]) for hola in table_omega]

In [24]:
plt.plot(table_omega,tableprob)


Out[24]:
[<matplotlib.lines.Line2D at 0x7f9534978410>]

In [15]:
loglike([0.6])


Out[15]:
-278.58971768190293

All for today!

You can try jla data

From http://supernovae.in2p3.fr/sdss_snls_jla/ReadMe.html download jla_likelihood_v6.tgz and covmat_v6.tgz.

  • The supernovae info is contained in the file jla_lcparams.txt go that file and remove the "#" from the first line

  • Load it using genfromtxt


In [3]:
jla_dataset = np.genfromtxt('../data/jla/data/jla_lcparams.txt', names=True, dtype=None)

This function is very clever and you can use the names of the variables instead of the number of the corresponding column, like that:


In [4]:
jla_dataset['zcmb']


Out[4]:
array([ 0.503084,  0.580724,  0.494795,  0.345928,  0.677662,  0.610712,
        0.866494,  0.330932,  0.798566,  0.449562,  0.371443,  0.291718,
        0.35582 ,  0.461274,  0.28361 ,  0.632234,  0.466414,  0.268624,
        0.94689 ,  0.924917,  0.693165,  0.625233,  0.896944,  0.608261,
        0.789054,  0.578292,  0.59028 ,  0.719626,  0.210032,  0.76659 ,
        0.858513,  0.367904,  0.558746,  0.848509,  0.99639 ,  0.776594,
        0.582932,  0.583743,  0.796552,  0.588721,  0.913473,  0.768587,
        0.948455,  0.513793,  0.733609,  0.637701,  0.765588,  0.434861,
        0.983407,  0.583743,  0.700631,  0.662058,  0.838126,  0.621874,
        0.370586,  0.416632,  0.35857 ,  0.451681,  0.522762,  0.733996,
        0.701965,  0.742008,  0.746118,  0.34956 ,  0.591835,  0.514746,
        0.620632,  0.643646,  1.002795,  0.470586,  0.610643,  0.263487,
        0.358336,  0.742697,  0.451556,  0.910752,  0.552605,  0.337534,
        0.750681,  0.983777,  0.822697,  0.813696,  0.950763,  0.817732,
        0.34052 ,  0.960751,  0.81072 ,  0.756687,  0.611252,  0.877971,
        0.548325,  0.809032,  1.0288  ,  0.627239,  0.422452,  0.9339  ,
        0.701158,  0.697164,  0.685179,  0.767075,  0.931901,  0.863978,
        0.7491  ,  0.514353,  0.468406,  0.470398,  0.958871,  0.927909,
        0.840522,  0.630699,  0.562751,  0.615698,  0.828528,  0.564741,
        0.578745,  0.557743,  0.864497,  0.735608,  0.858507,  0.261984,
        0.585743,  0.76158 ,  0.4888  ,  0.858516,  0.688656,  0.558743,
        0.324532,  0.480704,  0.185371,  0.922221,  0.681942,  0.475707,
        0.92221 ,  0.893183,  0.428646,  0.631882,  0.69996 ,  0.736007,
        0.575823,  0.418633,  0.511746,  0.535771,  0.641893,  0.734996,
        0.35156 ,  0.60986 ,  0.349557,  0.701969,  0.355261,  0.98328 ,
        0.872152,  0.75903 ,  0.92622 ,  0.643642,  0.419569,  0.515587,
        0.890729,  0.805696,  0.480568,  0.800697,  0.580604,  0.715652,
        0.805695,  0.766684,  0.664638,  0.901736,  0.745665,  0.71868 ,
        0.73669 ,  0.5796  ,  0.370531,  0.96074 ,  0.850729,  0.64864 ,
        0.219462,  0.936755,  0.647639,  0.461564,  0.600603,  0.670654,
        0.760698,  0.246492,  0.470573,  0.169455,  0.497379,  0.638234,
        0.507371,  0.535327,  0.587285,  0.773081,  0.699166,  0.370511,
        0.761089,  0.699169,  0.78806 ,  0.373513,  0.405478,  0.852994,
        0.808048,  0.603272,  0.58329 ,  0.534335,  0.72014 ,  0.400485,
        0.837004,  0.371508,  0.806041,  0.840005,  0.500728,  0.532765,
        0.533766,  0.932229,  0.822103,  0.553797,  0.125298,  0.842123,
        0.443668,  0.727675,  0.683659,  0.442548,  0.726679,  0.28251 ,
        0.404549,  0.519611,  0.690652,  1.060801,  0.576612,  0.268489,
        0.72068 ,  0.250498,  0.760691,  0.698167,  0.550315,  0.730128,
        0.847985,  0.997826,  0.435446,  0.40947 ,  0.30159 ,  1.299106,
        1.020807,  0.839734,  1.12085 ,  1.230892,  0.97423 ,  0.970782,
        0.064229,  0.12903 ,  0.10289 ,  0.04413 ,  0.11304 ,  0.25611 ,
        0.38026 ,  0.14547 ,  0.08889 ,  0.26059 ,  0.07891 ,  0.08304 ,
        0.16565 ,  0.16817 ,  0.17063 ,  0.17028 ,  0.12044 ,  0.27653 ,
        0.2316  ,  0.30569 ,  0.07997 ,  0.17415 ,  0.13868 ,  0.14607 ,
        0.08133 ,  0.2338  ,  0.25655 ,  0.24562 ,  0.12593 ,  0.22351 ,
        0.10264 ,  0.11967 ,  0.18055 ,  0.19709 ,  0.22916 ,  0.26451 ,
        0.10337 ,  0.36441 ,  0.2056  ,  0.17981 ,  0.19959 ,  0.15653 ,
        0.1618  ,  0.20953 ,  0.24179 ,  0.23646 ,  0.22668 ,  0.29678 ,
        0.20852 ,  0.25104 ,  0.25012 ,  0.11771 ,  0.21971 ,  0.14901 ,
        0.28742 ,  0.1436  ,  0.32837 ,  0.24625 ,  0.11966 ,  0.32838 ,
        0.21513 ,  0.14478 ,  0.13243 ,  0.21082 ,  0.20399 ,  0.27952 ,
        0.04428 ,  0.18037 ,  0.2607  ,  0.05654 ,  0.2126  ,  0.36941 ,
        0.17386 ,  0.14265 ,  0.17789 ,  0.32837 ,  0.25351 ,  0.30001 ,
        0.10589 ,  0.22328 ,  0.11845 ,  0.17653 ,  0.1828  ,  0.29762 ,
        0.19699 ,  0.15295 ,  0.2485  ,  0.13818 ,  0.20704 ,  0.21468 ,
        0.30632 ,  0.24695 ,  0.19801 ,  0.22256 ,  0.13508 ,  0.21054 ,
        0.27258 ,  0.2466  ,  0.22068 ,  0.27447 ,  0.17656 ,  0.31088 ,
        0.18435 ,  0.15935 ,  0.21976 ,  0.25205 ,  0.18122 ,  0.17373 ,
        0.38047 ,  0.17865 ,  0.24568 ,  0.2091  ,  0.24947 ,  0.13392 ,
        0.17706 ,  0.28111 ,  0.17383 ,  0.19654 ,  0.36397 ,  0.37835 ,
        0.31646 ,  0.18322 ,  0.24102 ,  0.20203 ,  0.16869 ,  0.19794 ,
        0.12276 ,  0.20318 ,  0.12739 ,  0.27549 ,  0.14468 ,  0.33436 ,
        0.1958  ,  0.19392 ,  0.24973 ,  0.1486  ,  0.15596 ,  0.09582 ,
        0.15065 ,  0.30933 ,  0.24857 ,  0.36554 ,  0.11771 ,  0.19176 ,
        0.18255 ,  0.23472 ,  0.24647 ,  0.28845 ,  0.2096  ,  0.19198 ,
        0.11402 ,  0.11389 ,  0.1988  ,  0.22058 ,  0.18257 ,  0.15895 ,
        0.07849 ,  0.13864 ,  0.18079 ,  0.17757 ,  0.17066 ,  0.23381 ,
        0.07153 ,  0.15463 ,  0.0901  ,  0.17658 ,  0.13045 ,  0.18204 ,
        0.25617 ,  0.13811 ,  0.1707  ,  0.21674 ,  0.24555 ,  0.14382 ,
        0.24862 ,  0.14533 ,  0.13588 ,  0.05874 ,  0.03652 ,  0.27749 ,
        0.20515 ,  0.28051 ,  0.19867 ,  0.15999 ,  0.21075 ,  0.07195 ,
        0.23782 ,  0.14139 ,  0.15506 ,  0.37013 ,  0.09391 ,  0.11863 ,
        0.15363 ,  0.25356 ,  0.10725 ,  0.12961 ,  0.26665 ,  0.21226 ,
        0.27822 ,  0.2454  ,  0.13362 ,  0.17656 ,  0.11368 ,  0.32038 ,
        0.11271 ,  0.10602 ,  0.39135 ,  0.15575 ,  0.18667 ,  0.19671 ,
        0.19191 ,  0.19084 ,  0.17625 ,  0.12837 ,  0.12262 ,  0.12715 ,
        0.21323 ,  0.21861 ,  0.26856 ,  0.20466 ,  0.2671  ,  0.21757 ,
        0.24185 ,  0.2935  ,  0.35346 ,  0.18563 ,  0.28543 ,  0.19494 ,
        0.07592 ,  0.16357 ,  0.25721 ,  0.21955 ,  0.17558 ,  0.22667 ,
        0.15322 ,  0.21025 ,  0.12164 ,  0.29135 ,  0.29448 ,  0.16455 ,
        0.30702 ,  0.19863 ,  0.26821 ,  0.40128 ,  0.13668 ,  0.29542 ,
        0.29192 ,  0.09007 ,  0.20557 ,  0.14582 ,  0.11767 ,  0.05499 ,
        0.1742  ,  0.24495 ,  0.22452 ,  0.24558 ,  0.28351 ,  0.18056 ,
        0.10391 ,  0.12961 ,  0.24352 ,  0.33136 ,  0.24348 ,  0.31468 ,
        0.21865 ,  0.26014 ,  0.28243 ,  0.30041 ,  0.15186 ,  0.26357 ,
        0.21382 ,  0.16284 ,  0.1244  ,  0.19465 ,  0.20554 ,  0.10667 ,
        0.18772 ,  0.16387 ,  0.29021 ,  0.09401 ,  0.22786 ,  0.10751 ,
        0.31065 ,  0.14564 ,  0.08784 ,  0.14839 ,  0.28662 ,  0.12281 ,
        0.18655 ,  0.12893 ,  0.19422 ,  0.148   ,  0.21179 ,  0.17962 ,
        0.26349 ,  0.19215 ,  0.3388  ,  0.11741 ,  0.1431  ,  0.16035 ,
        0.28872 ,  0.12303 ,  0.26405 ,  0.12608 ,  0.17374 ,  0.1644  ,
        0.24961 ,  0.15999 ,  0.24448 ,  0.24851 ,  0.22893 ,  0.08528 ,
        0.062818,  0.27745 ,  0.27544 ,  0.33051 ,  0.36193 ,  0.33103 ,
        0.1605  ,  0.30021 ,  0.11635 ,  0.21829 ,  0.11967 ,  0.15473 ,
        0.1781  ,  0.25037 ,  0.25174 ,  0.12953 ,  0.30929 ,  0.27852 ,
        0.18581 ,  0.06651 ,  0.258   ,  0.29888 ,  0.27044 ,  0.27916 ,
        0.29287 ,  0.18965 ,  0.26576 ,  0.12376 ,  0.18255 ,  0.05634 ,
        0.31288 ,  0.30915 ,  0.20061 ,  0.3264  ,  0.21163 ,  0.17969 ,
        0.3024  ,  0.10924 ,  0.08522 ,  0.20231 ,  0.12649 ,  0.21457 ,
        0.32045 ,  0.21835 ,  0.1901  ,  0.09243 ,  0.37986 ,  0.21134 ,
        0.25833 ,  0.18327 ,  0.21295 ,  0.11651 ,  0.14401 ,  0.25249 ,
        0.25539 ,  0.21698 ,  1.265901,  0.935774,  0.049519,  0.030325,
        0.074579,  0.025198,  0.014411,  0.020792,  0.035218,  0.041657,
        0.0424  ,  0.018902,  0.07884 ,  0.063345,  0.027637,  0.049325,
        0.070153,  0.023909,  0.053126,  0.024951,  0.015664,  0.048984,
        0.022399,  0.034854,  0.016966,  0.030721,  0.030482,  0.01055 ,
        0.014256,  0.027688,  0.05386 ,  0.017672,  0.01745 ,  0.015573,
        0.01006 ,  0.054931,  0.039083,  0.078317,  0.031764,  0.01444 ,
        0.013533,  0.026677,  0.02336 ,  0.036934,  0.023552,  0.017483,
        0.040259,  0.030381,  0.015504,  0.014151,  0.015487,  0.015554,
        0.037075,  0.015482,  0.013324,  0.014575,  0.015791,  0.024746,
        0.027748,  0.010888,  0.013652,  0.035755,  0.024725,  0.038086,
        0.021541,  0.019581,  0.030416,  0.039219,  0.034971,  0.028561,
        0.020997,  0.032538,  0.015469,  0.015678,  0.027499,  0.046856,
        0.033173,  0.010291,  0.080103,  0.012317,  0.024586,  0.014467,
        0.015154,  0.029086,  0.034781,  0.04492 ,  0.034074,  0.07487 ,
        0.01496 ,  0.020396,  0.040689,  0.026567,  0.026881,  0.023471,
        0.069122,  0.065193,  0.02298 ,  0.017387,  0.031485,  0.011184,
        0.021956,  0.031484,  0.023321,  0.02039 ,  0.022466,  0.033614,
        0.036969,  0.022629,  0.0147  ,  0.03695 ,  0.060959,  0.059289,
        0.069751,  0.029526,  0.033255,  0.024159,  0.015999,  0.062961,
        0.021706,  0.031638,  0.018599,  0.027064,  0.025468,  0.02381 ,
        0.023867,  0.022068])

You can ask which are the different columns or info in that file:


In [5]:
jla_dataset.dtype.names


Out[5]:
('name',
 'zcmb',
 'zhel',
 'dz',
 'mb',
 'dmb',
 'x1',
 'dx1',
 'color',
 'dcolor',
 '3rdvar',
 'd3rdvar',
 'cov_m_s',
 'cov_m_c',
 'cov_s_c',
 'set',
 'ra',
 'dec',
 'biascor')

In [7]:
plt.plot(jla_dataset['zcmb'],jla_dataset['mb'],'o')


Out[7]:
[<matplotlib.lines.Line2D at 0x7f5ae8c2c9d0>]

From http://supernovae.in2p3.fr/sdss_snls_jla/ReadMe.html download covmat_v6.tgz and use the example.py to compute the covariance matrix for a particular alpha, beta.

  • Copy the contents of example.py here
  • Change the names of the files according to your own file order
  • Change all the "numpy" for "np"

In [8]:
import pyfits
import glob
def mu_cov(alpha, beta):
    """ Assemble the full covariance matrix of distance modulus

    See Betoule et al. (2014), Eq. 11-13 for reference
    """
    Ceta = sum([pyfits.getdata(mat) for mat in glob.glob('../data/jla/covmat/C*.fits')])

    Cmu = np.zeros_like(Ceta[::3,::3])
    for i, coef1 in enumerate([1., alpha, -beta]):
        for j, coef2 in enumerate([1., alpha, -beta]):
            Cmu += (coef1 * coef2) * Ceta[i::3,j::3]

    # Add diagonal term from Eq. 13
    sigma = np.loadtxt('../data/jla/covmat/sigma_mu.txt')
    sigma_pecvel = (5 * 150 / 3e5) / (np.log(10.) * sigma[:, 2])
    Cmu[np.diag_indices_from(Cmu)] += sigma[:, 0] ** 2 + sigma[:, 1] ** 2 + sigma_pecvel ** 2
    
    return Cmu

Now you can compute the covariance matrix but it will depend on the nuisanse parammeters alfa and beta


In [9]:
Cmu = mu_cov(0.13, 3.1)

In [10]:
np.shape(Cmu)


Out[10]:
(740, 740)

Now I can compute the loglike in the same way as for the union2.1 data. Just copy the loglike function from before, add as params

  • OmDE
  • alpha
  • beta
  • M_b For the covariance matrix use the function from before, and instead of the "zandmu" variable use jla_dataset

In [15]:
def loglike_jla(params,h=0.7):
    """
    This function computes the logarithm of the likelihood. It recieves a vector
    params-> vector with three components (Omega Dark Energy, alpha, beta and M_b)
    """
    OmDE = params[0]
    alpha = params[1]
    beta = params[2]
    MB = params[3]
    covariance = mu_cov(alpha, beta)
    inv_covariance=np.linalg.inv(covariance)
# Ahora quiero calcular la diferencia entre el valor reportado y el calculado
    mu_obs = jla_dataset['mb']-(MB-alpha*jla_dataset['x1']+beta*jla_dataset['color'])
    mu_teo = 5.*np.log10(dl(jla_dataset['zcmb'],OmDE,h))+25
    delta = mu_teo - mu_obs
    chisquare = np.dot(delta,np.dot(inv_covariance,delta))
    return -chisquare/2

In [16]:
param = [0.65,0.13,3.1,-20]
loglike_jla(param)


Out[16]:
-2343.1642968305632

In [17]:
param = [0.7,0.13,3.1,-19]
loglike_jla(param)


Out[17]:
-358.18533951434921

And now we have a likelihood that depends on 4 parameters!


In [ ]: