Example to_hdf calls

Initialize the simulation with the tardis_example.yml configuration file.


In [1]:
from tardis.io.config_reader import Configuration
from tardis.simulation import Simulation

# Must have the tardis_example folder in the working directory.
config_fname = 'tardis_example/tardis_example.yml'
config = Configuration.from_yaml(config_fname)

simulation = Simulation.from_config(config)


[tardis.plasma.standard_plasmas][INFO   ]  Reading Atomic Data from tardis_example/kurucz_cd23_chianti_H_He.h5 (standard_plasmas.py:72)
tardis.plasma.standard_plasmas - INFO - Reading Atomic Data from tardis_example/kurucz_cd23_chianti_H_He.h5
tardis.atomic - INFO - Read Atom Data with UUID=5ca3035ca8b311e3bb684437e69d75d7 and MD5=21095dd25faa1683f4c90c911a00c3f8
[tardis.plasma.base  ][DEBUG  ]  Updating modules in the following order: (base.py:197)
tardis.plasma.base - DEBUG - Updating modules in the following order:
[tardis.montecarlo.base][DEBUG  ]  Electron scattering switched on (base.py:393)
tardis.montecarlo.base - DEBUG - Electron scattering switched on
[py.warnings         ][WARNING]  /home/vaibhav/anaconda2/lib/python2.7/site-packages/astropy/units/equivalencies.py:74: RuntimeWarning: divide by zero encountered in double_scalars
  (si.m, si.Hz, lambda x: _si.c.value / x),
 (equivalencies.py:74)
py.warnings - WARNING - /home/vaibhav/anaconda2/lib/python2.7/site-packages/astropy/units/equivalencies.py:74: RuntimeWarning: divide by zero encountered in double_scalars
  (si.m, si.Hz, lambda x: _si.c.value / x),

Run the simulation.


In [2]:
simulation.run()


[tardis.simulation.base][INFO   ]  Starting iteration 1/5 (base.py:196)
tardis.simulation.base - INFO - Starting iteration 1/5
[tardis.simulation.base][INFO   ]  Luminosity emitted = 8.10360e+42 erg / s Luminosity absorbed = 2.67587e+42 erg / s Luminosity requested = 1.07688e+43 erg / s (base.py:273)
tardis.simulation.base - INFO - Luminosity emitted = 8.10360e+42 erg / s Luminosity absorbed = 2.67587e+42 erg / s Luminosity requested = 1.07688e+43 erg / s
[tardis.simulation.base][INFO   ]  Plasma stratification:
	             t_rad    next_t_rad         w    next_w
	Shell                                               
	0      9967.488442  10074.800724  0.400392  0.452516
	5      9893.293062  10103.998786  0.211205  0.202916
	10     9820.194102  10007.439423  0.142695  0.130205
	15     9748.167438   9820.947309  0.104556  0.096257

 (base.py:264)
tardis.simulation.base - INFO - Plasma stratification:
	             t_rad    next_t_rad         w    next_w
	Shell                                               
	0      9967.488442  10074.800724  0.400392  0.452516
	5      9893.293062  10103.998786  0.211205  0.202916
	10     9820.194102  10007.439423  0.142695  0.130205
	15     9748.167438   9820.947309  0.104556  0.096257


[tardis.simulation.base][INFO   ]  t_inner 9974.969 K -- next t_inner 10736.934 K (base.py:266)
tardis.simulation.base - INFO - t_inner 9974.969 K -- next t_inner 10736.934 K
[tardis.plasma.base  ][DEBUG  ]  Updating modules in the following order: (base.py:197)
tardis.plasma.base - DEBUG - Updating modules in the following order:
[tardis.simulation.base][INFO   ]  Starting iteration 2/5 (base.py:196)
tardis.simulation.base - INFO - Starting iteration 2/5
[tardis.simulation.base][INFO   ]  Luminosity emitted = 1.08633e+43 erg / s Luminosity absorbed = 3.60272e+42 erg / s Luminosity requested = 1.07688e+43 erg / s (base.py:273)
tardis.simulation.base - INFO - Luminosity emitted = 1.08633e+43 erg / s Luminosity absorbed = 3.60272e+42 erg / s Luminosity requested = 1.07688e+43 erg / s
[tardis.simulation.base][INFO   ]  Plasma stratification:
	              t_rad    next_t_rad         w    next_w
	Shell                                                
	0      10074.800724  10480.642737  0.452516  0.485511
	5      10103.998786  10542.429569  0.202916  0.203942
	10     10007.439423  10413.108276  0.130205  0.127795
	15      9820.947309  10179.669303  0.096257  0.094281

 (base.py:264)
tardis.simulation.base - INFO - Plasma stratification:
	              t_rad    next_t_rad         w    next_w
	Shell                                                
	0      10074.800724  10480.642737  0.452516  0.485511
	5      10103.998786  10542.429569  0.202916  0.203942
	10     10007.439423  10413.108276  0.130205  0.127795
	15      9820.947309  10179.669303  0.096257  0.094281


[tardis.simulation.base][INFO   ]  t_inner 10736.934 K -- next t_inner 10713.534 K (base.py:266)
tardis.simulation.base - INFO - t_inner 10736.934 K -- next t_inner 10713.534 K
[tardis.plasma.base  ][DEBUG  ]  Updating modules in the following order: (base.py:197)
tardis.plasma.base - DEBUG - Updating modules in the following order:
[tardis.simulation.base][INFO   ]  Starting iteration 3/5 (base.py:196)
tardis.simulation.base - INFO - Starting iteration 3/5
[tardis.simulation.base][INFO   ]  Luminosity emitted = 1.08260e+43 erg / s Luminosity absorbed = 3.52022e+42 erg / s Luminosity requested = 1.07688e+43 erg / s (base.py:273)
tardis.simulation.base - INFO - Luminosity emitted = 1.08260e+43 erg / s Luminosity absorbed = 3.52022e+42 erg / s Luminosity requested = 1.07688e+43 erg / s
[tardis.simulation.base][INFO   ]  Plasma stratification:
	              t_rad    next_t_rad         w    next_w
	Shell                                                
	0      10480.642737  10711.113330  0.485511  0.493541
	5      10542.429569  10819.470987  0.203942  0.199112
	10     10413.108276  10633.892704  0.127795  0.125270
	15     10179.669303  10359.259776  0.094281  0.092971

 (base.py:264)
tardis.simulation.base - INFO - Plasma stratification:
	              t_rad    next_t_rad         w    next_w
	Shell                                                
	0      10480.642737  10711.113330  0.485511  0.493541
	5      10542.429569  10819.470987  0.203942  0.199112
	10     10413.108276  10633.892704  0.127795  0.125270
	15     10179.669303  10359.259776  0.094281  0.092971


[tardis.simulation.base][INFO   ]  t_inner 10713.534 K -- next t_inner 10699.352 K (base.py:266)
tardis.simulation.base - INFO - t_inner 10713.534 K -- next t_inner 10699.352 K
[tardis.plasma.base  ][DEBUG  ]  Updating modules in the following order: (base.py:197)
tardis.plasma.base - DEBUG - Updating modules in the following order:
[tardis.simulation.base][INFO   ]  Starting iteration 4/5 (base.py:196)
tardis.simulation.base - INFO - Starting iteration 4/5
[tardis.simulation.base][INFO   ]  Luminosity emitted = 1.07886e+43 erg / s Luminosity absorbed = 3.48159e+42 erg / s Luminosity requested = 1.07688e+43 erg / s (base.py:273)
tardis.simulation.base - INFO - Luminosity emitted = 1.07886e+43 erg / s Luminosity absorbed = 3.48159e+42 erg / s Luminosity requested = 1.07688e+43 erg / s
[tardis.simulation.base][INFO   ]  Plasma stratification:
	              t_rad    next_t_rad         w    next_w
	Shell                                                
	0      10711.113330  10866.508115  0.493541  0.489288
	5      10819.470987  11017.147737  0.199112  0.192830
	10     10633.892704  10795.422555  0.125270  0.121775
	15     10359.259776  10499.976778  0.092971  0.090524

 (base.py:264)
tardis.simulation.base - INFO - Plasma stratification:
	              t_rad    next_t_rad         w    next_w
	Shell                                                
	0      10711.113330  10866.508115  0.493541  0.489288
	5      10819.470987  11017.147737  0.199112  0.192830
	10     10633.892704  10795.422555  0.125270  0.121775
	15     10359.259776  10499.976778  0.092971  0.090524


[tardis.simulation.base][INFO   ]  t_inner 10699.352 K -- next t_inner 10694.430 K (base.py:266)
tardis.simulation.base - INFO - t_inner 10699.352 K -- next t_inner 10694.430 K
[tardis.plasma.base  ][DEBUG  ]  Updating modules in the following order: (base.py:197)
tardis.plasma.base - DEBUG - Updating modules in the following order:
[tardis.simulation.base][INFO   ]  Starting iteration 5/5 (base.py:196)
tardis.simulation.base - INFO - Starting iteration 5/5
[tardis.simulation.base][INFO   ]  Luminosity emitted = 1.07897e+43 erg / s Luminosity absorbed = 3.45406e+42 erg / s Luminosity requested = 1.07688e+43 erg / s (base.py:273)
tardis.simulation.base - INFO - Luminosity emitted = 1.07897e+43 erg / s Luminosity absorbed = 3.45406e+42 erg / s Luminosity requested = 1.07688e+43 erg / s
[tardis.simulation.base][INFO   ]  Simulation finished in 5 iterations and took 50.01 s (base.py:223)
tardis.simulation.base - INFO - Simulation finished in 5 iterations and took 50.01 s

You can now use to_hdf method, to save properties to a HDF file.

Parameters

file_path: Path where the HDF file should be stored. (Required)
path: Path inside the HDF store to store the elements. (Optional)
name: Name of the group inside HDF store, under which properties will be saved.(Optional)


In [3]:
simulation.to_hdf('/tmp/full_example.hdf')
#simulation.to_hdf(file_path='/tmp/full_example.hdf', path='/', name='simulation')


[py.warnings         ][WARNING]  /home/vaibhav/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py:1299: PerformanceWarning: 
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->block0_values] [items->[0]]

  return pytables.to_hdf(path_or_buf, key, self, **kwargs)
 (pytables.py:2675)
py.warnings - WARNING - /home/vaibhav/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py:1299: PerformanceWarning: 
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->block0_values] [items->[0]]

  return pytables.to_hdf(path_or_buf, key, self, **kwargs)

[py.warnings         ][WARNING]  /home/vaibhav/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py:1299: PerformanceWarning: 
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->values] [items->None]

  return pytables.to_hdf(path_or_buf, key, self, **kwargs)
 (pytables.py:2675)
py.warnings - WARNING - /home/vaibhav/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py:1299: PerformanceWarning: 
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->values] [items->None]

  return pytables.to_hdf(path_or_buf, key, self, **kwargs)

Open the stored HDF file with pandas and print its structure.


In [4]:
import pandas as pd
data = pd.HDFStore('/tmp/full_example.hdf')
print data


<class 'pandas.io.pytables.HDFStore'>
File path: /tmp/full_example.hdf
/simulation/model/homologous_density/density_0               series       (shape->[21])      
/simulation/model/homologous_density/scalars                 series       (shape->[1])       
/simulation/model/scalars                                    series       (shape->[1])       
/simulation/model/t_radiative                                series       (shape->[20])      
/simulation/model/v_inner                                    series       (shape->[20])      
/simulation/model/v_outer                                    series       (shape->[20])      
/simulation/model/w                                          series       (shape->[20])      
/simulation/plasma/abundance                                 frame        (shape->[6,20])    
/simulation/plasma/atomic_mass                               series       (shape->[6])       
/simulation/plasma/beta_rad                                  series       (shape->[20])      
/simulation/plasma/beta_sobolev                              frame        (shape->[29224,20])
/simulation/plasma/density                                   series       (shape->[20])      
/simulation/plasma/electron_densities                        series       (shape->[20])      
/simulation/plasma/excitation_energy                         series       (shape->[4439])    
/simulation/plasma/f_lu                                      series       (shape->[29224])   
/simulation/plasma/g                                         series       (shape->[4439])    
/simulation/plasma/g_electron                                series       (shape->[20])      
/simulation/plasma/general_level_boltzmann_factor            frame        (shape->[4439,20]) 
/simulation/plasma/ion_number_density                        frame        (shape->[94,20])   
/simulation/plasma/ionization_data                           frame        (shape->[88,1])    
/simulation/plasma/j_blues                                   frame        (shape->[29224,20])
/simulation/plasma/level_boltzmann_factor                    frame        (shape->[4439,20]) 
/simulation/plasma/level_number_density                      frame        (shape->[4439,20]) 
/simulation/plasma/levels                                    frame        (shape->[1,1])     
/simulation/plasma/lines                                     frame        (shape->[29224,12])
/simulation/plasma/lines_lower_level_index                   series       (shape->[29224])   
/simulation/plasma/lines_upper_level_index                   series       (shape->[29224])   
/simulation/plasma/metastability                             series       (shape->[4439])    
/simulation/plasma/nu                                        series       (shape->[29224])   
/simulation/plasma/number_density                            frame        (shape->[6,20])    
/simulation/plasma/partition_function                        frame        (shape->[94,20])   
/simulation/plasma/phi                                       frame        (shape->[88,20])   
/simulation/plasma/scalars                                   series       (shape->[1])       
/simulation/plasma/selected_atoms                            series       (shape->[6])       
/simulation/plasma/stimulated_emission_factor                frame        (shape->[29224,20])
/simulation/plasma/t_electrons                               series       (shape->[20])      
/simulation/plasma/t_rad                                     series       (shape->[20])      
/simulation/plasma/tau_sobolevs                              frame        (shape->[29224,20])
/simulation/plasma/transition_probabilities                  frame        (shape->[87672,20])
/simulation/plasma/w                                         series       (shape->[20])      
/simulation/plasma/wavelength_cm                             series       (shape->[29224])   
/simulation/runner/j_estimator                               series       (shape->[20])      
/simulation/runner/last_interaction_in_nu                    series       (shape->[500000])  
/simulation/runner/last_interaction_type                     series       (shape->[500000])  
/simulation/runner/last_line_interaction_in_id               series       (shape->[500000])  
/simulation/runner/last_line_interaction_out_id              series       (shape->[500000])  
/simulation/runner/last_line_interaction_shell_id            series       (shape->[500000])  
/simulation/runner/montecarlo_virtual_luminosity             series       (shape->[10000])   
/simulation/runner/nu_bar_estimator                          series       (shape->[20])      
/simulation/runner/output_energy                             series       (shape->[500000])  
/simulation/runner/output_nu                                 series       (shape->[500000])  
/simulation/runner/packet_luminosity                         series       (shape->[500000])  
/simulation/runner/spectrum/_frequency                       series       (shape->[10001])   
/simulation/runner/spectrum/luminosity                       series       (shape->[10000])   
/simulation/runner/spectrum_reabsorbed/_frequency            series       (shape->[10001])   
/simulation/runner/spectrum_reabsorbed/luminosity            series       (shape->[10000])   
/simulation/runner/spectrum_virtual/_frequency               series       (shape->[10001])   
/simulation/runner/spectrum_virtual/luminosity               series       (shape->[10000])   

Access model.homologous_density.density_0 under simulation, which is a one-dimensional array


In [5]:
print data['/simulation/model/homologous_density/density_0']


0     1970.527174
1       13.360318
2       10.146658
3        7.786621
4        6.033444
5        4.717122
6        3.718946
7        2.954982
8        2.365191
9        1.906156
10       1.546154
11       1.261789
12       1.035646
13       0.854653
14       0.708918
15       0.590901
16       0.494811
17       0.416168
18       0.351490
19       0.298047
20       0.253691
dtype: float64

Scalars are stored in a scalars pandas.Series for every module. For example to access model.t_inner under simulation, one would need to do the following.

Note: Quantities are always stored as their SI values.


In [6]:
print data['/simulation/model/scalars']['t_inner']


10694.430019

Breakdown of the various to_hdf methods

Every module in TARDIS has its own to_hdf method responsible to store its own data to an HDF file.

Plasma

The following call will store every plasma property to /tmp/plasma_output.hdf under /parent/plasma


In [7]:
simulation.plasma.to_hdf('/tmp/plasma_output.hdf', path='parent')

In [8]:
import pandas
with pandas.HDFStore('/tmp/plasma_output.hdf') as data:
    print data


<class 'pandas.io.pytables.HDFStore'>
File path: /tmp/plasma_output.hdf
/parent/plasma/abundance                                 frame        (shape->[6,20])    
/parent/plasma/atomic_mass                               series       (shape->[6])       
/parent/plasma/beta_rad                                  series       (shape->[20])      
/parent/plasma/beta_sobolev                              frame        (shape->[29224,20])
/parent/plasma/density                                   series       (shape->[20])      
/parent/plasma/electron_densities                        series       (shape->[20])      
/parent/plasma/excitation_energy                         series       (shape->[4439])    
/parent/plasma/f_lu                                      series       (shape->[29224])   
/parent/plasma/g                                         series       (shape->[4439])    
/parent/plasma/g_electron                                series       (shape->[20])      
/parent/plasma/general_level_boltzmann_factor            frame        (shape->[4439,20]) 
/parent/plasma/ion_number_density                        frame        (shape->[94,20])   
/parent/plasma/ionization_data                           frame        (shape->[88,1])    
/parent/plasma/j_blues                                   frame        (shape->[29224,20])
/parent/plasma/level_boltzmann_factor                    frame        (shape->[4439,20]) 
/parent/plasma/level_number_density                      frame        (shape->[4439,20]) 
/parent/plasma/levels                                    frame        (shape->[1,1])     
/parent/plasma/lines                                     frame        (shape->[29224,12])
/parent/plasma/lines_lower_level_index                   series       (shape->[29224])   
/parent/plasma/lines_upper_level_index                   series       (shape->[29224])   
/parent/plasma/metastability                             series       (shape->[4439])    
/parent/plasma/nu                                        series       (shape->[29224])   
/parent/plasma/number_density                            frame        (shape->[6,20])    
/parent/plasma/partition_function                        frame        (shape->[94,20])   
/parent/plasma/phi                                       frame        (shape->[88,20])   
/parent/plasma/scalars                                   series       (shape->[1])       
/parent/plasma/selected_atoms                            series       (shape->[6])       
/parent/plasma/stimulated_emission_factor                frame        (shape->[29224,20])
/parent/plasma/t_electrons                               series       (shape->[20])      
/parent/plasma/t_rad                                     series       (shape->[20])      
/parent/plasma/tau_sobolevs                              frame        (shape->[29224,20])
/parent/plasma/transition_probabilities                  frame        (shape->[87672,20])
/parent/plasma/w                                         series       (shape->[20])      
/parent/plasma/wavelength_cm                             series       (shape->[29224])   

Plasma's to_hdf method can also accept a collection parameter which can specify which types of plasma properties will be stored. For example if we wanted to only store Input plasma properties, we would do the following:


In [9]:
from tardis.plasma.properties.base import Input
simulation.plasma.to_hdf('/tmp/plasma_input_output.hdf', collection=[Input])

In [10]:
import pandas
with pandas.HDFStore('/tmp/plasma_input_output.hdf') as data:
    print data


<class 'pandas.io.pytables.HDFStore'>
File path: /tmp/plasma_input_output.hdf
/plasma/abundance            frame        (shape->[6,20])
/plasma/density              series       (shape->[20])  
/plasma/scalars              series       (shape->[1])   
/plasma/t_rad                series       (shape->[20])  
/plasma/w                    series       (shape->[20])  

Model

The following call will store properties of the Radial1DModel to /tmp/model_output.hdf under /model.


In [11]:
simulation.model.to_hdf('/tmp/model_output.hdf')

MontecarloRunner

The following call will store properties of the MontecarloRunner to /tmp/runner_output.hdf under /runner


In [12]:
simulation.runner.to_hdf('/tmp/runner_output.hdf')

In [13]:
import pandas
with pandas.HDFStore('/tmp/runner_output.hdf') as data:
    print data


<class 'pandas.io.pytables.HDFStore'>
File path: /tmp/runner_output.hdf
/runner/j_estimator                               series       (shape->[20])    
/runner/last_interaction_in_nu                    series       (shape->[500000])
/runner/last_interaction_type                     series       (shape->[500000])
/runner/last_line_interaction_in_id               series       (shape->[500000])
/runner/last_line_interaction_out_id              series       (shape->[500000])
/runner/last_line_interaction_shell_id            series       (shape->[500000])
/runner/montecarlo_virtual_luminosity             series       (shape->[10000]) 
/runner/nu_bar_estimator                          series       (shape->[20])    
/runner/output_energy                             series       (shape->[500000])
/runner/output_nu                                 series       (shape->[500000])
/runner/packet_luminosity                         series       (shape->[500000])
/runner/spectrum/_frequency                       series       (shape->[10001]) 
/runner/spectrum/luminosity                       series       (shape->[10000]) 
/runner/spectrum_reabsorbed/_frequency            series       (shape->[10001]) 
/runner/spectrum_reabsorbed/luminosity            series       (shape->[10000]) 
/runner/spectrum_virtual/_frequency               series       (shape->[10001]) 
/runner/spectrum_virtual/luminosity               series       (shape->[10000])