This notebook is available at https://github.com/tardis-sn/tardis/tree/master/docs/io/output/to_hdf_notebook.ipynb
Example to_hdf calls¶
Initialize the simulation with the tardis_example.yml configuration file.
[1]:
from tardis.io.config_reader import Configuration
from tardis.simulation import Simulation
# Must have the tardis_example folder in the working directory.
config_fname = 'tardis_example/tardis_example.yml'
config = Configuration.from_yaml(config_fname)
simulation = Simulation.from_config(config)
[tardis.plasma.standard_plasmas][INFO ] Reading Atomic Data from tardis_example/kurucz_cd23_chianti_H_He.h5 (standard_plasmas.py:72)
tardis.plasma.standard_plasmas - INFO - Reading Atomic Data from tardis_example/kurucz_cd23_chianti_H_He.h5
tardis.atomic - INFO - Read Atom Data with UUID=5ca3035ca8b311e3bb684437e69d75d7 and MD5=21095dd25faa1683f4c90c911a00c3f8
[tardis.plasma.base ][DEBUG ] Updating modules in the following order: (base.py:197)
tardis.plasma.base - DEBUG - Updating modules in the following order:
[tardis.montecarlo.base][DEBUG ] Electron scattering switched on (base.py:393)
tardis.montecarlo.base - DEBUG - Electron scattering switched on
[py.warnings ][WARNING] /home/vaibhav/anaconda2/lib/python2.7/site-packages/astropy/units/equivalencies.py:74: RuntimeWarning: divide by zero encountered in double_scalars
(si.m, si.Hz, lambda x: _si.c.value / x),
(equivalencies.py:74)
py.warnings - WARNING - /home/vaibhav/anaconda2/lib/python2.7/site-packages/astropy/units/equivalencies.py:74: RuntimeWarning: divide by zero encountered in double_scalars
(si.m, si.Hz, lambda x: _si.c.value / x),
Run the simulation.
[2]:
simulation.run()
[tardis.simulation.base][INFO ] Starting iteration 1/5 (base.py:196)
tardis.simulation.base - INFO - Starting iteration 1/5
[tardis.simulation.base][INFO ] Luminosity emitted = 8.10360e+42 erg / s Luminosity absorbed = 2.67587e+42 erg / s Luminosity requested = 1.07688e+43 erg / s (base.py:273)
tardis.simulation.base - INFO - Luminosity emitted = 8.10360e+42 erg / s Luminosity absorbed = 2.67587e+42 erg / s Luminosity requested = 1.07688e+43 erg / s
[tardis.simulation.base][INFO ] Plasma stratification:
t_rad next_t_rad w next_w
Shell
0 9967.488442 10074.800724 0.400392 0.452516
5 9893.293062 10103.998786 0.211205 0.202916
10 9820.194102 10007.439423 0.142695 0.130205
15 9748.167438 9820.947309 0.104556 0.096257
(base.py:264)
tardis.simulation.base - INFO - Plasma stratification:
t_rad next_t_rad w next_w
Shell
0 9967.488442 10074.800724 0.400392 0.452516
5 9893.293062 10103.998786 0.211205 0.202916
10 9820.194102 10007.439423 0.142695 0.130205
15 9748.167438 9820.947309 0.104556 0.096257
[tardis.simulation.base][INFO ] t_inner 9974.969 K -- next t_inner 10736.934 K (base.py:266)
tardis.simulation.base - INFO - t_inner 9974.969 K -- next t_inner 10736.934 K
[tardis.plasma.base ][DEBUG ] Updating modules in the following order: (base.py:197)
tardis.plasma.base - DEBUG - Updating modules in the following order:
[tardis.simulation.base][INFO ] Starting iteration 2/5 (base.py:196)
tardis.simulation.base - INFO - Starting iteration 2/5
[tardis.simulation.base][INFO ] Luminosity emitted = 1.08633e+43 erg / s Luminosity absorbed = 3.60272e+42 erg / s Luminosity requested = 1.07688e+43 erg / s (base.py:273)
tardis.simulation.base - INFO - Luminosity emitted = 1.08633e+43 erg / s Luminosity absorbed = 3.60272e+42 erg / s Luminosity requested = 1.07688e+43 erg / s
[tardis.simulation.base][INFO ] Plasma stratification:
t_rad next_t_rad w next_w
Shell
0 10074.800724 10480.642737 0.452516 0.485511
5 10103.998786 10542.429569 0.202916 0.203942
10 10007.439423 10413.108276 0.130205 0.127795
15 9820.947309 10179.669303 0.096257 0.094281
(base.py:264)
tardis.simulation.base - INFO - Plasma stratification:
t_rad next_t_rad w next_w
Shell
0 10074.800724 10480.642737 0.452516 0.485511
5 10103.998786 10542.429569 0.202916 0.203942
10 10007.439423 10413.108276 0.130205 0.127795
15 9820.947309 10179.669303 0.096257 0.094281
[tardis.simulation.base][INFO ] t_inner 10736.934 K -- next t_inner 10713.534 K (base.py:266)
tardis.simulation.base - INFO - t_inner 10736.934 K -- next t_inner 10713.534 K
[tardis.plasma.base ][DEBUG ] Updating modules in the following order: (base.py:197)
tardis.plasma.base - DEBUG - Updating modules in the following order:
[tardis.simulation.base][INFO ] Starting iteration 3/5 (base.py:196)
tardis.simulation.base - INFO - Starting iteration 3/5
[tardis.simulation.base][INFO ] Luminosity emitted = 1.08260e+43 erg / s Luminosity absorbed = 3.52022e+42 erg / s Luminosity requested = 1.07688e+43 erg / s (base.py:273)
tardis.simulation.base - INFO - Luminosity emitted = 1.08260e+43 erg / s Luminosity absorbed = 3.52022e+42 erg / s Luminosity requested = 1.07688e+43 erg / s
[tardis.simulation.base][INFO ] Plasma stratification:
t_rad next_t_rad w next_w
Shell
0 10480.642737 10711.113330 0.485511 0.493541
5 10542.429569 10819.470987 0.203942 0.199112
10 10413.108276 10633.892704 0.127795 0.125270
15 10179.669303 10359.259776 0.094281 0.092971
(base.py:264)
tardis.simulation.base - INFO - Plasma stratification:
t_rad next_t_rad w next_w
Shell
0 10480.642737 10711.113330 0.485511 0.493541
5 10542.429569 10819.470987 0.203942 0.199112
10 10413.108276 10633.892704 0.127795 0.125270
15 10179.669303 10359.259776 0.094281 0.092971
[tardis.simulation.base][INFO ] t_inner 10713.534 K -- next t_inner 10699.352 K (base.py:266)
tardis.simulation.base - INFO - t_inner 10713.534 K -- next t_inner 10699.352 K
[tardis.plasma.base ][DEBUG ] Updating modules in the following order: (base.py:197)
tardis.plasma.base - DEBUG - Updating modules in the following order:
[tardis.simulation.base][INFO ] Starting iteration 4/5 (base.py:196)
tardis.simulation.base - INFO - Starting iteration 4/5
[tardis.simulation.base][INFO ] Luminosity emitted = 1.07886e+43 erg / s Luminosity absorbed = 3.48159e+42 erg / s Luminosity requested = 1.07688e+43 erg / s (base.py:273)
tardis.simulation.base - INFO - Luminosity emitted = 1.07886e+43 erg / s Luminosity absorbed = 3.48159e+42 erg / s Luminosity requested = 1.07688e+43 erg / s
[tardis.simulation.base][INFO ] Plasma stratification:
t_rad next_t_rad w next_w
Shell
0 10711.113330 10866.508115 0.493541 0.489288
5 10819.470987 11017.147737 0.199112 0.192830
10 10633.892704 10795.422555 0.125270 0.121775
15 10359.259776 10499.976778 0.092971 0.090524
(base.py:264)
tardis.simulation.base - INFO - Plasma stratification:
t_rad next_t_rad w next_w
Shell
0 10711.113330 10866.508115 0.493541 0.489288
5 10819.470987 11017.147737 0.199112 0.192830
10 10633.892704 10795.422555 0.125270 0.121775
15 10359.259776 10499.976778 0.092971 0.090524
[tardis.simulation.base][INFO ] t_inner 10699.352 K -- next t_inner 10694.430 K (base.py:266)
tardis.simulation.base - INFO - t_inner 10699.352 K -- next t_inner 10694.430 K
[tardis.plasma.base ][DEBUG ] Updating modules in the following order: (base.py:197)
tardis.plasma.base - DEBUG - Updating modules in the following order:
[tardis.simulation.base][INFO ] Starting iteration 5/5 (base.py:196)
tardis.simulation.base - INFO - Starting iteration 5/5
[tardis.simulation.base][INFO ] Luminosity emitted = 1.07897e+43 erg / s Luminosity absorbed = 3.45406e+42 erg / s Luminosity requested = 1.07688e+43 erg / s (base.py:273)
tardis.simulation.base - INFO - Luminosity emitted = 1.07897e+43 erg / s Luminosity absorbed = 3.45406e+42 erg / s Luminosity requested = 1.07688e+43 erg / s
[tardis.simulation.base][INFO ] Simulation finished in 5 iterations and took 50.01 s (base.py:223)
tardis.simulation.base - INFO - Simulation finished in 5 iterations and took 50.01 s
You can now use to_hdf method, to save properties to a HDF file.
Parameters¶
file_path: Path where the HDF file should be stored. (Required)path: Path inside the HDF store to store the elements. (Optional)name: Name of the group inside HDF store, under which properties will be saved.(Optional)[3]:
simulation.to_hdf('/tmp/full_example.hdf')
#simulation.to_hdf(file_path='/tmp/full_example.hdf', path='/', name='simulation')
[py.warnings ][WARNING] /home/vaibhav/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py:1299: PerformanceWarning:
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->block0_values] [items->[0]]
return pytables.to_hdf(path_or_buf, key, self, **kwargs)
(pytables.py:2675)
py.warnings - WARNING - /home/vaibhav/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py:1299: PerformanceWarning:
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->block0_values] [items->[0]]
return pytables.to_hdf(path_or_buf, key, self, **kwargs)
[py.warnings ][WARNING] /home/vaibhav/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py:1299: PerformanceWarning:
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->values] [items->None]
return pytables.to_hdf(path_or_buf, key, self, **kwargs)
(pytables.py:2675)
py.warnings - WARNING - /home/vaibhav/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py:1299: PerformanceWarning:
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->values] [items->None]
return pytables.to_hdf(path_or_buf, key, self, **kwargs)
Open the stored HDF file with pandas and print its structure.
[4]:
import pandas as pd
data = pd.HDFStore('/tmp/full_example.hdf')
print data
<class 'pandas.io.pytables.HDFStore'>
File path: /tmp/full_example.hdf
/simulation/model/homologous_density/density_0 series (shape->[21])
/simulation/model/homologous_density/scalars series (shape->[1])
/simulation/model/scalars series (shape->[1])
/simulation/model/t_radiative series (shape->[20])
/simulation/model/v_inner series (shape->[20])
/simulation/model/v_outer series (shape->[20])
/simulation/model/w series (shape->[20])
/simulation/plasma/abundance frame (shape->[6,20])
/simulation/plasma/atomic_mass series (shape->[6])
/simulation/plasma/beta_rad series (shape->[20])
/simulation/plasma/beta_sobolev frame (shape->[29224,20])
/simulation/plasma/density series (shape->[20])
/simulation/plasma/electron_densities series (shape->[20])
/simulation/plasma/excitation_energy series (shape->[4439])
/simulation/plasma/f_lu series (shape->[29224])
/simulation/plasma/g series (shape->[4439])
/simulation/plasma/g_electron series (shape->[20])
/simulation/plasma/general_level_boltzmann_factor frame (shape->[4439,20])
/simulation/plasma/ion_number_density frame (shape->[94,20])
/simulation/plasma/ionization_data frame (shape->[88,1])
/simulation/plasma/j_blues frame (shape->[29224,20])
/simulation/plasma/level_boltzmann_factor frame (shape->[4439,20])
/simulation/plasma/level_number_density frame (shape->[4439,20])
/simulation/plasma/levels frame (shape->[1,1])
/simulation/plasma/lines frame (shape->[29224,12])
/simulation/plasma/lines_lower_level_index series (shape->[29224])
/simulation/plasma/lines_upper_level_index series (shape->[29224])
/simulation/plasma/metastability series (shape->[4439])
/simulation/plasma/nu series (shape->[29224])
/simulation/plasma/number_density frame (shape->[6,20])
/simulation/plasma/partition_function frame (shape->[94,20])
/simulation/plasma/phi frame (shape->[88,20])
/simulation/plasma/scalars series (shape->[1])
/simulation/plasma/selected_atoms series (shape->[6])
/simulation/plasma/stimulated_emission_factor frame (shape->[29224,20])
/simulation/plasma/t_electrons series (shape->[20])
/simulation/plasma/t_rad series (shape->[20])
/simulation/plasma/tau_sobolevs frame (shape->[29224,20])
/simulation/plasma/transition_probabilities frame (shape->[87672,20])
/simulation/plasma/w series (shape->[20])
/simulation/plasma/wavelength_cm series (shape->[29224])
/simulation/runner/j_estimator series (shape->[20])
/simulation/runner/last_interaction_in_nu series (shape->[500000])
/simulation/runner/last_interaction_type series (shape->[500000])
/simulation/runner/last_line_interaction_in_id series (shape->[500000])
/simulation/runner/last_line_interaction_out_id series (shape->[500000])
/simulation/runner/last_line_interaction_shell_id series (shape->[500000])
/simulation/runner/montecarlo_virtual_luminosity series (shape->[10000])
/simulation/runner/nu_bar_estimator series (shape->[20])
/simulation/runner/output_energy series (shape->[500000])
/simulation/runner/output_nu series (shape->[500000])
/simulation/runner/packet_luminosity series (shape->[500000])
/simulation/runner/spectrum/_frequency series (shape->[10001])
/simulation/runner/spectrum/luminosity series (shape->[10000])
/simulation/runner/spectrum_reabsorbed/_frequency series (shape->[10001])
/simulation/runner/spectrum_reabsorbed/luminosity series (shape->[10000])
/simulation/runner/spectrum_virtual/_frequency series (shape->[10001])
/simulation/runner/spectrum_virtual/luminosity series (shape->[10000])
Access model.homologous_density.density_0 under simulation, which is a one-dimensional array
[5]:
print data['/simulation/model/homologous_density/density_0']
0 1970.527174
1 13.360318
2 10.146658
3 7.786621
4 6.033444
5 4.717122
6 3.718946
7 2.954982
8 2.365191
9 1.906156
10 1.546154
11 1.261789
12 1.035646
13 0.854653
14 0.708918
15 0.590901
16 0.494811
17 0.416168
18 0.351490
19 0.298047
20 0.253691
dtype: float64
Scalars are stored in a scalars pandas.Series for every module. For example to access model.t_inner under simulation, one would need to do the following.
Note: Quantities are always stored as their SI values.
[6]:
print data['/simulation/model/scalars']['t_inner']
10694.430019
Breakdown of the various to_hdf methods¶
Every module in TARDIS has its own to_hdf method responsible to store its own data to an HDF file.
Plasma¶
The following call will store every plasma property to /tmp/plasma_output.hdf under /parent/plasma
[7]:
simulation.plasma.to_hdf('/tmp/plasma_output.hdf', path='parent')
[8]:
import pandas
with pandas.HDFStore('/tmp/plasma_output.hdf') as data:
print data
<class 'pandas.io.pytables.HDFStore'>
File path: /tmp/plasma_output.hdf
/parent/plasma/abundance frame (shape->[6,20])
/parent/plasma/atomic_mass series (shape->[6])
/parent/plasma/beta_rad series (shape->[20])
/parent/plasma/beta_sobolev frame (shape->[29224,20])
/parent/plasma/density series (shape->[20])
/parent/plasma/electron_densities series (shape->[20])
/parent/plasma/excitation_energy series (shape->[4439])
/parent/plasma/f_lu series (shape->[29224])
/parent/plasma/g series (shape->[4439])
/parent/plasma/g_electron series (shape->[20])
/parent/plasma/general_level_boltzmann_factor frame (shape->[4439,20])
/parent/plasma/ion_number_density frame (shape->[94,20])
/parent/plasma/ionization_data frame (shape->[88,1])
/parent/plasma/j_blues frame (shape->[29224,20])
/parent/plasma/level_boltzmann_factor frame (shape->[4439,20])
/parent/plasma/level_number_density frame (shape->[4439,20])
/parent/plasma/levels frame (shape->[1,1])
/parent/plasma/lines frame (shape->[29224,12])
/parent/plasma/lines_lower_level_index series (shape->[29224])
/parent/plasma/lines_upper_level_index series (shape->[29224])
/parent/plasma/metastability series (shape->[4439])
/parent/plasma/nu series (shape->[29224])
/parent/plasma/number_density frame (shape->[6,20])
/parent/plasma/partition_function frame (shape->[94,20])
/parent/plasma/phi frame (shape->[88,20])
/parent/plasma/scalars series (shape->[1])
/parent/plasma/selected_atoms series (shape->[6])
/parent/plasma/stimulated_emission_factor frame (shape->[29224,20])
/parent/plasma/t_electrons series (shape->[20])
/parent/plasma/t_rad series (shape->[20])
/parent/plasma/tau_sobolevs frame (shape->[29224,20])
/parent/plasma/transition_probabilities frame (shape->[87672,20])
/parent/plasma/w series (shape->[20])
/parent/plasma/wavelength_cm series (shape->[29224])
Plasma’s to_hdf method can also accept a collection parameter which can specify which types of plasma properties will be stored. For example if we wanted to only store Input plasma properties, we would do the following:
[9]:
from tardis.plasma.properties.base import Input
simulation.plasma.to_hdf('/tmp/plasma_input_output.hdf', collection=[Input])
[10]:
import pandas
with pandas.HDFStore('/tmp/plasma_input_output.hdf') as data:
print data
<class 'pandas.io.pytables.HDFStore'>
File path: /tmp/plasma_input_output.hdf
/plasma/abundance frame (shape->[6,20])
/plasma/density series (shape->[20])
/plasma/scalars series (shape->[1])
/plasma/t_rad series (shape->[20])
/plasma/w series (shape->[20])
Model¶
The following call will store properties of the Radial1DModel to /tmp/model_output.hdf under /model.
[11]:
simulation.model.to_hdf('/tmp/model_output.hdf')
MontecarloRunner¶
The following call will store properties of the MontecarloRunner to /tmp/runner_output.hdf under /runner
[12]:
simulation.runner.to_hdf('/tmp/runner_output.hdf')
[13]:
import pandas
with pandas.HDFStore('/tmp/runner_output.hdf') as data:
print data
<class 'pandas.io.pytables.HDFStore'>
File path: /tmp/runner_output.hdf
/runner/j_estimator series (shape->[20])
/runner/last_interaction_in_nu series (shape->[500000])
/runner/last_interaction_type series (shape->[500000])
/runner/last_line_interaction_in_id series (shape->[500000])
/runner/last_line_interaction_out_id series (shape->[500000])
/runner/last_line_interaction_shell_id series (shape->[500000])
/runner/montecarlo_virtual_luminosity series (shape->[10000])
/runner/nu_bar_estimator series (shape->[20])
/runner/output_energy series (shape->[500000])
/runner/output_nu series (shape->[500000])
/runner/packet_luminosity series (shape->[500000])
/runner/spectrum/_frequency series (shape->[10001])
/runner/spectrum/luminosity series (shape->[10000])
/runner/spectrum_reabsorbed/_frequency series (shape->[10001])
/runner/spectrum_reabsorbed/luminosity series (shape->[10000])
/runner/spectrum_virtual/_frequency series (shape->[10001])
/runner/spectrum_virtual/luminosity series (shape->[10000])