Downloading Waveforms from Different Catalogs

This tutorial demonstrates how to download and work with waveforms from various numerical relativity catalogs using PyART. PyART provides a unified interface to access data from multiple catalogs including:

  • SXS (Simulating eXtreme Spacetimes)

  • RIT (Rochester Institute of Technology)

  • CoRe (Computational Relativity)

  • GRA (GR-Athena++)

  • ICCUB (Institute of Cosmos Sciences)

  • SACRA

Setup

First, let’s import the necessary modules:

%matplotlib inline
%config InlineBackend.figure_format = 'retina'

from PyART.catalogs import sxs, rit, core, icc_public
from PyART.catalogs.cataloger import Cataloger

/opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/PyART/analysis/match.py:15: UserWarning: Wswiglal-redir-stdio:

SWIGLAL standard output/error redirection is enabled in IPython.
This may lead to performance penalties. To disable locally, use:

with lal.no_swig_redirect_standard_output_error():
    ...

To disable globally, use:

lal.swig_redirect_standard_output_error(False)

Note however that this will likely lead to error messages from
LAL functions being either misdirected or lost when called from
Jupyter notebooks.

To suppress this warning, use:

import warnings
warnings.filterwarnings("ignore", "Wswiglal-redir-stdio")
import lal

  import lal

WARNING: TEOBResumS not installed.

Downloading from a Single Catalog

Let’s start by downloading waveforms from the SXS catalog. We specify:

  • The catalog name

  • The simulation ID

  • Whether to download if not locally available

SXS

# Download and load an SXS waveform
# Note: You may need to adjust the path or set download=True
sxs_id = '0180'
wf_sxs = sxs.Waveform_SXS(path='./', ID=sxs_id, download=True, cut_N=300, ignore_deprecation=True)

print(f"SXS Waveform {sxs_id} loaded successfully")
print(f"Mass ratio q = {wf_sxs.metadata['q']:.3f}")
print(f"Total mass M = {wf_sxs.metadata['M']:.3f}")
print(f"Chi1z = {wf_sxs.metadata['chi1z']:.3f}")
print(f"Chi2z = {wf_sxs.metadata['chi2z']:.3f}")

---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
Cell In[2], line 4
      1 # Download and load an SXS waveform
      2 # Note: You may need to adjust the path or set download=True
      3 sxs_id = '0180'
----> 4 wf_sxs = sxs.Waveform_SXS(path='./', ID=sxs_id, download=True, cut_N=300, ignore_deprecation=True)
      6 print(f"SXS Waveform {sxs_id} loaded successfully")
      7 print(f"Mass ratio q = {wf_sxs.metadata['q']:.3f}")

File /opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/PyART/catalogs/sxs.py:135, in Waveform_SXS.__init__(self, path, ID, order, level, cut_N, cut_U, ellmax, load, download, downloads, load_m0, nu_rescale, src, ignore_deprecation, basename)
    131     logging.info(
    132         f"The path {self.sxs_data_path} does not exist or contains no 'Lev*' directory."
    133     )
    134     logging.info("Downloading the simulation from the SXS catalog.")
--> 135     self.download_simulation(
    136         ID=self.ID,
    137         path=path,
    138         downloads=downloads,
    139         level=self.level,
    140         ignore_deprecation=ignore_deprecation,
    141         extrapolation_order=order,
    142     )
    143 else:
    144     logging.warning(
    145         "Use download=True to download the simulation from the SXS catalog."
    146     )

File /opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/PyART/catalogs/sxs.py:290, in Waveform_SXS.download_simulation(self, ID, path, downloads, level, ignore_deprecation, extrapolation_order)
    288 sys.stdout = open(os.devnull, "w")
    289 sys.stderr = open(os.devnull, "w")
--> 290 sxs_sim = sxsmod.load(
    291     name_level,
    292     extrapolation_order=extrapolation_order,
    293     ignore_deprecation=ignore_deprecation,
    294     progress=True,
    295 )
    296 logging.info(f"Loaded SXS simulation {name_level}.")
    298 # Set Level if not already set

File /opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/sxs/handlers.py:288, in load(location, download, cache, progress, truepath, **kwargs)
    285     return sxscatalog.load(location, download=download, **kwargs)
    287 elif sxs_id_version_lev_exact_re.match(location):
--> 288     return Simulation(location, download=download, cache=cache, progress=progress, **kwargs)
    290 else:
    291     # Try to find an appropriate SXS file in the simulations
    292     simulations = Simulations.load(
    293         download=download,
    294         local=kwargs.get("local", False),
    295         annex_dir=kwargs.get("annex_dir", None)
    296     )

File /opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/sxs/simulations/simulation.py:155, in Simulation(location, *args, **kwargs)
    153 # Attach metadata to this object
    154 metadata = Metadata(simulations[simulation_id])
--> 155 series = simulations.dataframe.loc[simulation_id]
    157 # If input_version is not the default, remove "files" from metadata
    158 version_is_not_default = (
    159     input_version
    160     and input_version != max(metadata.get("DOI_versions", []), default="")
    161 )

File /opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/sxscatalog/simulations/simulations.py:629, in Simulations.dataframe(self)
    564         gotten = gotten.map(mapper)
    565     return gotten.rename(new_name)
    567 sims_df = SimulationsDataFrame(pd.concat((
    568     get(simulations, "reference_mass_ratio", floater),
    569     get(simulations, "reference_chi_eff", floater),
    570     get(simulations, "reference_chi1_perp", floater),
    571     get(simulations, "reference_chi2_perp", floater),
    572     get(simulations, "reference_eccentricity", floater),
    573     get(simulations, "reference_eccentricity", floaterbound, new_name="reference_eccentricity_bound"),
    574     get(simulations, "reference_time", floater),
    575     three_vector_dataframe(simulations, "reference_dimensionless_spin1"),
    576     three_vector_dataframe(simulations, "reference_dimensionless_spin2"),
    577     get(simulations, "reference_mean_anomaly", floater),
    578     three_vector_dataframe(simulations, "reference_orbital_frequency"),
    579     (
    580         get(simulations, "reference_position1", three_vec)
    581         - get(simulations, "reference_position2", three_vec)
    582     ).map(norm).rename("reference_separation"),
    583     get(simulations, "reference_position1", three_vec),
    584     get(simulations, "reference_position2", three_vec),
    585     get(simulations, "reference_mass1", floater),
    586     get(simulations, "reference_mass2", floater),
    587     get(simulations, "reference_dimensionless_spin1", norm, new_name="reference_chi1_mag"),
    588     get(simulations, "reference_dimensionless_spin2", norm, new_name="reference_chi2_mag"),
    589     get(simulations, "relaxation_time", floater),
    590     # get(simulations, "merger_time", floater),
    591     get(simulations, "common_horizon_time", floater),
    592     get(simulations, "remnant_mass", floater),
    593     three_vector_dataframe(simulations, "remnant_dimensionless_spin"),
    594     three_vector_dataframe(simulations, "remnant_velocity"),
    595     # get(simulations, "final_time", floater),
    596     get(simulations, "EOS", np.nan).fillna(get(simulations, "eos", np.nan)),
    597     get(simulations, "disk_mass", floater),
    598     get(simulations, "ejecta_mass", floater),
    599     get(simulations, "object_types", "").astype("category"),
    600     get(simulations, "initial_data_type", "").astype("category"),
    601     get(simulations, "initial_separation", floater),
    602     get(simulations, "initial_orbital_frequency", floater),
    603     get(simulations, "initial_adot", floater),
    604     get(simulations, "initial_ADM_energy", floater),
    605     three_vector_dataframe(simulations, "initial_ADM_linear_momentum"),
    606     three_vector_dataframe(simulations, "initial_ADM_angular_momentum"),
    607     get(simulations, "initial_mass1", floater),
    608     get(simulations, "initial_mass2", floater),
    609     get(simulations, "initial_mass_ratio", floater),
    610     three_vector_dataframe(simulations, "initial_dimensionless_spin1"),
    611     three_vector_dataframe(simulations, "initial_dimensionless_spin2"),
    612     get(simulations, "initial_position1", three_vec),
    613     get(simulations, "initial_position2", three_vec),
    614     # get(simulations, "object1", "").astype("category"),
    615     # get(simulations, "object2", "").astype("category"),
    616     # get(simulations, "url", ""),
    617     # get(simulations, "simulation_name", ""),
    618     # get(simulations, "alternative_names", []),
    619     # get(simulations, "metadata_path", ""),
    620     # get(simulations, "end_of_trajectory_time", floater),
    621     # get(simulations, "merger_time", floater),
    622     get(simulations, "number_of_orbits", floater),
    623     get(simulations, "number_of_orbits_from_start", floater),
    624     get(simulations, "number_of_orbits_from_reference_time", floater),
    625     get(simulations, "DOI_versions", []),
    626     get(simulations, "keywords", []),
    627     get(simulations, "date_link_earliest", datetime_from_string),
    628     get(simulations, "date_run_earliest", datetime_from_string),
--> 629     get(simulations, "date_run_latest", datetime_from_string),
    630     get(simulations, "date_postprocessing", datetime_from_string),
    631 ), axis=1))
    633 # If `tag` or `published_at` are present, add them as attributes
    634 if hasattr(self, "tag"):

File /opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/sxscatalog/simulations/simulations.py:564, in Simulations.dataframe.<locals>.get(df, col, mapper, new_name)
    562 gotten = df.get(col, default_series)
    563 if use_mapper:
--> 564     gotten = gotten.map(mapper)
    565 return gotten.rename(new_name)

File /opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/pandas/core/series.py:4675, in Series.map(self, func, na_action, engine, **kwargs)
   4673 if callable(func):
   4674     func = functools.partial(func, **kwargs)
-> 4675 new_values = self._map_values(func, na_action=na_action)
   4676 return self._constructor(new_values, index=self.index, copy=False).__finalize__(
   4677     self, method="map"
   4678 )

File /opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/pandas/core/base.py:1020, in IndexOpsMixin._map_values(self, mapper, na_action)
   1017 arr = self._values
   1019 if isinstance(arr, ExtensionArray):
-> 1020     return arr.map(mapper, na_action=na_action)
   1022 return algorithms.map_array(arr, mapper, na_action=na_action)

File /opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/pandas/core/arrays/base.py:2692, in ExtensionArray.map(self, mapper, na_action)
   2672 def map(self, mapper, na_action: Literal["ignore"] | None = None):
   2673     """
   2674     Map values using an input mapping or function.
   2675 
   (...)   2690         a MultiIndex will be returned.
   2691     """
-> 2692     return map_array(self, mapper, na_action=na_action)

File /opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/pandas/core/algorithms.py:1710, in map_array(arr, mapper, na_action)
   1708 values = arr.astype(object, copy=False)
   1709 if na_action is None:
-> 1710     return lib.map_infer(values, mapper)
   1711 else:
   1712     return lib.map_infer_mask(values, mapper, mask=isna(values).view(np.uint8))

File pandas/_libs/lib.pyx:3071, in pandas._libs.lib.map_infer()

File /opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/sxscatalog/utilities/string_converters.py:50, in datetime_from_string(x)
     48 except:
     49     pass  # No timezone information present; assuming UTC
---> 50 return dt.tz_localize(None)

KeyboardInterrupt:

RIT

The RIT catalog waveforms are downloaded via wget from the RIT website, rather than from a dedicated API. Horizon data is not available for RIT waveforms.

# Note: This requires having a RIT data path set up
# or will download the data if download=True
# For this example, we'll show the structure without executing

rit_id = 1096
wf_rit = rit.Waveform_RIT(path='./', ID=rit_id, 
                           download=True, nu_rescale=True)
print(f"RIT Waveform {rit_id} loaded")
print(f"Mass ratio q = {wf_rit.metadata['q']:.3f}")
print(f"Total mass M = {wf_rit.metadata['M']:.3f}")
print(f"Chi1z = {wf_rit.metadata['chi1z']:.3f}")
print(f"Chi2z = {wf_rit.metadata['chi2z']:.3f}")

CoRe database

CoRe database waveforms are downloaded from the gitlab public repository, using git-lfs.

For the download of THC waveforms, modify the code argument accordingly.

core_id = '0001'
wf_core = core.Waveform_CoRe(path='./', ID=core_id, download=True, code='BAM')

print(f"CoRe Waveform {core_id} loaded")
print(f"Mass ratio q = {wf_core.metadata['q']:.3f}")
print(f"Total mass M = {wf_core.metadata['M']:.3f}")
print(f"Chi1z = {wf_core.metadata['chi1z']:.3f}")
print(f"Chi2z = {wf_core.metadata['chi2z']:.3f}")

ICCUB (public repository)

wf_icc = icc_public.Waveform_ICC(path='./', ID='0001', download=True, ellmax=4)

print(f"ICCUB Waveform 0001 loaded")
print(f"Mass ratio q = {wf_icc.metadata['q']:.3f}")
print(f"Total mass M = {wf_icc.metadata['M']:.3f}")
print(f"Chi1z = {wf_icc.metadata['chi1z']:.3f}")
print(f"Chi2z = {wf_icc.metadata['chi2z']:.3f}")

GR-Athena++

TODO: implement and demonstrate

Using the Cataloger for Bulk Operations

The Cataloger class provides a convenient way to work with multiple simulations from a catalog at once. This is particularly useful for computing mismatches or other comparative analyses across a set of simulations.

Here’s an example structure:

# Example: Working with multiple RIT simulations
sim_list = list(range(1096, 1100))  # List of simulation IDs

cat = Cataloger(
    path='./local_data/rit/', 
    catalog='rit', 
    sim_list=sim_list,
    add_opts={'download': True, 'nu_rescale': True}
)

# Plot all waveforms in the catalog
cat.plot_waves()

Comparing Waveforms Between Catalogs

One of PyART’s strengths is the ability to easily compare waveforms from different catalogs. Since all catalogs use the same interface, you can load waveforms from different sources and compare them directly.

For detailed mismatch calculations between catalogs, see the Mode-by-mode Mismatch tutorial.

Summary

This tutorial covered:

  • How to download waveforms from the SXS catalog

  • How to access waveform metadata

  • How to compute and plot waveform modes

  • The structure for working with other catalogs (RIT, CoRe, etc.)

  • Using the Cataloger class for bulk operations

Next Steps