Downloading Waveforms from Different Catalogs

This tutorial demonstrates how to download and work with waveforms from various numerical relativity catalogs using PyART. PyART provides a unified interface to access data from multiple catalogs including:

  • SXS (Simulating eXtreme Spacetimes)

  • RIT (Rochester Institute of Technology)

  • CoRe (Computational Relativity)

  • GRA (GR-Athena++)

  • ICCUB (Institute of Cosmos Sciences)

  • SACRA

Setup

First, let’s import the necessary modules:

%matplotlib inline
%config InlineBackend.figure_format = 'retina'

from PyART.catalogs import sxs, rit, core, icc_public
from PyART.catalogs.cataloger import Cataloger

/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/PyART/analysis/match.py:15: UserWarning: Wswiglal-redir-stdio:

SWIGLAL standard output/error redirection is enabled in IPython.
This may lead to performance penalties. To disable locally, use:

with lal.no_swig_redirect_standard_output_error():
    ...

To disable globally, use:

lal.swig_redirect_standard_output_error(False)

Note however that this will likely lead to error messages from
LAL functions being either misdirected or lost when called from
Jupyter notebooks.

To suppress this warning, use:

import warnings
warnings.filterwarnings("ignore", "Wswiglal-redir-stdio")
import lal

  import lal

WARNING: TEOBResumS not installed.

Downloading from a Single Catalog

Let’s start by downloading waveforms from the SXS catalog. We specify:

  • The catalog name

  • The simulation ID

  • Whether to download if not locally available

SXS

# Download and load an SXS waveform
# Note: You may need to adjust the path or set download=True
sxs_id = '0180'
wf_sxs = sxs.Waveform_SXS(path='./', ID=sxs_id, download=True, cut_N=300, ignore_deprecation=True)

print(f"SXS Waveform {sxs_id} loaded successfully")
print(f"Mass ratio q = {wf_sxs.metadata['q']:.3f}")
print(f"Total mass M = {wf_sxs.metadata['M']:.3f}")
print(f"Chi1z = {wf_sxs.metadata['chi1z']:.3f}")
print(f"Chi2z = {wf_sxs.metadata['chi2z']:.3f}")

---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
Cell In[2], line 4
      1 # Download and load an SXS waveform
      2 # Note: You may need to adjust the path or set download=True
      3 sxs_id = '0180'
----> 4 wf_sxs = sxs.Waveform_SXS(path='./', ID=sxs_id, download=True, cut_N=300, ignore_deprecation=True)
      6 print(f"SXS Waveform {sxs_id} loaded successfully")
      7 print(f"Mass ratio q = {wf_sxs.metadata['q']:.3f}")

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/PyART/catalogs/sxs.py:135, in Waveform_SXS.__init__(self, path, ID, order, level, cut_N, cut_U, ellmax, load, download, downloads, load_m0, nu_rescale, src, ignore_deprecation, basename)
    131     logging.info(
    132         f"The path {self.sxs_data_path} does not exist or contains no 'Lev*' directory."
    133     )
    134     logging.info("Downloading the simulation from the SXS catalog.")
--> 135     self.download_simulation(
    136         ID=self.ID,
    137         path=path,
    138         downloads=downloads,
    139         level=self.level,
    140         ignore_deprecation=ignore_deprecation,
    141         extrapolation_order=order,
    142     )
    143 else:
    144     logging.warning(
    145         "Use download=True to download the simulation from the SXS catalog."
    146     )

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/PyART/catalogs/sxs.py:290, in Waveform_SXS.download_simulation(self, ID, path, downloads, level, ignore_deprecation, extrapolation_order)
    288 sys.stdout = open(os.devnull, "w")
    289 sys.stderr = open(os.devnull, "w")
--> 290 sxs_sim = sxsmod.load(
    291     name_level,
    292     extrapolation_order=extrapolation_order,
    293     ignore_deprecation=ignore_deprecation,
    294     progress=True,
    295 )
    296 logging.info(f"Loaded SXS simulation {name_level}.")
    298 # Set Level if not already set

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/sxs/handlers.py:288, in load(location, download, cache, progress, truepath, **kwargs)
    285     return sxscatalog.load(location, download=download, **kwargs)
    287 elif sxs_id_version_lev_exact_re.match(location):
--> 288     return Simulation(location, download=download, cache=cache, progress=progress, **kwargs)
    290 else:
    291     # Try to find an appropriate SXS file in the simulations
    292     simulations = Simulations.load(
    293         download=download,
    294         local=kwargs.get("local", False),
    295         annex_dir=kwargs.get("annex_dir", None)
    296     )

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/sxs/simulations/simulation.py:135, in Simulation(location, *args, **kwargs)
    132 from ..metadata.metric import MetadataMetric
    134 # Load the simulation catalog
--> 135 simulations = load("simulations")
    136 v = Version(simulations.tag)
    137 latest_version = f"v{v.major}.{v.minor}"

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/sxs/handlers.py:285, in load(location, download, cache, progress, truepath, **kwargs)
    282     return Catalog.load(download=download)
    284 elif location in ["simulations", "dataframe"]:
--> 285     return sxscatalog.load(location, download=download, **kwargs)
    287 elif sxs_id_version_lev_exact_re.match(location):
    288     return Simulation(location, download=download, cache=cache, progress=progress, **kwargs)

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/sxscatalog/__init__.py:16, in load(location, download, **kwargs)
     13 from .simulations import Simulations
     15 if location == "simulations":
---> 16     return Simulations.load(
     17         download=download,
     18         tag=kwargs.get("tag", ""),
     19         local=kwargs.get("local", False),
     20         annex_dir=kwargs.get("annex_dir", None),
     21         output_file=kwargs.get("output_file", None),
     22         compute_md5=kwargs.get("compute_md5", False),
     23         show_progress=kwargs.get("show_progress", False),
     24         ignore_cached=kwargs.get("ignore_cached", False),
     25     )
     27 elif location == "dataframe":
     28     return load("simulations", download=download, **kwargs).dataframe

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/sxscatalog/simulations/simulations.py:406, in Simulations.load(cls, download, tag, local, annex_dir, output_file, compute_md5, show_progress, ignore_cached)
    404 temp_zip = cache_path.with_suffix(".temp.bz2")
    405 try:
--> 406     download_file(cls.url.format(tag=tag), temp_json, progress=progress, if_newer=False)
    407 except Exception as e:
    408     raise RuntimeError(
    409         f"\nFailed to download '{cls.url.format(tag=tag)}'."
    410         f"\nMaybe {tag=} does not exist?"
    411     ) from e

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/sxscatalog/utilities/downloads.py:67, in download_file(url, path, progress, if_newer)
     64     if token:
     65         session.headers.update({"Authorization": f"token {token}"})
---> 67 r = session.get(url, stream=True, allow_redirects=True)
     68 if r.status_code != 200:
     69     print(f"An error occurred when trying to access <{url}>.")

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/requests/sessions.py:602, in Session.get(self, url, **kwargs)
    594 r"""Sends a GET request. Returns :class:`Response` object.
    595 
    596 :param url: URL for the new :class:`Request` object.
    597 :param \*\*kwargs: Optional arguments that ``request`` takes.
    598 :rtype: requests.Response
    599 """
    601 kwargs.setdefault("allow_redirects", True)
--> 602 return self.request("GET", url, **kwargs)

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/requests/sessions.py:589, in Session.request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    584 send_kwargs = {
    585     "timeout": timeout,
    586     "allow_redirects": allow_redirects,
    587 }
    588 send_kwargs.update(settings)
--> 589 resp = self.send(prep, **send_kwargs)
    591 return resp

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/requests/sessions.py:703, in Session.send(self, request, **kwargs)
    700 start = preferred_clock()
    702 # Send the request
--> 703 r = adapter.send(request, **kwargs)
    705 # Total elapsed time of the request (approximately)
    706 elapsed = preferred_clock() - start

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/requests/adapters.py:644, in HTTPAdapter.send(self, request, stream, timeout, verify, cert, proxies)
    641     timeout = TimeoutSauce(connect=timeout, read=timeout)
    643 try:
--> 644     resp = conn.urlopen(
    645         method=request.method,
    646         url=url,
    647         body=request.body,
    648         headers=request.headers,
    649         redirect=False,
    650         assert_same_host=False,
    651         preload_content=False,
    652         decode_content=False,
    653         retries=self.max_retries,
    654         timeout=timeout,
    655         chunked=chunked,
    656     )
    658 except (ProtocolError, OSError) as err:
    659     raise ConnectionError(err, request=request)

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/urllib3/connectionpool.py:787, in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, preload_content, decode_content, **response_kw)
    784 response_conn = conn if not release_conn else None
    786 # Make the request on the HTTPConnection object
--> 787 response = self._make_request(
    788     conn,
    789     method,
    790     url,
    791     timeout=timeout_obj,
    792     body=body,
    793     headers=headers,
    794     chunked=chunked,
    795     retries=retries,
    796     response_conn=response_conn,
    797     preload_content=preload_content,
    798     decode_content=decode_content,
    799     **response_kw,
    800 )
    802 # Everything went great!
    803 clean_exit = True

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/urllib3/connectionpool.py:534, in HTTPConnectionPool._make_request(self, conn, method, url, body, headers, retries, timeout, chunked, response_conn, preload_content, decode_content, enforce_content_length)
    532 # Receive the response from the server
    533 try:
--> 534     response = conn.getresponse()
    535 except (BaseSSLError, OSError) as e:
    536     self._raise_timeout(err=e, url=url, timeout_value=read_timeout)

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/urllib3/connection.py:571, in HTTPConnection.getresponse(self)
    568 _shutdown = getattr(self.sock, "shutdown", None)
    570 # Get the response from http.client.HTTPConnection
--> 571 httplib_response = super().getresponse()
    573 try:
    574     assert_header_parsing(httplib_response.msg)

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/http/client.py:1415, in HTTPConnection.getresponse(self)
   1413 try:
   1414     try:
-> 1415         response.begin()
   1416     except ConnectionError:
   1417         self.close()

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/http/client.py:330, in HTTPResponse.begin(self)
    328 # read until we get a non-100 response
    329 while True:
--> 330     version, status, reason = self._read_status()
    331     if status != CONTINUE:
    332         break

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/http/client.py:291, in HTTPResponse._read_status(self)
    290 def _read_status(self):
--> 291     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
    292     if len(line) > _MAXLINE:
    293         raise LineTooLong("status line")

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/socket.py:718, in SocketIO.readinto(self, b)
    716 while True:
    717     try:
--> 718         return self._sock.recv_into(b)
    719     except timeout:
    720         self._timeout_occurred = True

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/ssl.py:1314, in SSLSocket.recv_into(self, buffer, nbytes, flags)
   1310     if flags != 0:
   1311         raise ValueError(
   1312           "non-zero flags not allowed in calls to recv_into() on %s" %
   1313           self.__class__)
-> 1314     return self.read(nbytes, buffer)
   1315 else:
   1316     return super().recv_into(buffer, nbytes, flags)

File /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/ssl.py:1166, in SSLSocket.read(self, len, buffer)
   1164 try:
   1165     if buffer is not None:
-> 1166         return self._sslobj.read(len, buffer)
   1167     else:
   1168         return self._sslobj.read(len)

KeyboardInterrupt:

RIT

The RIT catalog waveforms are downloaded via wget from the RIT website, rather than from a dedicated API. Horizon data is not available for RIT waveforms.

# Note: This requires having a RIT data path set up
# or will download the data if download=True
# For this example, we'll show the structure without executing

rit_id = 1096
wf_rit = rit.Waveform_RIT(path='./', ID=rit_id, 
                           download=True, nu_rescale=True)
print(f"RIT Waveform {rit_id} loaded")
print(f"Mass ratio q = {wf_rit.metadata['q']:.3f}")
print(f"Total mass M = {wf_rit.metadata['M']:.3f}")
print(f"Chi1z = {wf_rit.metadata['chi1z']:.3f}")
print(f"Chi2z = {wf_rit.metadata['chi2z']:.3f}")

CoRe database

CoRe database waveforms are downloaded from the gitlab public repository, using git-lfs.

For the download of THC waveforms, modify the code argument accordingly.

core_id = '0001'
wf_core = core.Waveform_CoRe(path='./', ID=core_id, download=True, code='BAM')

print(f"CoRe Waveform {core_id} loaded")
print(f"Mass ratio q = {wf_core.metadata['q']:.3f}")
print(f"Total mass M = {wf_core.metadata['M']:.3f}")
print(f"Chi1z = {wf_core.metadata['chi1z']:.3f}")
print(f"Chi2z = {wf_core.metadata['chi2z']:.3f}")

ICCUB (public repository)

wf_icc = icc_public.Waveform_ICC(path='./', ID='0001', download=True, ellmax=4)

print(f"ICCUB Waveform 0001 loaded")
print(f"Mass ratio q = {wf_icc.metadata['q']:.3f}")
print(f"Total mass M = {wf_icc.metadata['M']:.3f}")
print(f"Chi1z = {wf_icc.metadata['chi1z']:.3f}")
print(f"Chi2z = {wf_icc.metadata['chi2z']:.3f}")

GR-Athena++

TODO: implement and demonstrate

Using the Cataloger for Bulk Operations

The Cataloger class provides a convenient way to work with multiple simulations from a catalog at once. This is particularly useful for computing mismatches or other comparative analyses across a set of simulations.

Here’s an example structure:

# Example: Working with multiple RIT simulations
sim_list = list(range(1096, 1100))  # List of simulation IDs

cat = Cataloger(
    path='./local_data/rit/', 
    catalog='rit', 
    sim_list=sim_list,
    add_opts={'download': True, 'nu_rescale': True}
)

# Plot all waveforms in the catalog
cat.plot_waves()

Comparing Waveforms Between Catalogs

One of PyART’s strengths is the ability to easily compare waveforms from different catalogs. Since all catalogs use the same interface, you can load waveforms from different sources and compare them directly.

For detailed mismatch calculations between catalogs, see the Mode-by-mode Mismatch tutorial.

Summary

This tutorial covered:

  • How to download waveforms from the SXS catalog

  • How to access waveform metadata

  • How to compute and plot waveform modes

  • The structure for working with other catalogs (RIT, CoRe, etc.)

  • Using the Cataloger class for bulk operations

Next Steps