Astronomical observations have confirmed that the heavy elements in the universe were synthesized in stars and stellar explosions, yet many questions concerning how and where these elements formed remain outside the purview of direct observation. Through the ExaStar multiphysics codes, exascale computing will be used to simulate supernova explosions and neutron star mergers, addressing fundamental questions in astrophysics, such as the origin of elements, the physics of ultra-dense matter, and the sources of gravitational waves.
Summary
Exploding stars, known as supernova, that ripped through the universe before our sun was a star are regarded as a source of most of the heavy elements in the known universe by ejecting them into space with the violent force of nuclear blasts. Yet the process of how massive stars collapse after burning all their fuel until they explode is still largely unknown, and there remain many open questions of fundamental physics. These include questions about how ultra-dense matter, strong gravity, and energetic neutrinos behave under extreme conditions. But the next generation of supercomputing, exascale computing, is providing a new way to probe physics at these extremes.
An exploding star brings together a lot of complicated physics into one problem: Fluid mechanics, particle physics, and the mystery of how neutrinos drive the explosion and accompanying shock wave all have to come together to solve sets of equations at the same time to make a simulation that captures all the violence and turbulence of an exploding star. That challenge, which has taken seven years on the world’s fastest supercomputer, is showing how stardust becomes heavy elements like iron and titanium and particles that populate Earth, its inhabitants, and the known universe.
ExaStar, as the simulation program is called, required major overhauls in developing codes of new physics modules that solve problems related to Einstein’s theory of general relativity (rest assured, it remains intact), and many of the known equations of classic physics. Among the largest challenges was integrating all the code into GPU architecture, as that is what the Frontier supercomputer uses at the exascale. Developing new physics codes and having them run on GPUs was a major step forward: The team successfully implemented GPU code for almost all multiphysics modules, including neutrino transport, general relativistic gravity, nuclear kinetics, and hydrodynamics.
The simulation encloses the helium shell of the star, which has a radius around 10,000 km, with a maximum spatial resolution finer than 1 km in the innermost regions. At least 20 energy groups were used to resolve the spectra of neutrinos of all “flavors” (i.e., electron, mu, tau, and their antiparticles) from 0 to 300 MeV. It produced a detailed set of neutrino-matter interaction rates that are integrated with a high-density equation of state that provides pressures, entropies, and other thermodynamic variables. Through a better understanding of the locations in which the heaviest elements are made, ExaStar can help focus experimental efforts on the Facility for Rare Isotope Beams and other science facilities that are now a priority of the next decade of nuclear physics.
ExaStar now has multiple 3D simulations running of an exploding star. The star’s core collapses into a little nugget, but the rest of the star is gigantic, so the algorithms are quite complex, and the equations complicated. The goal is to have an entire suite of simulations that physicists can use to understand how such a star explodes, and perhaps answer longstanding questions of how the heaviest elements in the universe were born and thrust into space. The complication is that there are many different types of stars, and the code hopes to simulate how they turn into different types of supernovae and produce the different elements that swarm the universe. It turns out it takes a lot of work to build codes with that kind of capability. The software must simulate capture behavior of matter at very high density and calculate the radiation transport of neutrinos. Predicting signals of such major events requires massive complexity in the software, which the team has completed. Thus exascale computing through the ExaStar suite is poised to answer some of the most fundamental questions of the universe, and perhaps the most basic of all – where did we come from?
Technical Discussion
ExaStar focuses on developing a new component-based multiphysics code suite with adaptive mesh refinement (AMR) that can accurately simulate coupled hydrodynamics, radiation transport, thermonuclear kinetics, and nuclear microphysics for stellar explosion simulations. The code will reach exascale efficiency by building upon current multicore and many-core efficient local physics packages integrated into a task-based asynchronous execution framework by using current AMR technology. The fundamental goal is to understand the production of the heavier chemical elements produced and expelled in explosions, such a supernovae and neutron star mergers.
Although astronomical observations reveal that the production of the heaviest nuclei began early in galactic history, it is unknown exactly how and where these elements were formed. To address this topic via laboratory measurements, a series of nuclear science long-range plans have supported the construction of radioactive ion beam facilities, culminating in the Facility for Rare Isotope Beams (FRIB). Although FRIB is designed to acquire extensive data on the nuclei relevant for astrophysical nucleosynthesis, its end science goal cannot be met unless those experimental data are integrated into high-fidelity simulations of stellar explosions, such as supernovae and neutron star mergers, that define the conditions under which such heavy element production occurs. Through a better understanding of the sites in which the heaviest elements are made, ExaStar can help focus experimental efforts at FRIB on the reactions of greatest influence.
Simulating stellar explosions represents a quintessential multiphysics challenge. The hydrodynamics of the astrophysical gas must include a specialized equation of state, capturing the behavior of matter at extreme density. The impact of gravity should ideally be treated by solving Einstein’s equations of general relativity. Accurately calculating the radiation transport of neutrinos is crucial for driving the explosion shockwave and setting the neutron-to-proton ratio that determines the nucleosynthetic yields. To determine those yields, in situ nuclear kinetics with ideally thousands of isotopes must follow the complex flow of reactions. Finally, predicting the observable signals of the event (i.e., photons, neutrinos, and gravitational waves) requires the simulation to be postprocessed with multi-angle spectral radiation transport.
The ExaStar base challenge problem is a 3D simulation of the explosion of a massive star in a core-collapse supernova. The initial conditions will be taken from stellar evolution models of a late-stage star at the point of iron core collapse. The physical domain of the simulation will enclose the helium shell of the star, which has a radius around 10,000 km, with a maximum spatial resolution enabled by AMR that is finer than 1 km in the innermost regions. At least 20 energy groups will be used to resolve the spectra of neutrinos of all flavors (i.e., electron, mu, tau, and their antiparticles) from 0 to 300 MeV. A detailed set of tabulated neutrino-matter interaction rates will be used to model emission, absorption, scattering, and pair production processes and will be integrated with a high-density equation of state that provides pressures, entropies, and other thermodynamic variables. A Newtonian multi-pole solver will be available to simulate gravity with the potential to upgrade this with a full general relativistic dynamical space-time solver.
In Figure 1, Hot material roils around a newly-born neutron star at the center of a core-collapse supernova. Shown is the matter entropy from a three-dimensional simulation of the first half second of the explosion. Driven by intense heating from neutrinos coming from the cooling neutron star at the center, the shock wave will be driven outward, eventually ripping the star apart and flinging the elements that make up our solar system and ourselves into interstellar space.
Figure 1: ExaStar 3D simulation of a Core-collapse supernova performed using the ORNL Chimera multiphysics code
