Transforming Combustion Science and Technology with Exascale Simulations

A Conversation with Jackie Chen of Sandia National Laboratories

Jackie Chen of Sandia National Laboratories leads a project called Combustion-Pele. ECP Communications chatted with her recently about this research, which involves predictive simulation of in-cylinder combustion processes to explore the potential for groundbreaking efficiencies while limiting the formation of pollutants. This is an edited transcript.

Combustion-based systems are not going away any time soon. They will dominate the marketplace for decades. How is the Combustion-Pele project endeavoring to optimize the systems for energy efficiency and reduced emissions?

Pele is an exascale combustion project that we are working on, and we are developing a suite of high-fidelity simulation tools, tools that will allow us to look with unprecedented detail at some of the underlying combustion processes that occur in practical devices that we all use every day, to transport goods or people, and [for] airplanes, and to create electricity. With this better understanding of the underlying combustion processes, we can develop predictive models that engine designers can use to design more fuel-efficient, cleaner-burning engines using not only our traditional hydrocarbon-based fuels but also renewable fuels.

How do the nation’s 2020 greenhouse gas and 2030 carbon emission goals factor into what you’re doing?

Well, these government regulations certainly give a sense of urgency to developing the simulation capability—the high-fidelity simulation capabilities for combustion science. So we plan to use the science computations at the exascale to clarify some of the underlying combustion challenges related to engine efficiency and emissions. As you know, to achieve lower levels of CO₂ emissions you need to increase engine efficiency. In IC engines better strategies for mixture formation in spray combustion are needed that control transient ignition and combustion processes, and also influence soot generation. Increasing pressure and burning at lower temperatures will help, but chemistry and it’s coupling with transport processes is not well understood in those regimes. There’s also an increasing emphasis on renewable energy—wind, solar, and biomass—as a way to generate carbon-free electricity. But intermittency poses a challenge—if the sun doesn’t shine or the wind doesn’t blow, you’re not going to get power from these sources. Challenges resulting from intermittency may be mitigated by blending fuels that contain hydrogen with more conventional fuels like natural gas for use both in baseload as well as in demand-matching power generation. And this will occur through better understanding of the underlying combustion processes in a new generation of more responsive, fuel-flexible gas turbines, both flexible in terms of responding to changing load demands as well as providing flexibility with different fuels. Some of the technical challenges that we hope to address with exascale simulations include understanding flame instabilities, flashback, lean blow off, and how to mitigate increases in nitric oxide emissions caused by higher combustion temperatures. With the set of detailed simulation tools, we will try to address some of these research questions that impact future designs.

Of course, asking the right questions is at the heart of the scientific method. What key questions are driving your research?

Some key questions that are driving our research involve interactions between turbulence, turbulent mixing, and chemistry. So almost all practical combustors operate under extremely high turbulence levels to increase the rate of combustion providing high efficiency, but there are still outstanding challenges in understanding how turbulence affects auto-ignition, meaning how a fuel mixture spontaneously ignites as the mixture is isentropically compressed by the piston in an engine to the point where low-temperature reactions commence, as it would in a diesel engine, for example. You want ignition and subsequent combustion to occur in the right phasing of the piston motion as it moves up and down in your engine cylinders to provide the greatest amount of efficiency and while minimizing soot emissions. And so understanding how to prepare the optimum fuel/air mixture in a turbulent environment, that is, providing the right combinations of fuel and temperature stratification tailored to the chemical and physical properties of a given fuel composition are challenges that high-fidelity simulations can address with unprecedented detail not available to experimentation. Many of these fuels are liquid hydrocarbon fuels that are injected into heated ambient diluted air at between 750 to 1000K at high pressure, and through entrainment and mixing, undergoes atomization and evaporation upstream of the combustion process—these are further multi-physics challenges that we need to understand with simulation.

And because engines operate under extremely adverse environments at very, very high pressures—upwards of 100 atmospheres sometimes and temperatures in excess of 2,000 degrees—it’s very difficult to find out what’s going on inside of the engine only through experimentation because of the limited diagnostics at those extreme conditions. And so we are increasingly relying on computation—you can imagine inserting a small probe into an engine in which you can numerically measure everything including the chemical species concentrations, temperature, pressure as well as the flow inside—complex turbulent flow inside of the cylinder and how all of this interacts in time, to affect spontaneous ignition occurring in multiple stages, soot generation, flame propagation and soot generation. Once transient auto-ignition has occurred in the cylinder it generates thin flame fronts that propagate upstream to a point where the flame is stabilized downstream of the fuel injector by the oncoming turbulent flow containing low-temperature ignition intermediate species. And so the dynamics of mixing, ignition, flame stabilization and soot formation are intertwined processes we can probe with high-fidelity simulation. We’ve not been able to do this in the past because computers simply weren’t big enough and the algorithms that we used were not sophisticated enough to treat highly non-linear, multi-phase combustion phenomenon.

What are some of the simulation challenges that you and your team are having to address?

Our team has a number of challenges that we’re trying to address from the algorithmic, physics, and computational standpoints. Trying to solve the non-linear Navier–Stokes equations coupled with conservation equations for species and total energy for large numbers of species [10’s to 100]—results in large systems of partial differential equations with large nonlinearities, and we’re tackling this with adaptive mesh refinement [AMR], to place grids where there are steep flow or combustion induced gradients. With AMR we have to worry about load balancing and communication issues across large networks on heterogeneous architectures at the exascale. We’re designing communication-avoiding linear solvers that allow us to trade communication for computation. We’re working with computer scientists to develop programming model strategies for asynchronous task execution and also including in situ analysis and visualization methods, because the amount of data that we’re going to generate at the exascale will be huge. We can’t afford to write all of the data out to I/O, i.e., onto mechanical disks, which is very slow.

In addition, we also have challenges with introducing a set of physical processes requiring a hybrid particle-in-cell [PIC] approach wherein Lagrangian particles couple with an Eulerian block structured adaptive mesh refinement capability. These physical processes include polydisperse sprays, soot, and thermal radiation with real gas effects present at very high pressures that future engines will operate in. We want to maximize the fidelity of the physics models at the same time keeping the problem tractable at the exascale.

Lastly, currently at the petascale we can approach moderately high pressures in very simple, canonical geometries, but we really can’t deal with the complex geometries that are typical of practical combustors. So in this project we’re adding an embedded boundary method capability to treat nontrivial geometry in the context of these complicated linear solvers. This capability will add another element of realism to our combustion simulations.

Has your team taken advantage of any of ECP’s allocations of computer time?

Yes, we have taken advantage of ECP’s computer allocations under the ALCC [Advanced Scientific Computing Research Leadership Computing Challenge] program. On Titan, we ran a large calculation of a non-reacting turbulent acetone spray jet on a uniform mesh. This served as a validation exercise to ensure that the models that we’ve implemented for our spray treatment are accurate, and experimental results exist that we compared our numerical simulations against. Once we were convinced that the spray models are accurate and correctly implemented, then we implemented them in the Pele adaptive mesh refinement codes. We’ve also used the access to computing time on Cori, Edison, and Titan to do numerical benchmarking exercises to provide a baseline metric for our figure of merit, against which we can calibrate future additions of multi-physics as well as code improvement capabilities.

How would you sum up the project’s milestone accomplishments to date?

The project has been very successful at meeting all of the milestones to date, and so we’re very pleased with our progress. We’ve added a chemistry augmentation to provide drop-in chemical models, derived from theoretical chemistry, for our combustion simulations. So it’s full steam ahead. It’s almost 2 years since the project started, and we’ve cobbled together enough new code capabilities that we can start to demonstrate some cool new science towards our ultimate target problem of simulating the conditions inside of an IC engine or gas turbine at the exascale.

What do you anticipate will be the ultimate outcomes and impacts of your efforts?

The ultimate impact of our work will be the development of an exascale-ready high-fidelity combustion simulation suite of codes, i.e., low-Mach and compressible adaptive mesh refinement codes that accurately treat real gas effects, sprays, soot, and thermal radiation in real geometries. A second impact is the development of a framework for incorporating automated chemical mechanism reduction [based on theoretical chemistry with quantified uncertainties] that will enable drop-in chemistry tailored to specific science problems that we want to explore—i.e., the science underpinning the design of next-generation, fuel-efficient, clean-burning engines.

What new collaborations or partnerships have you formed through the ECP?

Our strongest collaboration is with the AMReX co-design center, which is providing us with the underlying adaptive mesh refinement capabilities that we’re using in both our low-Mach as well as compressible combustion codes. We’re also working with some of the visualization and software stack teams, including the Alpine visualization team and the Legion programming models team. Collaboration with the Legion team has enabled us to assess the ability of asynchronous dynamic task-based programming models to hide latencies on the node and across the network on heterogeneous machines, affecting the time to solution and scalability of the code in both weak and strong limits.

With a little reflection, what do you think are some of the immeasurable, intangible benefits of being part of the ECP?

One of the immeasurable, intangible benefits of being part of the ECP is the connection with other researchers—computer scientists, computational scientists, and applied mathematicians—that have a common goal. The interdisciplinary co-design approach is necessary to get to the exascale as the problem is too complex for a single PI to solve. And so I think the interaction with people with complementary expertise is definitely the greatest benefit of being part of the ECP.

What’s on the horizon for the Combustion-Pele project for the rest of 2018?

On the horizon for the Pele project in 2018 is to complete the compressible Pele AMR code including the embedded boundary and spray capabilities. We will run performance benchmarks and perform some science runs highlighting these new capabilities, hopefully by the end of the year.