Advancing the Additive Manufacturing Revolution

ExaAM’s collection of optimized simulation codes brings exascale power to predicting how entire metal parts can be 3D printed by industry

By Coury Turczyn
Oak Ridge National Laboratory

The methods used to fabricate metal parts have changed remarkably little in the past few centuries. Although machining technology has certainly advanced, it still follows basic metalworking methods to produce the parts in our vehicles, our appliances, our buildings, and more: pour metal into a mold, carve it away from a block, stamp it, extrude it, or roll it. But all those legacy processes introduce limitations to the metal parts being made—from their potential geometries to their physical properties.

What if a new approach could create metal parts in shapes so complex that they’d otherwise be impossible to construct? What if their microstructures were so accurately designed and fabricated that different sections of a single part could yield different material properties?

That scenario is no longer hypothetical with the advent of additive manufacturing (AM). Rather than taking a piece of metal and reforming it into something else, AM fabricators make parts from scratch by sequentially adding layers of liquid metal into patterns until they form 3D shapes that solidify as they cool. If that sounds like a consumer-grade 3D printer, that’s because the basic technology is the same. However, taken to an industrial scale, this fabrication method can go far beyond printing plastic models on your desktop. AM heralds a new era of precision manufacturing that could make a big impact on the aerospace, biomedical, energy, and automotive industries, where it has already been making inroads.

The ExaAM project, which is a collection of simulation codes optimized for exascale-class supercomputers (i.e., 1 quintillion or more calculations per second), aims to advance AM as a viable method for industrial manufacturing. Supported by the US Department of Energy’s (DOE’s) Exascale Computing Project (ECP), ExaAM consists of a team of researchers at DOE’s Oak Ridge National Laboratory (ORNL), Lawrence Livermore National Laboratory, Los Alamos National Laboratory, the National Institute of Standards and Technology, and several academic collaborators. The team has developed the ExaAM tool set to provide insights into how metals behave throughout the AM process (i.e., melting, solidifying) and how they perform. This is critical information for designing new parts as well as gauging their reliability and functionality.

ECP's ExaAM project team in front of a screen from ExaAM

The ECP’s ExaAM team gathers in front of a screen from ExaAM, a simulation toolset for exascale supercomputers that can model entire metal parts being constructed through the 3D printing process. From left, standing: Sam Reeve, Alex Plotkowski, Matt Bement, Stephen DeWitt, Austin Isner, Matt Rolchigo. From left, sitting: Gerry Knapp, Adrian Sabau, Philip Fackler. Photo by Kelley Smith/ORNL.

“The purpose of ExaAM is to model the AM manufacturing process—you’ve got heat being deposited, metal melting, metal solidifying, and then you’re going to the next layer with more metal remelting and then solidifying. So, you get all this thermal cycling, which makes for a very complicated process,” said Matt Bement, head of ORNL’s Scalable Algorithms and Coupled Physics group and the project’s principal investigator.

“Being able to understand how that process gives you a microstructure and what that microstructure tells you about its properties—that’s what we’re trying to do,” Bement said. “If you can do that accurately, then you could start printing 3D parts and qualify them for critical missions because you really do understand what it is you just made.”

Predicting Entire Parts Before They’re Made

3D printing technologies have existed for a few decades, but metal AM has taken longer to mature due to its complexity—metal is much more complicated to print than plastic. When it cools and solidifies, crystals within the metal align to form microstructures that determine its properties as a material, which will ultimately affect the functionality of the part.

“Everything we know about how most parts have historically been made doesn’t help us as much as we would like with planning an AM part. The microstructure that you get from an AM part is very different from traditionally manufactured parts,” Bement said. “How the metallic crystals end up packing themselves together during AM makes a really big difference in what that metallic part is going to do—how it’s going to react to fatigue loading, how it’s going to react to impact loading, and how it’s going to react to different temperatures.”

And that’s where ExaAM comes in: it models how these parts can be precisely constructed through AM. It achieves this by combining individual software tools that each simulate different stages of AM to collectively capture the relevant physics of the process.

Some of the tools have been adapted from preexisting codes, such as the open-source fluid dynamics simulator OpenFOAM, which the team extended to model AM’s melting stage and redubbed as additiveFOAM. Solidification and its resulting microstructures can be simulated with a few different tools: ExaCA provides a fast-running but lower-accuracy simulation of solidification, whereas the phase-field codes AMPE, MEUMAPPS, and Tusas provide higher accuracy but more computationally expensive solidification modeling. ExaConstit handles predictions of mechanical properties, and Diablo provides a simulation of the whole part.

Overcoming Timescales

To model an entire part that’s constructed through AM, the team had to overcome a major computational hurdle: the time difference between the physical processes being simulated (i.e., melting, solidifying) vs. the actual time it takes to construct a part with an AM fabricator.

“One of the big challenges of doing a whole simulation is that things that happen in the melt pool are measured in microseconds. But if you were to sit there and watch the printer build the whole part, it would take hours,” Bement said. “There are a lot of microseconds in hours, so the sheer number of timesteps that you have to take becomes an enormous problem.”

ORNL researchers Matt Bement (left) and Alex Plotkowski

ORNL researchers Matt Bement (left) and Alex Plotkowski lead the effort to take the ExaAM simulation toolset into the industrial world of additive manufacturing. They are standing in front of MedUSA, located at ORNL’s Manufacturing Demonstration Facility. MedUSA is a hybrid manufacturing machine that enables both additive and subtractive manufacturing capabilities in one unique system. Photo by Kelley Smith/ORNL.

A similar computational issue was one of physical scale. AM’s melting and solidifying processes occur in a length scale of a few microns, but the team’s ultimate goal for ExaAM is to simulate large, whole parts.

“I don’t want to simulate something that’s just a few microns across or even a few hundred microns. I want something that’s centimeters long, a part that you can hold in your hand. So, we had to find a couple of ways to adapt those things,” said Alex Plotkowski, an R&D staff member of ORNL’s Materials Science and Technology group and researcher at ORNL’s Manufacturing Demonstration Facility (MDF).

The team had to devise a way to parallelize the time domain and scale up the simulations to handle entire parts—and they succeeded.

“We developed a time-parallel approach to take those types of simulations and scale them to very large supercomputers. That gives us much larger scale, and that’s one of the reasons why it’s well suited for the ECP as a whole—we can make good use of those large computing resources to be able to do these problems at a scale that is maybe not feasible with more conventional computing platforms,” Plotkowski said.

Creating a predictive model of a whole AM part—not just sections of it—is a breakthrough achievement in itself.

“If you had tried to do this with a previous system or a desktop computer, it would have been completely intractable. The amount of time for the computations would have been measured in years. But with an exascale machine, we can get highly resolved, whole-part simulations done on the order of several hours to a couple days,” Bement said.

Taking ExaAM to the Factory Floor

Plotkowski works at ORNL’s MDF, a 110,000 square-foot, off-campus building where the lab tests its advanced manufacturing technologies and shows them to industry partners. His background in physical metallurgy and alloy design for manufacturing gives him a factory floor perspective on assessing ExaAM’s potential for industrial applications.

“We’ve been developing capabilities for ExaAM that enable us to predict useful things about the AM process in terms of the construction properties of these materials. That has a pretty obvious application to a lot of industrial companies that are interested in being able to optimize AM processes by bringing those things in house and being able to fabricate components for themselves,” Plotkowski said.

ExaAM has been successfully tested at MDF by making parts on an industrial-scale laser powder-bed printer, which puts down layers of metal powder and then melts them with lasers. This has enabled the team to take measurements of the constructed parts’ microstructures and properties as well as the dynamics of the process while they’re being fabricated. They then compare those measurements against the predictions from the software tools to help guide the software development and confirm that it’s making reasonable predictions.

“We’re in the process of trying to develop a workflow in which we can take actual data that comes from the systems about how they’re performing. Then we can use that to drive the simulation tools so we’re doing a one-to-one simulation—a digital twin to get simulation data that corresponds one-to-one with components that we’re fabricating in real life,” Plotkowski said.

Ultimately, the team hopes to make ExaAM’s entire tool set public and has already put some of the codes online for anyone to download.

“Each of those is a scalable code, so it’s written in such a way that it will run on laptops or workstations but will also run very large things on supercomputers,” Plotkowski said. “It takes a little expertise to go all the way from one end to the other if you’re trying to use those codes, but it’s totally possible, and they’re written that way to make it usable for industrial partners in their own workstations or clusters.”

With the ECP scheduled to wind down in 2023 and 2024, the ExaAM project is on track to successfully deliver AM simulation capabilities to assist other DOE and industry efforts. The team’s next step is to use these tools to accelerate AM adoption for critical energy applications. The ExaAM codes are being incorporated into the MDF Digital Factory—which is funded by the DOE’s Advanced Materials & Manufacturing Technologies Office (AMMTO)—to help accelerate qualification of AM components for nuclear energy in the DOE Office of Nuclear Energy’s Advanced Manufacturing and Materials Technology program.

Meanwhile, ORNL researchers are working with General Motors to understand the AM process’s effects on the properties of lightweight automotive components via a DOE program called High-Performance Computing for Manufacturing that’s also funded by AMMTO. This effort may soon bring AM-printed parts to our daily lives.


A laser powder-bed 3D printer at Oak Ridge National Laboratory’s Manufacturing Demonstration Facility begins the process of constructing a metal part. Not unlike a desktop 3D printer, additive manufacturing fabricators make parts from scratch by sequentially adding layers of metal and using a laser to make complicated patterns by melting the metal, which then resolidifies and bonds with the layer below it, forming the desired 3D shapes. Video by Kelley Smith/ORNL.