ExaAM Project Aims to Transform Additive Manufacturing through Exascale Simulation

By Scott Gibson

John Turner of Oak Ridge National Laboratory

Additive manufacturing (AM), or 3D printing, is revolutionary. The reason is that it enables customized and efficient design of parts and products—from very precise implants to military body armor, aeronautic parts, and myriad other items. 3D printing of human organs may be on the horizon.

In the latest episode of the Let’s Talk Exascale podcast, we are joined by John Turner of Oak Ridge National Laboratory. He is principal investigator of the ExaAM project within the US Department of Energy’s (DOE’s) Exascale Computing Project (ECP). ExaAM plans to transform AM through exascale computer simulations.

The discussion provides a look at AM and ExaAM from different angles: a brief history of AM, the focus of ExaAM’s efforts, bicycle frame design as an AM example, the significance of the microstructure of materials, the good news and bad news about AM, what ExaAM expects to deliver, Inconel and its importance, ExaAM’s challenge problem, the contributions of partners, the project’s near-term objectives, and its expected enduring legacy.

Interview Transcript

Gibson: We welcome John Turner of Oak Ridge National Laboratory. He leads ECP’s ExaAM project. Let’s begin by having you provide us a brief history of AM and the context for ExaAM.

Turner: Thanks, Scott.

Additive manufacturing, 3D printing, has been around for a number of decades. It began in the 1980s, initially as a means of rapidly creating prototypes.

It was not originally meant to be a way to manufacture final parts but to try out design concepts and see how they might work in real life. Over the next few decades, the concepts and the processes of additive manufacturing matured. In the early to mid 2000s, a number of things happened that led to a rapid expansion of the technology.

Some of the early patents expired. In technology, sensors and control systems became more sensitive and advanced. Think of the sensors in your phone that enable it to determine whether you’re holding it horizontally or vertically and those kinds of things. And computational capabilities had expanded. All these things came together to lead to an explosion in additive manufacturing.

The most visible expansion was in the consumer space where you saw a lot of very affordable desktop machines for making polymer plastic parts. In these machines you would partially melt this plastic polymer and extrude it—push it out—through a nozzle to build up whatever you were making. And you could make a lot of cool things with it.

Less visible were the dramatic strides in 3D printing of metal parts. It’s really not a consumer-facing thing. The systems to make metal parts were much more complex and much more expensive. But it also has potentially much bigger impact, large scale.

Most of these 3D printing processes—whether they’re the plastic parts or the metal parts—involve taking a 3D model of an object in the computer, mathematically slicing into a bunch of 2D layers, and then building the actual physical part, layer by layer.

With metals, there are a bunch of different ways of doing this. All of them involve some kind of feedstock material, which can be a powder or wire and some sort of energy source such as a laser or an electron beam.

Gibson: What is the ExaAM project centered on?

Turner: We’re focusing on powder processes where you take a very fine powder of some metal—it can be titanium, stainless steel, other alloys. You take that powder and you either blow it or layer it in a machine and then you melt it in precise locations to build up a part.

This titanium ball and robotic hand were made with 3D printing. Credit: Oak Ridge National Laboratory

That actually sounds pretty simple, and in some ways it is. It’s a lot like other things like welding. It’s like a lot of small welds that go together to weld these little beads of metal into a single part.

The main advantage of this whole process of additive manufacturing is in the design space. You have a lot of flexibility in optimizing for a particular use.

That includes the material choice. You have a choice of titanium or stainless steel and other nickel-based alloys. And you could even blend these together to have hybrid materials.

And you can even control the characteristics of a part in different locations of the final part that you want to make.

Think of a bicycle frame. The distribution of stresses varies dramatically between the top tube, the bottom bracket, the fork. And those stresses also depend on use. The stresses of a leisurely ride around the neighborhood are dramatically different from those a racer experiences on a stage on the Tour de France. And even there, there’ll be a big difference between, say, a time trial where there’s the cyclist delivering almost a steady, consistent power throughout several hours or around an hour, and an attack on a steep mountain climb where the cyclist is out of the saddle. So, you can just imagine there are very different stresses on the different locations of a bicycle frame. And over the years, the design of a bicycle frame has evolved to what is now based on those stresses and where bikes have failed in the past and things like that.

With additive manufacturing, we can adjust the parameters used in the process—the beam speed, power—and we can actually attain different properties at different locations. Ideally, we use this information within the design process to obtain a design that’s optimized for particular uses.

For racing bikes, we could minimize the weight. For recreational bikes, we could minimize cost and maximize lifetime, things like that. So, we have a lot more flexibility in the additive process.

Now, a big challenge is that there’s a very complex interplay between the physical phenomena that are involved in the additive process: heat transfer, melting, solidification, fluid flow. And ExaAM’s main goal is to improve our understanding of those processes, how they interact, and how they affect each other so that we can do a better job of that optimization.

Gibson: Will you speak about the significance of the microstructure of materials and how that ties into ExaAM?

Turner: Additive manufacturing seems very similar to other manufacturing processes that we have a lot of experience with, like welding and casting. We have decades of research and data and expertise in those processes. In reality, the fundamental processes in additive manufacturing are quite different and result in very different microstructures and, hence, properties. So, I’ll explain a little bit about what that means.

The properties of a given material—and here I’m mainly talking about metals—are determined not just by the material itself but also its structure. What I mean by that is the size and configuration of its so-called grains. Grains are small regions of material with the same orientation—crystal lattice orientation. The distribution of those grains, how they are aligned with each other, how big they are, that’s referred to as its microstructure. And how that collection of grains behaves collectively determines a material’s properties: strength, fatigue, resistance to fatigue, cracking, all of that kind of stuff.

The good news is that in additive manufacturing we have the ability to design in the properties that we want, microstructure that we want—even at specific locations within a part. The bad news is this is not easy to do. The parameter space is huge. How you adjust the beam movement and the power during that process is a huge space.

ExaAM, we’re developing a collection of software capabilities for simulating those various aspects of that process. The repeated melting and solidification of the powder and the formation of that microstructure and ultimately the prediction of the properties that the materials with those microstructures will have. That’s what ExaAM is developing. That collection of software capabilities. It really requires significant computational power to do all of those aspects of the physics, how they connect to each other.

Gibson: Could you elaborate on all that ExaAM expects to deliver?

Turner: Ultimately, what ExaAM is going to produce is both a collection of these software capabilities and experimental data and simulation data, rich experimental simulation data that can be used in the formation of different types of materials—from stainless steel to titanium, to Inconel parts. Many industries, such as aerospace, have an interest in Inconel type alloys. Having a better ability to optimize the design process to build parts out of those types of materials is extremely desirable, and at the end of ExaAM, we’re going to provide those to the community.

Gibson: What is Inconel and why is it important?

Turner: Inconel is a family of alloys that has been developed over a number of decades. They’re very important in the aerospace industry. They have a lot of desirable properties—corrosion resistance, behavior at high temperatures—for applications like like turbine blades and aircraft engines. So, it’s just a specific family of alloys that has a large amount of nickel—they’re often called nickel-based alloys. They’re a little different than the stainless steel.

Gibson: Please provide the context for the specific challenge problem that ExaAM is addressing.

Turner: Although we have a variety of simulation tools for different aspects of the additive process—like melting, solidification, and microstructure formation, and property prediction of microstructures—a lot of those capabilities are not specific to AM. They were developed for other types of processes or other materials. And so we needed to look across the landscape of the national labs and university space and broader community to look at what capabilities exist, where are there gaps in those, specifically, for additive manufacturing, identify those, and then we chose a collection of capabilities to then focus on bringing together for the specific types of phenomena we experience in additive manufacturing and then increase the level of fidelity to be beyond the descriptive and into the predictive realm. To do that, we really needed to improve their numerical capabilities and their ability to run on the larger machines that are coming to the national lab facilities like Summit and then Frontier afterwards and other exascale systems. The ability of these capabilities to run on those systems needs a significant amount of work.

So, that’s where ExaAM is really doing work across that landscape of both the domain side increasing the physics capabilities as well as the numerical and software capabilities and linking these together in an integrated fashion to be able to do the prediction of additive processes.

Gibson: What is ExaAM’s challenge problem?

Turner: Ultimately, what we want to do is provide the ability to predict in a location-specific sense the properties that a part built with additive manufacturing will have. So, the connection between the additive manufacturing process parameters, the microstructure it creates, and the properties at the end. To turn that into a concrete problem, we’ve chosen this benchmark problem that the National Institute of Standards and Technology, one of our partners, NIST, they’ve come up with a benchmark series called AM-Bench, and we’ve taken one of their problems. It’s not a terribly exciting problem geometrically, but it presents all the aspects of complexity that we need. It’s a little complex bridge with thin legs and thick legs, and it’s made out of one of these materials—Inconel—that we talked about earlier. And so we’re using that benchmark problem and the parameters that they’ve used to replicate the experiment and regenerate in a virtual sense the microstructure and then the properties of that part. Being able to do this for this particular part will enable us to do it for any part. We’ll have the ability to predict, in a location-specific sense, the properties of an additively manufactured part.

Now, you have to put in the appropriate material properties. If you switch to a different material, like titanium or stainless steel that’s going to have a different set of physical properties. But the core software simulation capabilities are there for any material.

Gibson: How have the respective ExaAM partners contributed to the project’s successes?

Turner: ExaAM is a collaboration between Oak Ridge National Laboratory, Lawrence Livermore National Laboratory, Los Alamos National Laboratory, and the National Institutes of Standards and Technology [NIST], and also a number of universities. They each bring unique expertise, and in some cases, they had existing software simulation capabilities that we’ve extended. And others, they’ve led the development of new capabilities.

For example, one of the really exciting things for the broader community is the role of NIST. They bring a unique experimental angle to this. Because right as ExaAM was starting, they were embarking on a community benchmark project called AM-Bench. So, since we’ve been involved from the beginning, we’ve worked with them to develop this community set of benchmarks that’s doing experiments, providing that data to the community in an open sense, and then bringing together modeling and experimental communities to compare those results. So, we’re using that. We’re both helping to participate in the development of those benchmarks and using the results to drive ExaAM goals.

At Lawrence Livermore, they’ve had a significant effort over the last few years in additive and a number of new capabilities have really been extended. One of which is evolving the microstructure. It’s called ExaCA, using a technique called cellular automata. Another is predicting the micromechanical behavior of those microstructures, so, essentially, the properties that I spoke of earlier. Based on those microstructures.

At Los Alamos, they’ve got significant expertise in this area. They have a casting simulation capability that’s been developed over a number of years. And we’ve extended that and modified it, specifically to additive processes. And that handles the melting and solidification of the process that provides the driving force for the microstructure and properties later on.

At Oak Ridge, we’re leading the project and participating in a number of the areas, with a lot of emphasis on the microstructure formation at a detailed level, the phase transitions, particularly at late time while the part is cooling. There are various phase transformations that happen that affect the microstructure over time. And then of course we’ve got the Summit and the next phase will be Frontier. We’re really targeting running on those platforms with the GPUs and developing all these capabilities to run on those platforms.

We’ve come quite a long way developing all of these capabilities, making them run significantly faster on these platforms and really set them up for the next platforms. The other thing we’ve made great strides in is integrating these various capabilities and enabled them to talk to each other and influence each other so that the physics is more accurate between these different stages of the additive process—from the melting to the microstructure, to the property prediction.

So, we’re not there yet. We’ve got quite a ways to go in the coupling of these physics. But I would say that’s one of our areas of success: taking all these capabilities, making them more capable of doing the additive process and enabling them to talk to each other and communicate.

Gibson:  Where is the work of the ExaAM project headed next?

Turner: One of our near-term goals by the end of the year is one of our initial predictions of the microstructure in those AM-Bench parts. So, we’re taking one particular location where NIST has obtained microstructural data. We’re using these capabilities to see if we can predict that microstructure. That’s going to be a big test. Will we be able to replicate what we’ve seen experimentally? We build these parts. We know the parameters that are used to build the parts, then we slice them up; we look at their grain structure. So, now we’re going to use that same information, feed it into our capabilities, and predict microstructure. That’s going to be a big achievement that we’ll hopefully see over the next couple of months. Next year we’re going to take that to the next level of predicting a larger volume and going to the next step, to the properties, and then scaling it up to the full part.

It’s going to be an exciting year or two. The next year or two is going to see a lot of building on the foundation that we’ve put in place and using it to do these predictions.

Gibson:  Final Question: What do you expect the enduring legacy of ExaAM will be?

Turner: It’s going to be two sets of things: One is the software capabilities themselves. This selection of software tools or capabilities to do the different parts of additive and collectively in a workflow predicting these physical phenomena because, ultimately, you want to wrap these capabilities in an optimization loop. This goes back to the idea of having location-specific properties in whatever part it is, whether a bicycle frame or an aircraft engine, being able to optimally design that and know the properties at every location that you’re building. So, that’s one thing—the software capabilities that we’re making open source and developing. The other is the data. We are working with NIST and other places to create unique experimental datasets but also high-fidelity simulations. As we run our challenge problem over the next couple of years, we’re going to be generating large sets of simulation data that’s extremely high fidelity that we intend to make available to the community. This has been done in other areas like Johns Hopkins years ago made available a turbulence database. So, large simulation sets that have been mined over a number of years by researchers to probe that high-fidelity data, use it to generate reduced-order models that run much faster but retain a lot of the accuracy and the physical description of the problem. That will be the case here as well. We’ll make this simulation data available, which can be mined by the broader community. It’s important to convey that will be both the experimental data and this high-fidelity simulation data. Both of those, I think, are extremely valuable.

Gibson: Well, John Turner from Oak Ridge National Laboratory, thank you very much for being on the program.

Turner: Thanks, Scott.