To enable high-fidelity simulations of wind farm aerodynamics—involving unsteady, turbulent atmospheric conditions, large motions, and overlapping components of variable lengths—a research team developing the Exascale Computing Project’s (ECP’s) ExaWind software suite investigated ways to improve the performance of Nalu-Wind, the ECP’s code for whole wind farm simulations. Their work was published in the November 2020 issue of the Journal of Computational Physics.
Nalu-Wind, an unstructured incompressible-flow computational fluid dynamics (CFD) solver, solves the entire domain with a single-linear-system strategy, using constraint equations to couple overlapping domains. This method incurs high MPI communication costs and greatly reduces the performance of linear solvers. This approach also requires reconstruction of global linear systems and preconditioners at every time step, adding further to the increased computational costs.
The team investigated the tradeoff between solution accuracy and possible speedups on a series of problems. They found that a decomposed-linear-system approach, by which algorithms decompose the global linear system into smaller linear systems (i.e., individual overset-mesh systems), effectively eliminates the constraint equations from linear systems while reducing the size of the linear systems being solved. For all case studies considered, the decomposed-linear-system approach was found to be numerically stable, enabled faster simulations, and computed quantities of interest within a percent error compared to Nalu-Wind.
For the ExaWind challenge problem, decomposing the linear systems for each overset mesh offers several advantages by (1) allowing the use of rigorously coupled, separate CFD codes wherein optimal solvers can be chosen for their respective domains and (2) speeding up the linear solvers. The literature is sparse with regard to formal comparisons of single- and decomposed-linear-system solver strategies for overset meshes; the team provides some of the first documented benchmark studies. This paper marks the first step in adopting a hybrid-solver strategy using CFD solvers Nalu-Wind and AMR-Wind.
Sharma, Ashesh, Shreyas Ananthan, Jayanarayanan Sitaraman, Stephen Thomas, and Michael A. Sprague. 2021. “Overset Meshes for Incompressible Flows: On Preserving Accuracy of Underlying Discretizations.” Journal of Computational Physics 428 (March): 109987. doi:10.1016/j.jcp.2020.109987. http://dx.doi.org/10.1016/j.jcp.2020.109987.
