AFOSR

Computational Mathematics
15 March 2011

Dr. Fariba Fahroo

Program Manager
AFOSR/RSL
Air Force Office of Scientific Research
Distribution A: Approved for public release; distribution is unlimited. 88ABW-2011-0773
 2011 AFOSR SPRING REVIEW
2304CX PORTFOLIO OVERVIEW
NAME: Fariba Fahroo

BRIEF DESCRIPTION OF PORTFOLIO:

Discover Mathematical Algorithms and Develop Computational
Techniques that Provide More Accurate, Reliable, and Efficient
Modeling and Simulation for the U.S. Air Force

LIST SUB-AREAS IN PORTFOLIO:

Multi-Scale Modeling
Multi-Physics Modeling
Computational Uncertainty
Multidisciplinary Optimization and Control

2
 Scientific Challenges for
Computational Math
• Goal: Simulation, Analysis and Design of complex
systems with radical cost and turn-time improvements ---
Need for Accurate, Reliable, and Efficient Algorithms

– High-Dimensional Problems
– Multi-Scale Problems --- Modeling and passage of
information across scales
– Multi-Physics Problems --- Interface formulation
– Effect of Uncertainty
– Effect of mesh and geometry
– Scalability of Algorithms --- Parallelism

3
 Transformational Opportunities
for Computational Math

• In the Tech Horizons Report (May 2010) multi-scale

simulation technologies and coupled multi-physics
simulations were noted among Priority Key
Technology Areas --- Computational Math is the Key
Enabling Science in these areas.
• The focus on development of high-order methods in
CFD, Plasma and other areas will allow for capturing
more complex physics with higher accuracy and more
efficiency --- enabling technologies in analysis, design
and control of aerospace systems
• Research in Uncertainty Quantification helps with V&V
and ultimately reliable design of aerospace systems.
4
 AFOSR-funded Research vs Others
• All agencies support research in multi-scale and multi-physics
modeling and simulation – differences are application, mission-based
• DOE and NSF are the biggest players in basic research in
computational math funding --- close interactions with these agencies
– attending review panels, sharing funding information, co-funding
• AFOSR Advantages: flexible in tailoring research activities around a
problem area and easier support of multidisciplinary research (math,
engineering, basic sciences)

• Goal: Find niche research areas that are fundamental and also AF-
relevant: (Aerospace Applications)
– Uncertainty Quantification,
– Model-reduction techniques for optimization and control of large-
scale systems,
– Strong emphasis on High-order Methods (space and time)
5
 Research Thrust Areas
• Multi-Physics Modeling --- Focus on AF Applications
 Robust and Fast Algorithms for Moving Interfaces

 Hybrid kinetics (particles and continuum) in a complex non-equilibrium

plasma
 Multiphase algorithms near singularities
 Algorithms for aero-elastic computations for large elastic deformations

• Multi-Scale Modeling --- Support of High-Order Methods for Efficiency

 Development of High-Order Methods (Discontinuous Galerkin)-Plasma
(multi-fluid models), aerodynamic flows
 Generalized FEM with global-local enrichment functions for multi-scale
problems in structures
 Energy-Consistent Multiscale Algorithms for Granular Flows
 High-Order Particle-Mesh Algorithms for Particle-Laden Shocked Flows
6
 Research Thrust Areas

• Multi-Disciplinary Optimization and Control – more

emphasis on Design Under Uncertainty
Reduced Order Modeling --- accuracy vs efficiency
 Multi-Fidelity Modeling and Design
Reduced-Basis Methods
 Discrete Empirical Interpolation Method (DEIM) for design
• Computation with Uncertainty – Uncertainty
Quantification (UQ)
Modeling and solving Stochastic PDEs – Elliptic, Hyperbolic
equations with error analysis
Stochastic Model Reduction Techniques
Stochastic Multi-Scale Modeling and Simulation

7
 Recent Transitions

 PI: (Suresh Menon , GA Tech) Hybrid Algorithms for

LES of Multi-Phase Flows –Transition to RW
 PI: (Uri Shumlak, University of Washington) Two-Fluid
Plasma Algorithm, implemented in the WARPX code ---
Transition to AFRL/RDHE to perform simulations of
the AFRL FRC-MTF (Field-Reversed Configuration for
Megnetized Target Fusion) plasma experiment.
 PI: (Marsha Berger, NYU) Inclusion of the
adaptation/adjoint module in the software package
Cart3D --- Transition to NASA, ONR, DOE, AFLRL, DIA
 PI: (Andrew Chrislieb, MSU) Gridless DSMC is being
merged into AFRL RDHE Kirtland ICEPIC code.
8
 High-Order Methods

• Higher Order Methods --- order of accuracy higher than 2

• The error of a numerical method is of order k+1 if the solution
error e decreases with mesh size h (=1/N ) according to e
(proportional) h^k+1
• For a fourth order scheme, if the grid size halves, the error
should be 1/16 of that on the coarse mesh
• High-order means high-accuracy
• High-order means high-order in space, time, and boundaries
• High-order is actually very efficient for a given level of accuracy
• High-order can be very efficient on massively parallel computers
• Use of high-order methods in CFD, Plasma, Structures and
Combustion have become increasingly essential to capture the
complex physics much better accuracy and efficiency.
9
 High-order CFD methods

• Develop efficient, fast and robust methods to tackle

complex vortex-dominated unsteady flows.
• The primary limitation of RANS (2nd order methods) is
the inability to predict turbulent, separated flow.
Application in flow control simulations:
• Flow separation and turbulent transition control.
• Airborne acoustic noise control.
Benefits:
• Improved dynamic performance and stability.
• Reduced drag means reduced fuel costs.
• Reduced aero-acoustic noise. 10
 Significant Achievement
ZJ Wang (Iowa State)

• L/D increases by over 10% in many cases using

passive flow control

Controlled model

Basic model

Separation bubble

Separation bubble
disappears resulting
in smaller drag D

Leading-edge bumps 11
 Multi Scale Plasma Simulations
Christlieb (MSU) – YIP 2007

• Goal: Devolve high order predictive tools for Plasmas.

• Plasmas are inherently multi-scale:
Two Types, fine structure within a model and multi model multi-scale

• Both high order methods and mesh adaptivity can address

fine structure within a model.
High order greatly reduces numerical diffusion, active equivalent
accuracy to low order method with greatly reduced computational
cost and time

• Achievement: First successful high order Adaptive Mesh

Refinement based on Weighted Essentially Non-Oscillatory
Finite Difference Methods, 5th order space - 3rd order time.
New parallel time stepping – 4th order in time of Forward
Euler 12
 Transformational Opportunities
PI Christlieb - 2010

• Tests show high order AMR more

accurate and less computational Double Mach
cost than Low order AMR Reflection
• CFD and Plasma applications AMR Domain
involving AMR would benefit.
• Current Challenges: incorporate
high order parallel time stepping
and develop high order embedded

Blowup Full Solution Blowup AMR Solution

TWO WEEKS ONE DAY

13
 Efficient High-Order Time-Integrators for Local
High-order Discretization Methods
Frank Giraldo (NPS) and Emil Constantinescu (Argonne)

Semi-implicit: Dynamics-based time-integration splitting

(fast-slow)
High order integrators yield the same accuracy for less
computational effort: Compressible Euler Relative error vs. CPU
Convergence of high
order methods in time
2nd order

3rd

4th
5th

high order -> same CPU effort

same error ->
high accuracy higher order -> 14
less CPU effort
smaller error
Balancing Errors in Space, Time, and Boundary
Conditions for High-Order Methods

Klein-Gordon Equation
Simulation: 24th order elements

15
 Scaling Experiments on Ranger
(the 15th fastest computer in the World)

Snapshots of the Simulation 64x64x64 elements with N=4

(32 Million Grid Points)
16
 Algorithm Development for
Emerging Computing Platforms

• Objective: To develop advanced algorithms with non-von

Neumann computer architecture (FPGA) and multi-core
GPUs for scientific computing, modeling, and simulation
– STTR AF07 – Phase II: Algorithm Development for
Reconfigurable Computing Architectures
– STTR AF09 – Phase I : Algorithm Development for
Multi-Core GPU-Based Computing Architectures
– Goal: Expand the capability of selected algorithms for
multidisciplinary computing into a production–level
product capable of solving complex problems in fluids,
structures, and electromagnetism taking full advantage
of the specific computing architecture.
17
 Algorithm Development
FPGA and GPUs
• FPGA Significant Accomplishments
– hprcARCHITECT™ Tool Suite
– National Repository of HPRC Algorithms
– LAPACKrc Library Released
• New iterative solvers created: CG and GMRES
• Performance typically exceeds 75x speedup
• GPU Significant Accomplishments
– Investigate Fourier Continuation (FC) solver on GPU
FC Solver Demonstrated 90.7 Speedup
– GPU-accelerated parallel tri-diagonal solver demonstrated in
OVERFLOW-2 and ARC2D
– Researched, developed, and ultimately released accelerated
solvers for banded matrices in CULA Library

18
 Uncertainty Quantification
MURI ‘09

•Karniadakis •Zabaras •Patera

•Hou
•Hesthaven •Koutsourelakis •Willcox
•Rozovsky
Multi-Scale Fusion of Information for
Uncertainty Quantification and Management
in Large-Scale Simulations

19
 Research Problem

Uncertainty Quantification (UQ) is important to the USAF:

 Electromagnetic scattering, Materials, …
 Flow-structure interactions, Aero-acoustics,
 Preliminary design of revolutionary vehicles, ….

Uncertainty Sources:
 Initial/boundary conditions, material properties,
constitutive laws, and geometry.
 Limited noisy experimental data, lack of knowledge or
from the inherent randomness of the underlying physical
phenomena.

Uncertainty can be represented in probabilistic terms by

transforming PDE-based models to stochastic PDEs
(SPDEs). 20
 Accomplishments

RA-1: Mathematical Analysis and Multiscale Formulation of SPDEs

• Weighted Wiener chaos and Malliavin calculus.

RA-2: Numerical Solution of SPDEs

• Adaptive ANOVA and high-dimensionality for fluid mechanics.
• Uncertainty quantification for time-dependent problems.

RA-3: Dimension Reduction

• Reduced basis methods for integral equations and multi-scattering problems.
• ANOVA expansions for parameter compression and reduced basis methods.

RA-4: Multiscale Property Estimation

• Data-driven sparse stochastic descriptions of micro-structure. KPCA, DDSMM.
• Bayesian framework for coarse-graining for material systems.

RA-5: Design & Control Under Uncertainty

• Nonintrusive strategies for optimization under uncertainty, adaptive SMC.
• Reduced basis methods for robust design, parameter estimation, and
model uncertainty.

21
 Reduced-Order Modeling

Objective Certified Reduced Basis Methods

Development, justification, optimal sampling procedures — efficient spaces

implementation, and demonstration  rapid convergence
of
a posteriori error estimation — rigorous error bounds
Certified  accurate uncertainty estimates
Reduced Basis Methods
offline/online strategies — low marginal cost
for rapid and reliable solution of
 O(100) computational speedups
parametrized
Partial Differential Equations (PDEs)
with application to
real-time and many-query
contexts such as
optimization and control,
parameter estimation,
uncertainty quantification,
and
multiscale analysis.
22
Benchmark I – Electromagnetic Scattering

23
23
 Model Reduction for Probabilistic Analysis
and Design Under Uncertainty
Heinkenschloss, Sorensen (Rice), Willcox (MIT)
• Model reduction has
Large-scale model Reduced-order
significant potential
Model model (ROM)
to enable control, Input order Input
parameters reduction
optimization, and parameters

uncertainty quantification Forcing Large Forcing Quantities

inputs scale inputs ROM
for complex systems. model
of interest

• For applications of interest, models exhibit nonlinear dependence on state

and/or parameters; previous projection-based model reduction methods
lead to nonlinear reduced models that are small but not fast to solve.
• Discrete Empirical Interpolation Method (DEIM) is a complexity reducing
approach shown to work well on many different applications
– Yielding dramatic speedups in simulation time
– Provably accurate capturing of nonlinear input/output behavior with
error estimates
– Has already been adopted internationally by several groups working
24
in optimization and control
 DEIM versatility and effectiveness
across many different applications.
Reacting flow Neural modeling
102x reduction in dof
102x reduction in
CPU time
R
reaction parameter
Mean estimate of
Statistical inference
of reaction
105x reduction parameters
in dof
106x reduction
in CPU time

Fluid-structure interaction Shape Optimization of Microfluidic

Work in progress Biochip
to implement 102x reduction in dof in 2D.
DEIM in AFRL
102x reduction in CPU time in 2D.
codes for
modeling limit Larger speedups
cycle oscillation expected in 3D.
with uncertain Current work
structural applies DEIM to
properties shape optimization
Figure courtesy of B. with uncertainties 25
in
Stanford and P. Beran
(AFRL). PDE.
 Future Directions

• Continued support of high-order methods in multiscale and

multiphysics modeling, support in time-integration methods
• Continued emphasis on UQ and V&V
– Numerical methods, Sampling methods, Design under
Uncertainty
• Algorithms for multi-core platforms: GPU computing
• Emphasis on scalability of algorithms for ultra-parallel large scale
computing
• Data-Driven Modeling and Computation
– Combination of new estimation techniques with large scale
computing: new ideas in data assimilation
• Understanding the impact of geometric discretization on the
fidelity of the analysis and computation
26