14th EuroAd Workshop - Markus Towara - An Effective Discrete Adjoint Model For OpenFOAM PDF

STCE
An (more) effective Discrete Adjoint Model for OpenFOAM

Markus Towara, Uwe Naumann
Software and Tools for Computational Engineering Science
RWTH Aachen University
EuroAD 2013, Oxford, 10. December 2013
CFD @ STCE
A. Sen (aboutFLOW Project, discrete OpenFOAM)
A. Daztouri (aboutFLOW Project, discrete ACE+)
M. Foerster (fortissiFOAM Project, OpenFOAM in the cloud)
me (discrete OpenFOAM)
STCE
,
STCE RWTH-Aachen, Discrete OpenFOAM, EuroAD 2013
2/ 35
Outline
STCE
OpenFOAM Introduction
Discrete OpenFOAM
Improvements
Verification and Results
Work in Progress - PISO Foam
,
3/ 35
Outline
STCE
Discrete OpenFOAM
Improvements
,
4/ 35
OpenFOAM
STCE
Open Field Operation and Manipulation
Open-Source (GPLv3) CFD solver
developed by OpenCFD Ltd., currently at version 2.2.x
> 1 Mio. Lines of Code
includes tools for meshing, pre-, post-processing
rising adoption in industry and academia due to lack of licence costs well suited for parallel
architectures
,
5/ 35
STCE
CFD Basics
I
Navier Stokes equations for incompressible steady flow:

1
p
v=0
v v = 2 v
momentum conservation
mass conservation
Or in three Dimensions:
vx
vx
vx
vx
x
vy
x
vz
x
+ vy
+ vy
+ vy
vx
y
vy
y
vz
y
+ vz
+ vz
+ vz
v
+
x
vx
z
vy
p0
x
p0
vz
p0
2 vx
x2
2 vy
x2
2 vz
x2
2 vx
y 2
2 vy
y 2
2 vz
y 2
2 vx
z 2
2 vy
z 2
2 vz
z 2
w
+
= 0
z
,
6/ 35
CFD Basics
STCE
(Decoupled) Solution of the partial differential equations (SIMPLE-Algorithm):

I
discretize / linearize momentum conservation equations
solve momentum equations for velocity v, assume pressure p as known
obtained velocity field fulfills momentum equation but not mass conservation equation as the
pressure field was guessed and not correct
discretize mass conservation equation
use mass conservation equation to correct pressure field
use new pressure field to correct velocity field
Loop...
,
7/ 35
STCE
Topology Optimization
no optimization
added material
reconstruction
Add penalty term to Navier-Stokes equation1 :

(v ) v = 2 v p v
1 C. Othmer: A continuous adjoint formulation for the computation of topological and surface sensitivities of ducted
flows. Intern. J. f. Num. Meth. in Fluids. p. 861877, 2008.
,
8/ 35
STCE
How to find appropriate

I
Define Cost Function J, e.g. total pressure loss between inlet and outlet:
Z
J=
p+
1 2
vn d
2
Calculate sensitivity of the Cost function w.r.t. parameters i

J
=???
i
Calculate updated porosity field n+1 , e.g.:

in+1 = in
J n
,
in
while insuring
0 < i < max
Loop until converged...
,
9/ 35
Outline
STCE
Discrete OpenFOAM
Improvements
,
10/ 35
Motivation for AD
in Context of Shape Optimization
AD promises:
I
I
I
I
STCE
greater flexibility w.r.t new objectives

easy adaption to new solver types / generations
calculated derivatives are exact w.r.t the used discretization
higher order derivatives available
Problems:
I
I
memory requirements
how to retain parallelism?
,
11/ 35
How to implement
STCE
n m, so Adjoint Mode is method of choice

I
I
source code transformation

operator overloading
I
I
I
I
due to complexity of OpenFOAM code (templates, massively object oriented) the only suitable
option
operator overloading tool dco/c++
every floating point value in OpenFOAM is replaced with custom adjoint datatype dco::a1s::type
needed intermediate values are stored in a structure called tape
,
12/ 35
How to implement
put dco into src/OpenFOAM
include dco.hpp
I
I
replace doubles with active datatype from dco

OpenFOAM has own typedef for scalar floating point values
just one substitution
in theory we now just need to recompile OpenFOAM and are ready to go
STCE
,
13/ 35
OpenFOAM a1s mode
STCE
in src/OpenFOAM/primitives/Scalar/doubleScalar/doubleScalar.h: replace:
namespace Foam
{
typedef double doubleScalar;
...
}
with:
#include dco.hpp
namespace Foam
{
typedef dco::a1s::type doubleScalar;
...
}
,
14/ 35
OpenFOAM t1s mode
STCE
in src/OpenFOAM/primitives/Scalar/doubleScalar/doubleScalar.h: replace:
namespace Foam
{
typedef double doubleScalar;
...
}
with:
#include dco.hpp
namespace Foam
{
typedef dco::t1s::type doubleScalar;
...
}
,
15/ 35
in Reality
STCE
some minor changes have to be made in the OpenFOAM code:

I
I
I
unions dont support active datatypes

some casts are missing and have to be done by hand
some functions (pow,max,min) have to be adapted
,
16/ 35
STCE
usage of discrete OpenFOAM - t1s

Black-Box tangent-linear Version of simpleFoam, calculates
J
i :
double sens = 0;
dco::t1s::set(alpha[i],1,1);
for (runTime++; !runTime.end(); runTime++)
{
... // solve for U,p
}
// Sum pressure over inlet faces scaled with face area
doubleScalar J = gSum( p.boundaryField()patch.magSf() );
dco::t1s::get(J,sens,1);
Need to do this N-times to get full sensitivity field!
,
17/ 35
usage of discrete OpenFOAM - a1s
STCE
Black-Box adjoint Version of simpleFoam, calculates gradient of J:

dco::a1s::static tape tape(tapeSize);
double sens = new double[alpha.size()];
for(int i=0; i<alpha.size(); i++)
tape.register variable(alpha[i]);
for (runTime++; !runTime.end(); runTime++)
{
... // solve for U,p
}
// Sum pressure over inlet faces scaled with face area
doubleScalar J = gSum( p.boundaryField()patch.magSf() );
dco::a1s::set(J,1,1);
tape.interpret adjoint();
for(int i = 0; i<alpha.size(); i++)
dco::a1s::get(alpha[i],sens[i],1);
,
18/ 35
Outline
STCE
Discrete OpenFOAM
Improvements
,
19/ 35
First Results
I
I
STCE
Black-Box approach severely bound by memory bandwith! (Need to allocate around 20 GB)
By doing Checkpointing we can actually get faster...
,
20/ 35
STCE
How to improve?
I
Knowledge and Profiling reveals that most of calculation time and tape memory is spent in
(iterative) linear solvers
Analytical insight allows us to treat the adjoints of linear solvers analytically
Lemma
For a Linear Equation Sytem Ax = b we can calculate the adjoints for A and b by:
b = AT x
AT b = x
A = b xT
I
This gives us an additional Linear Equation System which we have to solve during the gathering
of the adjoints
2 M. Giles, Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation, Advances
in Automatic Differentiation 2008
,
21/ 35
STCE
Treatment of Linear Solvers

Black-Box
Continuous Linear Solver
when we encounter an linear solver during the augmented forward run we can stop taping
when we encounter the gap in the tape during the interpretation we have to fill in the gap by
hand
have to remember A and x!
,
22/ 35
Application to OpenFOAM
STCE
Abstract Base Class for all Linear Solvers exists can tackle all implemented solvers at once
Discretization matrices are stored in a coordinate sparse matrix format transposing matrix
straightforward
take care not to mess up symmetry of matrices
,
23/ 35
Outline
STCE
Discrete OpenFOAM
Improvements
,
24/ 35
Results
STCE
we are seeing a nice improvement in both runtime and memory usage
,
25/ 35
STCE
Verification
I
I
I
We calculate a reference solution with solvers GAMG for U and smoothSolver for p, both with
tolerance 1e 10
After that we calculate solutions with different tolerances for the forward and adjoint linear
solvers and compare to the reference solution
11k Cells in mesh
Velocity
Sensitivity
,
26/ 35
STCE
Verification
Runtime
Tape-Memory
,
27/ 35
Error
STCE
,
28/ 35
Runtime
STCE
,
29/ 35
Outline
STCE
Discrete OpenFOAM
Improvements
,
30/ 35
Work in Progress - Unsteady Adjoints
STCE
Unsteady PISO solver with unsteady cost function (potentially depends on every timestep)
,
31/ 35
STCE
,
32/ 35
STCE
,
33/ 35
Summary and Outlook
Fully discrete mode uses loads of memory
Treatment of linear solvers greatly reduces this
Its also faster
Application to unsteady flow, heat transfer, turbulence in progress
STCE
,
34/ 35
STCE
Thank you!
Questions?
,
35/ 35

14th EuroAd Workshop - Markus Towara - An Effective Discrete Adjoint Model For OpenFOAM PDF

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

14th EuroAd Workshop - Markus Towara - An Effective Discrete Adjoint Model For OpenFOAM PDF

Încărcat de

Drepturi de autor:

Formate disponibile

STCE

An (more) effective Discrete Adjoint Model for OpenFOAM

A. Sen (aboutFLOW Project, discrete OpenFOAM)

A. Daztouri (aboutFLOW Project, discrete ACE+)

M. Foerster (fortissiFOAM Project, OpenFOAM in the cloud)

Open Field Operation and Manipulation

Open-Source (GPLv3) CFD solver

developed by OpenCFD Ltd., currently at version 2.2.x

> 1 Mio. Lines of Code

includes tools for meshing, pre-, post-processing

Navier Stokes equations for incompressible steady flow:

(Decoupled) Solution of the partial differential equations (SIMPLE-Algorithm):

discretize / linearize momentum conservation equations

solve momentum equations for velocity v, assume pressure p as known

discretize mass conservation equation

use mass conservation equation to correct pressure field

use new pressure field to correct velocity field

Add penalty term to Navier-Stokes equation1 :

How to find appropriate

Calculate sensitivity of the Cost function w.r.t. parameters i

Calculate updated porosity field n+1 , e.g.:

0 < i < max

Loop until converged...

greater flexibility w.r.t new objectives

n  m, so Adjoint Mode is method of choice

source code transformation

put dco into src/OpenFOAM

replace doubles with active datatype from dco

in theory we now just need to recompile OpenFOAM and are ready to go

OpenFOAM a1s mode

OpenFOAM t1s mode

some minor changes have to be made in the OpenFOAM code:

unions dont support active datatypes

usage of discrete OpenFOAM - t1s

Need to do this N-times to get full sensitivity field!

usage of discrete OpenFOAM - a1s

Black-Box adjoint Version of simpleFoam, calculates gradient of J:

Analytical insight allows us to treat the adjoints of linear solvers analytically

Treatment of Linear Solvers

Continuous Linear Solver

have to remember A and x!

take care not to mess up symmetry of matrices

we are seeing a nice improvement in both runtime and memory usage

Work in Progress - Unsteady Adjoints

Work in Progress - Unsteady Adjoints

Work in Progress - Unsteady Adjoints

Summary and Outlook

Fully discrete mode uses loads of memory

Treatment of linear solvers greatly reduces this

Its also faster

Application to unsteady flow, heat transfer, turbulence in progress

S-ar putea să vă placă și

n m, so Adjoint Mode is method of choice