Lecture 5

Two-way FSI Solving and Post

Processing
14. 5 Release

Solving FSI Applications Using

ANSYS Mechanical and ANSYS CFX
1

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Outline
Solution Process
Here we discuss starting and monitoring the FSI solution in serial and
parallel. The various log and output files are also discussed.

Post Processing
This section will discuss common post-processing in CFD-Post and how
to post-process FSI results in the Mechanical post-processor.

Appendix A: Restarting FSI Simulations

This section covers restarting FSI simulations from completed runs and
backup points. Making physics changes to the CFD and Mechanical
setups when restarting is also discussed.

Appendix B: Multifield APDL Commands

This appendix provides some useful APDL commands relevant to MFX
simulations.

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Starting the Run

Write the Definition file and proceed
to the CFX Solver Manager
WB writes the def file in the background
Run Definition panel:
Use Double Precision for FSI runs, since
the mesh deformations are often very
small compared to the size of the
domain

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Starting the Run

MultiField panel:
MFX Run Mode:
Start ANSYS and CFX
Both solvers started automatically
Start ANSYS only / Start CFX only
If running on different machine see
parallel discussion later
Process Input File only
Produces a .mf file
Used for physics changes; discussed later

Mechanical Input File

As set in CFX-Pre, but can be changed here

Process ANSYS Input File

Uncheck if the Mechanical Input File is a .mf
file (i.e. has already been processed)
See restarts discussion later
4

2011 ANSYS, Inc.

July 26, 2013

then Start Run

Release 14.5

Running in Parallel
CFX can use all the usual parallel methods

Set up as for any non-FSI run

MAPDL solver is limited to local parallel
To run MAPDL in local parallel specify,
for example, -np 4 in the Additional
Arguments field

Will run on the local machine

MAPDL can run on two cores without any
additional licenses

From the Command Line:

> cfx5solve -def CFX_Setup.def -double -mfx-run-mode "Start ANSYS
and CFX" -ansys-input ds.dat -ansys-arguments "-np 2"
Start both solvers using the files specified, run CFX in double precision, run MAPDL solver
on two partitions. Note that on some systems spaces in the argument strings will need to
be escaped, e.g. Start\ ANSYS\ and\ CFX and -np\ 2
5

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Running on Two Different Hosts

To run on different machines:

Start ANSYS first on machine A

MFX Run Mode = Start ANSYS only
CFX out file will show a port and
hostname that ANSYS is listening on

...
...
+--------------------------------------+
|
|
|
Starting ANSYS Solver
|
|
|
+--------------------------------------+
ANSYS solver listening on 1971@MACHINE1

Start CFX on machine B

Other machines can be used as slaves
Provide the port and hostname output
by ANSYS

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Running on Two Different Hosts

From the command line:
> cfx5solve -def CFX_Setup.def -mfx-run-mode Start
ANSYS only" -ansys-input ds.dat

Starts the ANSYS solver using the input file specified

> cfx5solve -def CFX_Setup.def -double -mfx-run-mode
"Start CFX only" -cplg-host 2249@MACHINE1

Starts the CFX solver using the specified def file in double precision,
connects to the ANSYS solver already running on MACHINE1 using
the specified port

Run cfx5solve help or see doc for all command options

7

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Running in Parallel
Slave processes/threads must be on different cores
Otherwise things will run VERY slow
The codes are (usually) running sequentially, it would seem logical
that they can share cores
However, a slave process pulls 100% of the available CPU while its in
waiting mode
The other code is starved of CPU
OK for CFX and MAPDL to share a master processor

Example: single quad-core machine

CFX: 4 partitions, MAPDL serial

OK only the master is shared

CFX: 3 partitions, MAPDL 2 partitions (cores)

2011 ANSYS, Inc.

OK when MAPDL is waiting 1 core will be in use, leaving 3 for CFX

July 26, 2013

Release 14.5

Another Look at the Workflow

MAPDL is the master process

CFX-Pre

Mechanical

Definition File

Input File

Contains CCL definition of

fluid model and interface /
coupling settings

Contains APDL definition

of structural model

MAPDL controls the transient

timestep, interface mapping,
interface convergence checks,
load transfer, etc

But all these settings are defined in

CFX-Pre
We have an input file from
Mechanical that contains the
structural model with the FSI
surfaces tagged

Process Input File

CFX adds APDL commands for
interface / coupling settings

Workbench writes this input file

into the project directory

CFX Solver
Settings from the Definition
file

2011 ANSYS, Inc.

July 26, 2013

MAPDL Solver
Input File + appended APDL
commands from CFX

Release 14.5

Another Look at the Workflow

CFX-Pre writes a Definition File
(.def) containing the CFD model
and the interface / MFX controls
When the run is started, CFX
creates APDL commands for the
interface / coupling settings and
combines them with the contents
of the input file from Mechanical

Important to understand whats

happening here when

performing restarts
See Appendix A for restart
details

CFX-Pre

Mechanical

Definition File

Input File

Contains CCL definition of

fluid model and interface /
coupling settings

Contains APDL definition

of structural model

Process Input File

CFX adds APDL commands for
interface / coupling settings

CFX Solver
Settings from the Definition
file

10

2011 ANSYS, Inc.

July 26, 2013

MAPDL Solver
Input File + appended APDL
commands from CFX

Release 14.5

MAPDL Commands Inserted by CFX

You can view the commands
generated by CFX in the
MAPDL Solver output,
following the end of the Input
File
End of Input File from
Mechanical

CFX generated these

commands

11

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Process Input File

If you disable Process ANSYS Input
File then CFX does not generate
any APDL commands

Therefore the Input File for

MAPDL must already contain the
interface and coupling settings

CFX-Pre

Mechanical

Definition File

Input File

Contains CCL definition of

fluid model

Occurs in some restart workflow

Contains APDL definition

of structural model and
interface / coupling
settings

Also see CFX documentation:

ANSYS CFX Solver Manager Users

Guide >> Working with Solver
Manager >> ANSYS Multi-field Run
>> Processing the ANSYS Input File

CFX Solver
Settings from the Definition
file

12

2011 ANSYS, Inc.

July 26, 2013

ANSYS Solver
Settings from Input File

Release 14.5

Solver Files Generated

CFX

The working directory will contain the CFX Output

and Results files as usual

The transient (.trn) and backup (.bak) results files
are stored in the <name>_001 directory

MAPDL Solver

The subdirectory <name>_001.ansys is created and

contains all the MAPDL Solver files

All results are stored in the .rst file
The auto-generated ANSYS.mf file can be viewed in
this subdirectory
ANSYS.stdout is the ANSYS Solver output as
displayed in the CFX Solver Manager

Workbench

The working directory is:

/<ProjectName>_files/dp0/CFX/CFX
13

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

CFX Solver Output

Start of Time Step
First Coupling Iteration
Solve Mesh Displacement
- Should be fully converged
CFX Coefficient Iterations
- Should be converging

Second Coupling Iteration

Solve Mesh Displacement
- Should be fully converged

CFX Coefficient Iterations

- Should be converged by the
end of the last Coupling Iteration

14

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

CFX Solver Output

ANSYS Field Solver and Interface Loads plot also shown

X-axis values do not

correspond to each
other or to CFX
Cumulative
ANSYS
Iterations
L2 normalized values should be less than
corresponding convergence criteria (CRIT)

15

2011 ANSYS, Inc.

July 26, 2013

Cumulative
Coupling
Iteration
Interface loads should be negative
at the end of each timestep

Release 14.5

MAPDL Output Check Interface Mapping

In the MAPDL output, ALWAYS check the interface mapping
Scroll to the top of the output file (ANSYS.stdout)

Right-click in the window and select Find

Search for the following make sure Match case is enabled:

This section provides information on the mapping

between the CFX and ANSYS meshes. If the
Non-Matching Area Fraction or the No. of UnMapped Nodes is non-zero, then some parts of
the FSI interface will not be passing loads
between the field solvers. In this case check that
the two side physically match and completely
overlap.
16

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

MAPDL Solver Output

Time Step and Coupling Iteration #
Start and End Time for this Time Step
Actual Force Loads Received
(and/or Thermal Loads)
Start and End Time for this Load Step
1:1 correspondence to Time Step
ANSYS Field Solver
convergence and iterations
Check if CFX & ANSYS Field
Solvers are Converged
Check Interface Load
Convergence
17

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Summary

The CFX Solver Manager or CFX Solver command line is used to

launch 2-way FSI simulations

Mapping is performed by the MAPDL solver. Mapping

diagnostics are provided in the MAPDL solver output file and
should always be checked.

FSI settings defined in CFX-Pre are passed to MAPDL at runtime by generating APDL commands
Understanding this workflow is important for restarts

18

2011 ANSYS, Inc.

Restarts can be made from completed runs, interrupted runs

and from backup points. Physics changes can be made when
restarting. See Appendix A.

July 26, 2013

Release 14.5

Outline
Solution Process
Here we discuss starting and monitoring the FSI solution in serial and
parallel. The various log and output files are also discussed.

Post Processing
This section will discuss common post-processing in CFD-Post and how
to post-process FSI results in the Mechanical post-processor.

Appendix A: Restarting FSI Simulations

This section covers restarting FSI simulations from completed runs and
backup points. Making physics changes to the CFD and Mechanical
setups when restarting is also discussed.

Appendix B: Multifield APDL Commands

This appendix provides some useful APDL commands relevant to MFX
simulations.

19

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Combined Post-Processing
Editing the CFX Results cell for a
2-way FSI case will automatically
load both the CFX and structural
results into CFD-Post
Can also use File > Load Results
and browse for .rst (structural)
or .rth (thermal) results

20

Enable Keep Current Case

Loaded to post-process
multiple sets of results

Disable Open in new view to

have CFX and Mechanical
results in a single window

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Combined Post-Processing
In CFD-Post CFX and structural results
entries and regions are shown in the
outline tree
When creating objects they can be scoped
to all or selected domains

21

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Combined Post-Processing
Use the Timestep Selector to load
different time steps as usual

Tab panels show the available

results from each solver

Sync Cases allows both sets of

data to be updated when a time
step from one set is selected

Animations can show both fluid and

structural results simultaneously

22

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Combined Post-Processing Animations

Transient blood flow through a
three-leaf valve
Symmetry used with 1/6th
of fluid domain
Non-Newtonian fluid (i.e.
blood) defined in Fluent
Non-Linear contact in Transient
Structural to cater for closure of
valve leaflets
Blood vessel wall has
combination of isotropic and
anisotropic (non-linear)
materials

23

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Combined Post-Processing Animations

Transient free surface flow in
a liquid storage tank with
baffles
Tank is excited by a time
varying gravitational load for
a duration of 10s
Baffles are non-metallic
and fixed to the tank with an
adhesive
Forces are transferred (oneway only) via co-simulation
to determine the stresses
acting on the baffles
Determine the integrity
and viability of the
adhesive bonding
24

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Combined Post-Processing
When selecting Variables to plot,
the variables list shows all variables
from CFD and structural results

If a variable is not available (e.g.

Pressure on a slice plane in a structural
region) is it plotted in the specified
Undefined Color

Some variables apply to both fluid

and structural regions, e.g.:

25

Total Mesh Displacement (X/Y/Z)

Velocity (u/v/w)

Mesh metrics (Area, Edge Length Ratio,

X, Y, Z, etc)

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Combined Post-Processing
CFD-Post read components from a .cm file
written out by the MAPDL solver

This is created by default for FSI cases using the

command CMWRITE,file,cm

Youll see a Mesh Regions branch in CFD-Post

containing the components

When load structural results containing

shell elements, CFD-Post reads and
displays data from the top side of the
shells

26

To read and display results from the bottom

side set the system environment variable
CFDPOST_RST_READ_BOTTOM_SHELL_DATA
to any value

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Post-Processing in Mechanical
Structural results are associated with the
CFX Solution and Results cells

To post-process in Mechanical:

27

Edit the Mechanical Solution cell

Select the Solution entry from the Outline

tree

Select Tools > Read Results Files and

browse for the .rst/.rth file
associated with the CFX FSI
system

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Outline
Solution Process
Here we discuss starting and monitoring the FSI solution in serial and
parallel. The various log and output files are also discussed.

Post Processing
This section will discuss common post-processing in CFD-Post and how
to post-process FSI results in the Mechanical post-processor.

Appendix A: Restarting FSI Simulations

This section covers restarting FSI simulations from completed runs and
backup points. Making physics changes to the CFD and Mechanical
setups when restarting is also discussed.

Appendix B: Multifield APDL Commands

This appendix provides some useful APDL commands relevant to MFX
simulations.

28

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Restarts from a Completed Run

This includes runs that were manually
stopped before reaching the End Time
On the CFX side restart as usual

If manually specifying a previous res file,

Continue History From must be enabled for
FSI restarts

MAPDL always resumes from a previous

database

Select the db file from the previous *.ansys

run directory

The Mechanical Input File must be

specified but is not used in this case

Enable Process ANSYS Input File:

APDL commands are created again from the
CFX Solver Input File and a RESUME command
is also inserted
29

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Restarts from a Completed Run

Example of the commands generated by CFX
shown
Note that theres no reference to the
Mechanical Input File

Any changes made there are not used

The MF commands are recreated from the
CFX Solver Input File

The MF commands stored in the db file are not

used

If Process ANSYS Input File is disabled, the

Mechanical Input File is used and is run as is

The Restart ANSYS Run toggle is ignored because

CFX doesnt generate a RESUME command
No MF commands generated
30

2011 ANSYS, Inc.

July 26, 2013

Typical Restart Commands

Generated by CFX
resume,ANSYS,db
/SOLU
MFAN,ON
MFCLEAR,MFLC
MFRS,-1,SING
MFTI,1
MFDT,1,1,1,OFF
MFLC,SURF,ANSYS,1,
MFLC,SURF,CFX,'Wall',
MFIT,20,1,20
MFCO,ALL,1e-3
MFRE,ALL,0.75,RELX
MFPS,group1,ANSYS
MFPS,group2,CFX
MFSO,group2,group1
CMWRITE,ANSYS,cm
KBC,1
/GST,ON,ON
SOLVE
SAVE
CMWRITE,ANSYS,cm
FINISH
/EXIT
Release 14.5

Restarts from a Backup Point

A failed run can be restarted using a CFX
backup file and MAPDL files produced
using the MFRC command

Must be from a consistent time point

Pick the CFX .def file and use the CFX
backup file as the Initial Values File, with
Continue History From enabled
Same MultiField settings as restarting
from a completed run, except the rdb
(restart database) file is used

This file is created by the MFRC command

Again, the Mechanical Input File is not
used and CFX re-creates the MF
commands
31

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Restarts from a Backup Point

Only the first FSI run will contain a rdb file. For multiple restarts
copy the rdb file to the most recent run directory

32

Any previous changes you have manually made in a .mf file must be
re-applied to every restart from a rdb file, since the rdb file
represents the structural setup from the first FSI run

The correct nodal displacements, velocities, etc are still used, since
the rdb file will read the .r00# file corresponding to the restart time

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Restarts with Physics Changes

Changes to CFX physics can be made as usual
In Workbench edit the Setup cell
Or create a new .def file and use the previous res file as the Initial Values
File, with Continue History From enabled

Changes to coupling, interface or time settings specified in

CFX can use the same procedure as a CFX physics change
Since CFX re-creates the MF commands each time, these changes will be
passed to MAPDL
Example: Extending a run by increasing the Coupling Time Duration

Changes to Mechanical settings are less straight-forward

since the Mechanical Input File is not used when restarting
Where should be changes be made?
33

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Restarts with Physics Changes

First, create the .mf file

This is what CFX is doing in the

background when Process ANSYS Input
File is checked
Use Process Input File only option
Select the Mechanical Input File and the
previous db file
Provide the CFX def file and Start Run

Run will complete in a few seconds

Inside the *.ansys/ run directory an
ANSYS.mf file is created
/RunName_002.ansys/ANSYS.mf

34

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Restarts with Physics Changes

Manually edit the ANSYS.mf file, inserting
APDL commands to make any changes
Start the run using the edited .mf file as the
Mechanical Input File
Disable Process ANSYS Input File

Select the previous db file

Can also use this approach to change
interface / coupling settings (MF commands)
instead of making a new .def file in CFX-Pre
You need to use this approach to restart a
steady-state 2-way FSI case as transient

35

Add TIMINT,ON to the .mf file to switch

Mechanical from steady-state to transient

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Restarts - General
MAPDL results are appended to the rst file
Old rst is copied into new run directory
Can be a very large file!

You can move the old rst file to a new directory before restarting to
prevent it being copied
A new rst file will be created for the new run

For all restarts also see the CFX documentation:

ANSYS CFX Solver Manager Users Guide >> Working with Solver
Manager >> ANSYS Multi-field Run >> Restart Procedure for ANSYS
Multi-field Run

36

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Outline
Solution Process
Here we discuss starting and monitoring the FSI solution in serial and
parallel. The various log and output files are also discussed.

Post Processing
This section will discuss common post-processing in CFD-Post and how
to post-process FSI results in the Mechanical post-processor.

Appendix A: Restarting FSI Simulations

This section covers restarting FSI simulations from completed runs and
backup points. Making physics changes to the CFD and Mechanical
setups when restarting is also discussed.

Appendix B: Multifield APDL Commands

This appendix provides some useful APDL commands relevant to MFX
simulations.

37

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Multi-Field Commands - Background

38

Note: the Command Reference section in the Mechanical APDL

documentation provide details on all MAPDL commands

MF commands are APDL commands that control Multifield parameters

They are created by CFX-Pre and appended to the MAPDL input at run-time

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Basic MF Commands
MFRS

Multifield restart time (for restarts only)

Usually has a value of -1, meaning restart from end of previous solution
MFAN

Turns on Multifield solver

MFTI

End time for the Multifield analysis

MFIT

Controls the number of Coupling Iterations

MFDT

Control the transient time step size

39

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Basic MF Commands
MFOU

Controls how often results are stored

MFRC

Control how often restart files are written

MFCO

Controls Field Convergence Criteria

MFRE

Field Relaxation Factors

MFLC,SURF,CFX,'Interface','Total Force', ANSYS, 1,FORC,CPP

Load transfer command

MFPS and MFSO

Which field goes first, sequential or parallel

40

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Other Relevant APDL Commands

BCSOPTION,,INCORE

For the sparse solver tries to retain the entire factorized matrix in memory
Runs faster, but significantly increases memory usage
KBC,1

Sets stepped loads - CFX will always append this command to the ANSYS input
If using Prep7 make sure you set it manually
OUTRES

Minimum data you can write and still restart

Note that you may need more data for post-processing (e.g Stress data)
OUTRES, ERASE
OUTRES, ALL, NONE
OUTRES, NSOL, LAST

Structural Solution Only

41

2011 ANSYS, Inc.

July 26, 2013

OR

OUTRES, ERASE
OUTRES, ALL, NONE
OUTRES, NSOL, LAST
OUTRES, ESOL, LAST

Structural + Thermal
or Thermal Solution
Release 14.5

Other Relevant APDL Commands

NEQIT

Sets the maximum number of equilibrium iterations

NCNV

The ANSYS solver will continue when convergence has not been achieved
Note that MAPDL runs until convergence or max. number of equilibrium
iterations, but the calculation will stop if max. number of equilibrium iterations is
reached. You can set NCNV so that MAPDL will continue if convergence is not
achieved and this can be combined with NEQIT to limit the number of
equilibrium iterations. However, you need to be careful when doing this you
must make sure MAPDL is converged by the end of the last Coupling Iteration.
If you are reaching the maximum number of Coupling Iterations then
something hasnt converged CFX, interface loads, or MAPDL this needs to
be corrected. If one of the field solvers is very poorly converged, it may pass
spurious loads to the other which could cause failure.

42

2011 ANSYS, Inc.

July 26, 2013

Release 14.5