Towards Industrial Adoption of High-Order Methods Towards Industrial - - PowerPoint PPT Presentation

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

Towards Industrial Adoption of High-Order Methods

PyFR Symposium 19th June 2020 Mark Allan

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

About Zenotech

Meet the team

Jamil Appa Mike Turner

Director Director Product Lead, EPIC

Founded in 2012 Based in the Bristol Business Park, Bristol, UK Background in industrialisation of Computational Engineering research for Aerospace Mark Allan

Product Lead, Acoustics

James Sharpe

Lead, Big Data & Security

Oliver Darbyshire

Lead CFD Engineer

David Standingford

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

Software Products

Delivering simulation at scale

• High performance computational fluid

dynamics solver

• Best in class algorithms and numerics for

accuracy and industrial performance

• Energy efficiency and high turnaround

using the latest in manycore hardware https://zcfd.zenotech.com/

• Scalable, secure & simple HPC on demand
• Connect with multiple providers, find the

resource that matches your requirements

• Pay-per-use model, no capital expenditure
• Manage computing clusters, jobs and data

https://epic.zenotech.com/

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• Overview
• zCFD solver features
• CFD Applications
• CAA Applications
• Summary
• Aim
• Provide an industrial perspective on the use of HO methods

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• zCFD solver features
• High fidelity, density based, compressible FV-CFD solver on CPU/GPU
• MG, local time-stepping, low Mach preconditioning, dual time-stepping, RANS turbulence

modelling, LES, DES

• Arbitrary polyhedral
• Pressure based incompressible solver under development
• High fidelity, density based, compressible DG-CFD solver on CPU/GPU
• PMG, local time-stepping, low Mach preconditioning, dual time-stepping, shock capturing,

2-eqn turbulence modelling via SST k-ln(⍵), LES, DES

• Known cell types + hanging faces
• Benefited from working with PyFR team (Innovate UK Project - Hyperflux)
• DG-CAA solver on CPU/GPU
• APE-4 equations
• Stochastic noise source generator
• Flow properties and turbulence statistics based on RANS
• Runs on host CPU

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• CFD Applications
• Automotive example
• Using HO, can we maintain a ”similar” level of accuracy but reduce simulation time

with coarse grids and higher spatial order?

• Built environment (building) example
• Using HO, turbulence modelling and GPU, can we observe extreme events for an

”affordable” cost?

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• Automotive example

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

Re = 520000, M = 0.11, 4M / 17M solution points + PMG + dual time stepping SST-DES model

“The sub-critical flow past a generic side mirror and it’s impact on sound generation and propagation”, Ask. J, and Davidson, L., AIAA-2006-2558, May 2006.

• Automotive example
• Generic wing mirror

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• Automotive example
• Generic wing mirror

1 2 3

Approximately equivalent number of core hours (CPU) Cost to converge inner dual time-stepping iterations prohibitive

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• Built environment (building) example
• Aim of study to assess potential of high order methods to simulate extreme events at

affordable cost

• Utilise DES modelling for high Reynolds number
• GPU + high order to reduce run time

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• Model dimensions: 1 m x 0.3m x 2m (1:50 scale)
• Experiment 𝑣#$% ~ 10 m/s @ 1m height
• Building chord length, 𝑀#$% = 1 m
• Reynolds number, 𝑆𝑓$-. ~ 800,000
• Varying wind incidence

A B C D

From L. Amerio, PhD Thesis, 2017

“Affordable modelling of complex extreme events in the built environment using GPU accelerated CFD”, Saeed, T., et. Al., GTC 2020.

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• Spikes Cp ~ -10 to -15 observed in top corner of leeward side for
• blique angle of incidence
• Hypothesis is that these may be due to a vortical structure that
• riginates off the top of the model

A

From L. Amerio, PhD Thesis, 2017

“Affordable modelling of complex extreme events in the built environment using GPU accelerated CFD”, Saeed, T., et. Al., GTC 2020.

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

Inlet – inlet profile Outlet – farfield pressure Building – no slip Ground – symmetry plane Side walls – free slip

• NB. Top hidden – free slip

Key: 14 m 20 m 4 m Building at 20°incidence 2 m 1 m Flow direction

Trendline Error Trendline Error Trendline Error 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 H (m) u/uref Experiment CFD

“Affordable modelling of complex extreme events in the built environment using GPU accelerated CFD”, Saeed, T., et. Al., GTC 2020.

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• Structured mesh
• Near wall refinement

Blue edges: mesh Red edges: high-order solution points e.g. P4

“Affordable modelling of complex extreme events in the built environment using GPU accelerated CFD”, Saeed, T., et. Al., GTC 2020.

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• Simulation details
• DGX-1 system with 8x Tesla V100 GPUs
• SST k-ln(⍵) DES model
• Dual time-stepping (dt = 1.0E-04s), PMG, local time-stepping
• 30M solution points (P2)
• 160 hours to simulate ~0.7s
• Resolved issue with inter GPU data transfer (~100 hours)

“Affordable modelling of complex extreme events in the built environment using GPU accelerated CFD”, Saeed, T., et. Al., GTC 2020.

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

𝑣

“Affordable modelling of complex extreme events in the built environment using GPU accelerated CFD”, Saeed, T., et. Al., GTC 2020.

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• Validation – Tile A

Reduction in PSD amplitude potentially due to RANS wall modelling at corner Insufficient amount of time simulated to adequately resolve spectrum / observe extreme events “Affordable modelling of complex extreme events in the built environment using GPU accelerated CFD”, Saeed, T., et. Al., GTC 2020.

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• Conclusions – HO CFD for low speed flows
• In terms of spatial accuracy, HO far more efficient
• Time-marching limitations reducing competitiveness of method
• Challenges - HO CFD for low speed flows
• For many engineers accuracy is secondary to turnaround time / cost
• Strong competition from LBM!
• Fast
• Easy meshing

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• Computational Aero-Acoustics
• Nose landing gear example
• Using HO and stochastic source modelling, can we predict broadband turbulence

generated noise from a complex geometry?

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

APE DG Solver Radiated noise Farassat 1A URANS DES LES RANS FRPM

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• FRPM
• Generates a divergence free turbulent velocity field

by convecting a random white noise field through the domain following mean flow streamlines

• Discrete random field is achieved by “particles” with

random values (1 in 2D, 3 in 3D)

• Required statistics achieved by filtering the

convecting random field to give a desired correlation

• We obtain a streamfunction from which we can

derive turbulent velocities via

𝑆 ⃗ 𝑠, 𝜐 = 4 𝑆𝑓

5 6 67𝑓 58 ⃗ #59:6

;

<=7

;

ψ 𝑦, 𝑢 = A

B

𝐵𝐻 𝑦 − 𝑧 𝑉 𝑧, 𝑢 𝑒𝑧

• 3. Ewert, R., Dierke, J., Siebert, J., Neifeld, A., Appel, C., Siefert, M., and Kornow, O., “CAA broadband noise prediction for

aeroacoustic design,” Journal of Sound and Vibration, Vol. 330, No. 17, 2011, pp. 4139–4160.

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• PDCC-NLG
• M = 0.166, Wheel diameter = 0.1397m
• zCFD Mesh
• 30 M cells
• Menter SST
• 3.5 hours on 24 x V100
• CAA
• 26.4M solution points (4th order spatial accuracy)
• < 1.2 hours on 24 x V100

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• PDCC-NLG
• FRPM source visualisation

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• PDCC-NLG Flyover

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• zCFD / CAA / FRPM runtime summary
• RANS (30M cells, 3.5 hours)
• FRPM – 30 mins (run concurrently with CAA)
• CAA - 0.06 seconds simulated (converged)
• 26.4M solution points (4th order accurate)
• 24 V100s
• 1.2 hours
• Total time: 4.7 hours (£80 (spot) - £300)

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• zCFD CAA-IB
• Pulse cylinder test case
• Mesh – Includes hanging faces

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• zCFD CAA-IB
• Pulse cylinder test case
• Solution (P4) using 1 x Nvidia K80 (2 devices)

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• zCFD CAA-IB
• Pulse cylinder test case
• Solution (P4) using 1 x Nvidia K80

(a) (b) (c) (d)

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• zCFD CAA meshing
• Body fitted Cartesian dominant
• Merge out small volumes

Polyhedral cells

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• zCFD CAA meshing
• 30P30N results
• 2D simulation, body fitted Cartesian dominant mesh

* Taken from: ”Computation of Broadband Slat Noise from Swept and Unswept High- Lift Wing Sections”, Dierke et. al., AIAA conference paper, Jan 2011.

* *

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

• Summary
• HO CFD remains of interest for high fidelity CFD simulation
• Further work required on time marching for low speed flows
• Incompressible?
• Mesh generation remains an issue
• LBM implementations can easily handle complex geometry and produce good

results

• CAA can easily benefit from HO methods
• Stable, efficient solutions
• Care must be take with mesh generation
• Interest in immersed boundary methods / arbitrary polyhedral near wall cells

Towards Industrial Adoption of High-Order Methods

Mark Allan Presenter [ Product Lead ]

19th June 2020

Questions mark.allan@zenotech.com