Today s contents Introduction The aim of a bio-numeric simulation - - PDF document

Bio-numerical Simulation with Grid Applied for the Oral Region

*K. Nozaki, *T. Akiyama, *T. Kaishima, *M. Nakagawa, **H. Tamagawa, J. Miura, **Y. Maeda, *S. Shimojo *CMC, Osaka University, Japan

**Faculty of Dentistry, Osaka University, Japan

Today’s contents

Introduction The aim of a bio-numeric simulation Why should Grid be required? About my simulation for the oral region How to mount the simulation on the Grid

environment

Acoustical analysis with Grid Computational Fluid Dynamic analysis with GRID

DentGrid proposal

Introduction

The aim of our research

To realize the evidence base dentistry by deploying

the grid environment

About me

Eccentric dentist

Speciality of the treatment of speech disorders

related with teeth.

Engineering officer of Super computer, SX5, NEC

Corp. GEMSS(Grid-Enabled Medical Simulation Services) , Simbio

http://www.gemss.de/

Surgical planning for the maxillofacial region Blood flow simulation of an artery

Biogrid

CMC, Osaka University Virtual Heart

K.Nakazawa, CMC-CAVE, 2004

Example of a bio-numeric simulation

The advantage of Bio-numeric simulations with Grid (1/2)

Low cost and high power

Plenty of the computational power Data sharing The visualization of over 2GB data

Bio-numeric simulation might teach me how to treat this disease.

The simulation need a large amount of computational resources Spec: P4 3.2GHz 1GB memory

Not enough I have two PC, but

• ne of them is not

always used. Idle machine

grid grid

Idle machine

Idle machine

Idle machine

certification

The advantage of Bio-numeric simulations with Grid (2/2)

Accumulation and utilization of the

simulation results combined with clinical data

Security

→PKI based

Search facility

→Semantic Grid

Certificate Authority (CA)

grid grid

confidence confidence

certification certification

certification

Storage Storage Storage

Storage Storage

Bio-numeric simulation with non-grid applied for the oral region

The aim of our simulation

To know how to fix the dental prostheses of the

patients who complain about speech disorders

To examine which part of the oral tract feature make

the difference of the sounds such as the sibilant /s/

I can’t speak correctly with my denture In my experience for this case, … Where should be fixed? How about the

• ther patient like

me?

Database

Simulation & database Simulation & database

Simulation & database Simulation & database

You’ll get used to it.

Bio-numeric simulation (1/14): for a Dental prosthesis

改訂 音声（ コ ロナ社）

An edentulous patient

Upper jaw lower jaw

denture

tongue

glottis larynx Lingual root pharynx uvular uvular

velum palatinum

trachea esophagus Expiratory air lips

This patient has the complain about the speech disorders with her denture. Especially, The speech disorders were found about the pronunciation of /s/.

Can’t pronounce sibilant /s/ well with my denture

Bio-numeric simulation (2/14): Conventional

way to treat the speech disorders of dentures

初心者のための総義歯製作法（ ク イ ンテッ センス出版）

Sibilant /s/

The area contacted by the tongue

powder

Paratogram:

One of the clinical technique for

examination of the speech disorders

The paratogram enables dentists to assess

the area contacted by the tongue

It is only indirect method to examine the speech

disorders with dentures

Bio-numeric simulation (3/14):

New approach for the speech disorders

How to modify the feature of

the dentures?

Examination

Change of the anterior teeth

angular (+30,normal,-30)

Acoustical analysis Computational Fluid Dynamic

analysis (CFD analysis) +30,

• 30

normal

Bio-numeric simulation (4/14):

Results (Acoustical analysis)

Acoustical analysis

Around high frequency

area, pink square, the signal intensity is different. +30°

• 30°

normal +30

• 30

MRI MRI

Laser scan

l

• n

g i t u d i n a l s e c t i

• n

a l v i e w l

• n

g i t u d i n a l s e c t i

• n

a l v i e w l

• n

g i t u d i n a l s e c t i

• n

a l v i e w

normal

Bio-numeric simulation (5/14):

Results (CFD analysis)

+

different different

Bio-numeric simulation (6/14): Discussion for semantic database

It is important for the clinical treatment to

combine the results of the acoustical analysis and CFD analysis related with the anterior teeth features

It will achieve the semantic database by storing

those clinical data with metadata. metadata

Storage

Bio-numeric simulation (7/14): Discussion for 3 dimension (3D)

The change of sibilant sounds depend on

the anterior teeth area.

It will be better to examine that area by 3D

simulation than to do whole area by 2D.

Oral region has a complicated 3D feature such

as teeth.

x y z 2D

Bio-numeric simulation (8/14): Taking an impression (3D)

Taking impression of 3D

• ral tract while

pronouncing the sibilant /s/

Dental silicon impression

paste

Hardened silicon could

reflect the relationship between the tongue and teeth

silicon

Bio-numeric simulation (9/14): Model construction

3D oral tract model for

CFD analysis

Laser scan of dental

plaster-models

Converting 3d points

data to polygon data on CAD software, SURFACER

Bio-numeric simulation (10/14): Results of the 3D simulation

CFD analysis was performed by using

STREAM (CRADLE Co. Ltd.)

Transparent model

Bio-numeric simulation (11/14):

Results of the 3D simulation

Helicity

In front of

anterior teeth

Time round

change distribution of the distributed area of helicity

v v curl v h ⋅ = ⋅ = ) ( ω

Bio-numeric simulation (12/14):

Results of the 3D simulation

Powel sound

source

Around the

anterior teeth

) ( v div × ω

Bio-numeric simulation (13/14):

Results of the 3D simulation Zoom up around anterior teeth

Left: vertical slice image Right: 3D powel sound source

Bio-numeric simulation (14/14): Discussion

The area of the sound source could be found by this methods. This bio-numeric simulation needed; A lot of computational resources

40GFlops, 36 hours

Large amount of storage areas

More than 2Gbyte for only 10 time steps of tensor data such as the velocity

(x,y,z), pressure in condition of 6,500,000 mesh

We needs 15,000 steps!

Large amount of memories

Working memory was restricted to be less than 1Gbyte because 32bit

visualization software can only visualize less than 2Gbyte data. Future plan

Computational Aero-Acoustics (CAA) will be mounted, however CAA will

need much more those resources.

• 2

• 2

• 1

• 1

• 5

CAA FFT

comparison

voice

Pressure transition

Grid testbed (Acoustical Analysis)

Globus2.4 + GridPort2.3.1 + SRBv2 + FFT

Single sign on with myProxy

Put wave file from local server to web server Select acoustical analysis types

Result and put the result to SRB Attach the metadata to this result

/ s /

Frequency

Power

time

Grid testbed (1/5) (CFD analysis)

Some difficulty come from Grid

Computation of CFD by using MPI

Piled up a communication latency Connectivity among clusters

Occasionally cluster has the private IP.

Grid enabled MPI solutions for Clusters, Matthias M. et al, 2002

latency

Step #1 Step #2 Step #3 Step #n

Computation time

• Front end node: global IP
• Node #2
• Node #3

・ ・

• Node #n

Private IP

Cluster

Grid testbed (2/5) (CFD analysis)

MPICH-G2 was used as parallel processing for the bio-

numeric simulations

It has been often said, “GRID is not good at CFD”.

Hypothesis: The volume of “computation/1node” might be the

key to brake such as a latency problem.

Volume size of “computation/1node” The ratio of the communication in total simulation time

100 200 300 400 500 600 (MB)

Grid testbed (3/5) (CFD analysis)

MPICH-G2 can’t realize the use of several

clusters simultaneously.

• Front end node: global IP
• Node #2
• Node #3

・ ・

• Node #n

Private IP

Cluster#1 Cluster#2

• Gridport2.3.2
• SimpleCA
• Globus2.4
• MPICH-G2

X

RSL file (hoge.rsl)

+ ( &(resourceManagerContact="dentgrid.ais.cmc.osaka-u.ac.jp") (count=1) (label="subjob 0") (environment=(GLOBUS_DUROC_SUBJOB_INDEX 0) (LD_LIBRARY_PATH /opt/gt2/lib/)) (arguments= "-b" "/home/nozaki/works/nast3dgp/f1/F1.bin" "- p3") (directory="/usr/local/nast3dgp/bin") (executable="/usr/local/nast3dgp/bin/navcalcmpi") ) ( &(resourceManagerContact="sibbs.ais.cmc.osaka-u.ac.jp") (count=12) (jobtype=mpi) // if you use a vendor supply MPI (label="subjob 1") (environment=(GLOBUS_DUROC_SUBJOB_INDEX 1) (LD_LIBRARY_PATH /r/Globus/lib/)) (arguments= "-b" "/home/nozaki/work/nast3dgp/F1.bin" "-p3") (directory="/usr/local/nast3dgp/bin") (executable="/usr/local/nast3dgp/bin/navcalcmpi") )

• Front end node: global IP
• Node #2: global IP
• Node #3: global IP

・ ・

• Node #n: global IP

mpirun –globusrsl hoge.rsl

Machines “sibbs.xxx.cmc.oska-u.ac.jp” 12 “dentgrid.xxx.cmc.osaka-u.acjp” 1 “cfd.xxx.cmc.osaka-u.ac.jp” 1 “133.1.69.250” 1 “133.1.69.251” 1 “tea.xxx.cmc.osaka-u.ac.jp” 40

Grid testbed (3/5) (CFD analysis) : How to solve this problem?

Iptunnel VPN jojo (AIST)

• H. Nakata, S. Matsuoka, S. Sekiguchi, A Java-based Programming

Environment for the Grid: Jojo, CCGrid 2004 , April 2004.

IPv6 All nodes have their own IP.

GT3.2 + Kernel 2.6!? G

• d

!

To be continued….

Grid testbed (3/5) (CFD analysis) : Developing CFD analysis system

GT 3.2 MPICH-G2 NaSt3DGP

A Parallel 3D Navier-Stokes Solver

• M. Griebel, T. Dornseifer and T. Neunhoeffer,

Numerical Simulation in Fluid Dynamics, a Practical Introduction, SIAM, Philadelphia,(1998)

VRML 1.0 importer

Rhinoceros 3.0J

SURFLASER (3d laser scanner)

VTK4.2 exporter, etc.

Testbed map of CFD with GRID

(under construction)

MCAT server

Scommands

Web Server

Client

Dental Clinics

Scommands

Clusters

https

• Gridport3
• SimpleCA
• GT3.2
• MPICH-G2

SRB servers

GSI GridFTP

• GT3.2
• PostgreSQL
• SRB v2
• GT3.2
• MPICHI-G2

GT 3.2

• GT3.2
• MPICH-G2
• VTK4.2
• SRB v2

GSI

100Mbps/s

100Mbps/s 100Mbps/s 100Mbps/s 100Mbps/s

100Mbps/s

GSI

GIIS Server

GSI GridFTP

100Mbps/s

GSI

• GT3.2
• SRB v2

Scommands

GridFTP

Summary of this talk

There is an optimal relationship between

network communication time and data size to perform the bio-numeric simulation.

Future medical treatment should be

required the Evidence why the treatment should be needed and selected.

Thank you !

I really appreciate to the audience and

especially, Mr. Seventeen Chen who invite me here.

Special contribution by T. Kaishima and T.

Akiyama, who help me to mount the globus software kindly.

Kazunori Nozaki nozaki@cmc.osaka-u.ac.jp