Embedded Supercomputing: (to combat the malaise that is big - - PowerPoint PPT Presentation

▶

Jul 15, 2023 17 likes •358 views

Embedded Supercomputing: (to combat the malaise that is big sofuware) Radio Astronomy at the Limit Simon Ratclifge & Bruce Merry SKA South Africa JSNTB In the beginning.... more recently = Johannes Hevelius 60fu - 1673 (the Ewan

SLIDE 1

(to combat the malaise that is big sofuware)

Embedded Supercomputing: Radio Astronomy at the Limit

SKA South Africa

Simon Ratclifge & Bruce Merry

JSNTB

SLIDE 2

In the beginning....

SLIDE 3

more recently

Johannes Hevelius 60fu - 1673

(the Ewan McTeagle of his day)

SLIDE 4

everything we know so far

http://xkcd.com/273/

SLIDE 5

getting the full picture

+ = = +

Jansky VLA, New Mexico

SLIDE 6

Watt?

1 Jy = 10-26 Wm-2Hz-1

I cannae do it, captain, ye cannae change the laws of physics

SLIDE 7

how much wood could a wood chuck, chuck

0.00000000000007 J 0.00000016 J 1.008 J

SLIDE 8

“Know your enemy and know yourself”

105 Jy 108 Jy 0 Jy

Sun @ 5 GHz GSM Phone @ 1km 'Smart' Phone @ 1m

SLIDE 9

something a little bigger

SLIDE 10 X X X X X X

SKY Image

Detect & amplify Digitise & delay Correlate Process

Calibrate, grid, FFT

Integrate

s B . s Astronomical signal (EM wave)

photon to image

SLIDE 11

Will it blend ?

Andrew Cooper

1000000000000000000 B

@ 1 bit per grain of standardised* sand

* assumptions apply (10gpmm^3, 3kmx2kmx700m, only valid when calculated on paper napkin, just say no to assumptions)

X

SLIDE 12

You sir, are a blaggard and a coward

1 exbibyte

1 exabyte

38, 230 x

SLIDE 13

Big Iron

SLIDE 14

Medium Iron (and a fair bit of aluminium)

SLIDE 15

In theory

IO / Cache / FLOPS / kW / $ Vij = Mij Bij Gij Dij Eij Pij Tij Vij

IDEAL

MAGIC

MeerKAT64

SLIDE 16

“My god it's full of data”

68

Gibps

input data rate

50

hour

bservation

1

PiB

bufger

@ =

SLIDE 17

50 hour totals

1.1

Tibps

bufger read rate

1.7

ExaFlop

total FP operations

1.9

working memory

SLIDE 18

easy…..

SLIDE 19

whither Mr. Fusion ?

20

available power

SLIDE 20

rosetta stone (edition 2010)

SLIDE 21

...shake your windows, and rattle your walls

TK1: 327 GFLOPs / 12.7 GiBps

SLIDE 22

measurement equation (redux)

V (u,v,w)=∫ A (l,m,w ) I (l,m) e−2 πi [ul+vm]dldm

input efgects image Fourier transform baseline direction

SLIDE 23

convolutional gridding

V (u,v,w )=∫ A (l,m,w ) I (l,m) e−2 πi [ul+vm]dldm

SLIDE 24

Working Model

8 Hours, 64 Antennas, Single Channel, 4k Image

SLIDE 25

I cannae push it any faster, Captain!

SLIDE 26

Trade-ofg between gridding and FFT costs

SLIDE 27

Anatomy of a modern radio telescope

SLIDE 28

Graphs all the way down until you hit the turtles...

SLIDE 29

How embarrassing is your parallelism ?

SLIDE 30

ignore the logo, it's clearly an internal design.

SLIDE 31

How big ?

peltier exchange

Ground Loop

Hive

FERRO

100Nodes 50TFLOPS 0.4TB RAM 50TB SSD 40Gbps Eth 0.7kW

pre cambrian cooling inc.

backplane

SSD SoC Carrier Thermal Compound

SLIDE 32

deep fried

SLIDE 33

The build TEGRA X1 TESLA K40

1056 Nodes

Tegra X1 4 GB RAM 512 GB SSD

22 Switches

2 x 10 GbE SFP+ 48 x 1 GbE

11 Pods

15M Ground Loop 50L Mineral Oil

50 Servers

2 x Tesla K40 2 x E5-2660v3 6 x 2TB SATA 64 GB RAM

3 Switches

4 x 40 GbE QSFP

36 x 10 GbE SFP+

3 Racks

Just a rack

SLIDE 34

Super green ? Super green. TEGRA X1 TESLA K40

$350 kilo

$310k Hardware $40k Infrastructure

12.4 kW

11.9 kW Hardware

0.5 kW Cooling

$1,056 kilo

$816k Hardware $58k Infrastructure

57.5 kW

44.3 kW Hardware 13.2 kW Cooling