[PPT] - SKARAB2 AUGUST 2017 Francois Kapp (and the DBE team, particularly PowerPoint Presentation

SLIDE 1

AUGUST 2017 Francois Kapp (and the DBE team, particularly Adam and Jason) (also using a few SKAO slides)

SKARAB2

www.ska.ac.za

SLIDE 2

SKA– Key Science Drivers: The history of the Universe

Cosmic Dawn (First Stars and Galaxies) Galaxy Evolution (Normal Galaxies z~2-3) Cosmology (Dark Energy, Large Scale Structure) Cosmic Magnetism (Origin, Evolution) Cradle of Life (Planets, Molecules, SETI) T esting General Relativity (Strong Regime, Gravitational Waves) Exploration of the Unknown

Broadest range of science of any facility, worldwide

Image and text by SKA Organisati

SLIDE 3

www.ska.ac.z a

Overview

Square Kilometre Array

3 sites: 2 telescopes + HQ = 1 Observatory
Now in design Phase: ~ €200M; 600 scientists + engineers
Phase 1
Construction: 2019 – 2024
Construction cost cap: €674.1M (infmation-adjusted)
Operations planning now underway
MeerKat integrated
Observatory Development Programme
SKA Regional centres (locally funded)
Phase 2: start mid-2020s
Advanced Instrumentation Program to mature relevant technologies now
~2000 dishes across 3500km of Southern Africa
Major expansion of SKA1-Low across Western Australia

SLIDE 4

SLIDE 5

SKA HQ in UK

SLIDE 6

SKA1-LOW
50 – 350 MHz
Phase 1: ~130,000 antennas
across 40 - 50km
SKA1-Mid:
350 MHz – 24 GHz
Phase 1: 130 15-m dishes

across 120km

Integration of 64 dish MeerKAT

to complete phase 1 array

SKA telescopes in AUS & RSA

SLIDE 7

SKA Low prototype

SLIDE 8

Space for SKA

SLIDE 9

SKA Mid prototype: Foundations

SLIDE 10

SKA Mid prototype: Panel molds

Picture: CETC54

SLIDE 11

www.ska.ac.z a

SKAO is the central design authority
For SKA 1:
Dish – The physical dishes, feeds and digitisation
SaDT – Signal and Data T

ransport (Included Synchronisation and Timing)

CSP – Central Signal Processing (Correlator, Beamformers and Non-imaging

processing)

SDP – Science Data Processing
INFRA SA
INFRA AUS
LFAA – Low Frequency Aperture Array
TM – T

elescope Manager

AIV – Assembly, Integration and Verifjcation
FOR the Advanced Instrumentation Program (SKA 2):
MFAA
WBSPF
PAF
Consortia are performing element designs

SKA Consortia

SLIDE 12

www.ska.ac.z a

Now preparing for the element and system Critical Design Reviews
Starting this year and extending through 2018
Creation of a new Inter-governmental Organisation (IGO)
Plan is to have agreements for the IGO in place this year
Followed by ratifjcation in each member country
IGO comes into force early 2019
Construction phase begins
“CDR T
morrow”

SKA: What is next?

SLIDE 13

1 3

From Alistair McPherson’s presentation at the SKA Engineering Meeting 2017 (and 2016)

“Gentlemen, we have run out of money . It's time to start thinking.” ― Ernest Rutherford

SLIDE 14

SKA1 construction provisional totals – Phil Diamond SKAO

Footer text E l e me n t E s t i ma t e ( M E u r

i

n c l . c

n

t i n g e n c y )

F

e b 2 1 7 E s t i ma t e ( M E u r

i

n c l . c

n

t i n g e n c y )

J

u n 2 1 7 C h a n g e % C h a n g e R e a s

n

A I V 3 2 . 8 3 4 . 3 1 . 6 5 P r

d

u c t A s s u r a n c e a n d A d m i n f u n c t i

n

s i n c r e a s e d p e r C

s

t R e v i e w C S P 1 2 5 . 9 1 1 . 1

1

5 . 8

1

3 F r e q u e n c y s l i c e a r c h i t e c t u r e f

r

M i d . C B F a d

p

t e d ( E C P

1

7 1 7 i n t r a n s i t ) D S H 1 7 3 . 7 1 7 3 . 2

.

5 I N A U 9 6 . 9 4 . 1

1

. 9

2

I N S A 6 7 . 9 6 8 . 4 . 5 1 L F A A 1 7 . 9 1 1 1 . 7 3 . 8 4 S h i p p i n g c

s

t s i n c l u d e d p e r C

s

t R e v i e w S a D T 6 6 . 7 5 7 . 1

9

. 6

1

4 U p d a t e d v e n d

r

q u

t

e s ; s

m

e r e

u

s e

f

M e e r K A T t i m e s c a l e ; r e d u c e d c

m

p

n

e n t c

s

t s ; r e fj n e d s

f

t w a r e l a b

u

r c

s

t s S D P 1 1 4 . 5 1 1 4 . 5 . T M 4 3 . 7 4 3 .

.

6

1

T

t

a l s 8 2 9 . 8 6 . 4

2

2 . 6

3

€917M  €806M

SLIDE 15

Updated CCP list (page 1/2)

W S / Origin Descript ion LOW / M ID / COMM ON Science Implication Science Impact 5 .3 9

INFRA_SA Renew able energy t

out

er dishes M ID None 1

5 .3

M axim ise use of code produced during Pre-Const ruct ion COM M ON None 1

5 .3 8

Sim plify DDBH LOW LOW None 1

5 .3 8

Sim plify DDBH M ID M ID None 1

5 .2 5 .2

Reduce PSS-M ID: A, 7 5 nodes t

5

nodes M ID Likely none, or sm all reduct ion of pulsar search param et er space. 1

5 .2 5 .2

Reduce PSS-LOW : A, 2 5 nodes t

1

6 7 nodes LOW Likely none, or sm all reduct ion of pulsar search param et er space. 1

5 .3 5

Reduce CBF-M ID: Freq. Slice variant of CSP design vs. M eerKAT-based design M ID None 1

5 .1 9

M ID Frequency and Tim ing Standard: SaDT solution vs. M eerKAT-based solut ion M ID None 1

5 .3 6

M ID SPF Digit isers: DSH solut ion vs. M eerKAT-based solution M ID None 1

5 .2 6/ 5 .2 9

LOW RPF: Early Digit al Beam Form at ion vs. Analogue Beam Form at ion LOW None 1

2

LOW Ant enna: Log Periodic Design vs. Dipole Design LOW None of t he current designs m eet the L1 requirem ents 3

8

SDP- HPC: Deploy 2 Pflops (rat her t han 2 60 Pflops) COM M ON Low er allow ed dut y cycle for HPC- int ensive observat ions. 2

5 .2 4 .3

Reduce Bm ax M ID from 1 5 t

1

2 km : Case A, rem

ve 3

dishes, but keep infra t

1

5 km M ID Reduct ion of m axim um achievable resolut ion by 2 %, although can be part ially recovered w it h dat a w eighting and longer int egrat ion tim es. 2

5 .2 4 .2

Reduce Bm ax M ID from 1 5 t

1

2 km : Case B, rem

ve infra, but

add dishes t

core

M ID Reduct ion of m axim um achievable resolut ion by 2 %, although can be part ially recovered w it h dat a w eighting and longer int egrat ion tim es. 2

5 .2 4 .1

Reduce Bm ax M ID from 1 5 t

1

2 km : Case C, rem

ve infra, rem
ve dishes

M ID Reduct ion of m axim um achievable resolut ion by 2 %, although can be part ially recovered w it h dat a w eighting and longer int egrat ion tim es. 2

5 .5 .2

Reduce M ID Band 5 feeds: A, from 1 3 to 6 7 M ID Placem ent to be det erm ined based on full com m unit y consultat ion. 2

5 .2 5 .2

Reduce PSS-LOW : B, 1 6 7 nodes t

1

2 5 nodes LOW Likely reduct ion in processed PSS beam num ber (1 .3 x) or pulsar search param et er space 2

5 .2 5 .2

Reduce PSS-M ID: B, 5 nodes t

3

7 5 nodes M ID Likely reduct ion in processed PSS beam num ber (1 .3 x) or pulsar search param et er space 2

SLIDE 16

Updated CCP list (page 2/2)

8

SDP- HPC: Deploy 1 5 Pflops (from 2 Pflops) COM M ON Low er allow ed dut y cycle for HPC- int ensive observations. 3

5 .3 .0

Reduce Bm ax LOW t

5

km : A, rem

ve infra, add 1

8 st ations t

core

LOW Science Risk t

EoR: Bm

ax. 3

5 .3 .0

Reduce Bm ax LOW t

5

km : B, rem

ve 1

8 st at ions LOW Science Risk t

EoR: Bm

ax 3

5 .3 a

Reduce Bm ax LOW t

4

km : C, rem

ve next

1 8 stat ions LOW Science Risk t

EoR: Bm

ax 3

8

SDP- HPC: Deploy 1 Pflops (from 1 5 Pflops) COM M ON Low er allow ed dut y cycle for HPC-int ensive observat ions. 4

8

SDP- HPC: Deploy 5 Pflops (from 1 Pflops) COM M ON Low er allow ed dut y cycle for HPC- int ensive observations. 4

5 .3 1

Reduce CBF-LOW BW : A, 3 t

2

M Hz LOW Longer observing t im es for cont inuum applicat ions (1 .5 x) 4

5 .2 5 .2/ D e e p e r S a v in g s

Reduce PSS-LOW : C, 1 2 5 nodes to 8 3 nodes LOW Likely reduct ion in processed PSS beam num ber (2 x) or pulsar search param eter space 4

5 .2 5 .2/ D e e p e r S a v in g s

Reduce PSS-M ID: B, 3 7 5 nodes t

2

5 nodes M ID Likely reduct ion in processed PSS beam num ber (2 x) or pulsar search param eter space 4

5 .1 3 .2

Reduce Bandw idth out put

f band 5

t

2

.5 GHz M ID Longer Band 5

bserving t

im es for som e applicat ions (2 x) 4

5 .35

Reduce M ID CBF and DSH BW : 5 to 1 .4 GHz M ID Longer observing t im es to achieve continuum sensit ivit y in Band 5 (3 .6 x) 4

5 .2 4/ D e e p e r S a v in g s

Rem

ve 1

1 M ID Dishes from core M ID 1 % Array sensit ivit y loss in core 4

5 .3 0/ D e e p e r S a v in g s

Rem

ve 5

4 LOW st at ions from core LOW 1 % Array sensit ivit y loss in core 4

5 .2 4/ D e e p e r S a v in g s

Rem

ve addit

ional 1 1 M ID Dishes from core M ID 2 % Array sensit ivit y loss in core 4

5 .3 0/ D e e p e r S a v in g s

Rem

ve addit

ional 5 4 LOW st at ions from core LOW 2 % Array sensit ivit y loss in core 4

5 .2 4 .2

Reduce Bm ax M ID from 1 2 t

10

km : D, rem

ve infra, rem
ve next

3 dishes M ID Lose Science (Planet ary disks, High resolution Star Form ation) 4

5 .5 .1

Rem

ve M

ID Band 1 feeds: 1 5 to 0 M ID Lose Science (Cosm

logy, Galaxy

Evolut ion) 4

5 .5 .2

Reduce M ID Band 5 feeds: B, from 6 7 t

M ID Lose Science (Planet ary disks, St ar Form at ion) 4

SLIDE 17

“DCE” ‘proposal’

What would happen if we reused as much of MeerKAT as possible?
Goal is to have no impact on science
Only SKA1 Mid
Allowed half of Euro 674 M cost cap
Construction starts in 2019…
… but we only need to deploy a large scale system by 2022
SKARAB2 could be Virtex Ultrascale+ or Stratix 10 with HBM
SKARAB2 is a risk mitigation for SKARAB3, which is the preferred

family for SKA in ~2022 timeframe

Increase density to 4 devices per 1U
Power up to 130W per device, ~23kW per rack

SLIDE 18

A few key SKA CSP specifjcations

Up to 5GHz instantaneous bandwidth (band 5)
130 15m antennas + 64 13.5m MeerKAT antennas
Beamformers for:
VLBI – up to 4 beams with adjustable bandwidth
PSS – search. Up to 1500 beams at 300MHz

bandwidth

PST – timing. 16 beams at full bandwidth
Correlation
64k channels WB
Zoom windows (4, 8, 16, 32, 64, 128, 256 MHz), 16k

channels

SLIDE 19

SKARAB2 Block Diagram

FAN FAN

QSFP & Cage QSFP & Cage FPGA

FMC FAN FAN

QSFP & Cage QSFP & Cage FPGA

FMC FAN FAN

QSFP & Cage QSFP & Cage FPGA

FMC FAN FAN

QSFP & Cage QSFP & Cage FPGA

FMC PSU FAN FAN

SLIDE 20

SKARAB2 Hardware Proposal

SKARAB vs SKARAB2

S K A R A B S K A R A B 2 E t h e r n e t : 4 x 4 G b E E t h e r n e t : 2 x 1 G b E M e m

r

y : 4 G B H M C x 3 M e m

r

y : H B M F P G A : 1 x V i r t e x 7 F P G A : 4 x V i r t e x U l t r a S c a l e + D i m e n s i

n

s : 1 U D i m e n s i

n

s : 1 U P

w

e r C

n

s u m p t i

n

: 1 2 5 W P

w

e r C

n

s u m p t i

n

: 6 8 4 W C

l

i n g : F a n A i r F l

w

C

l

i n g : F r

n

t t

B

a c k F a n A i r F l

w

( T B D )

SLIDE 21

SKARAB vs SKARAB2 FPGA Specs

SKARAB Virtex 7 (XC7VX690T) 693120 Logic Cells 80 x SERDES

64 used for I/O mezzanine sites @ 10 Gbps 1 used for PCI-E to COM-E module site

1470 x 36Kb RAM Blocks (~52 Mb) 3600 DSP Slices 1927 pins 1000 single ended I/O SKARAB 2 Virtex UltraScale (VU33P/VU35P) 962000-1907000 Logic Cells 32-64 x SERDES

@ 32.75 Gbps

RAM Blocks (~23.6 - 47.3 Mb) UltraRam (90-180 Mb) HBM: 8GB 2880-5952 DSP Slices 208 - 416 single ended I/O 2-5 x 100G Ethernet

SLIDE 22

SKARAB FPGA Specs in Detail

SLIDE 23

SKARAB2 FPGA Specs in Detail

SKARAB 2 Virtex UltraScale (VU33P/VU35P – package compatible) 962000-1907000 Logic Cells 32-64 x SERDES

@ 32.75 Gbps

RAM Blocks (~23.6 - 47.3 Mb) UltraRam (90-180Mb) HBM: 8GB 2880-5952 DSP Slices 208 - 416 single ended I/O 2-5 x 100G Ethernet

SLIDE 24

Thanks

SKA South Africa, a Business Unit of the National Research Foundation, is supervising South Africa’s involvement in the SKA on behalf of the Department of Science & T echnology.