SMA Real-time Software Attila Kovcs SAO SMA Advisory Committee - - PowerPoint PPT Presentation

▶
sma real time software
SMART_READER_LITE
LIVE PREVIEW

SMA Real-time Software Attila Kovcs SAO SMA Advisory Committee - - PowerPoint PPT Presentation

Jun 02, 2023 94 likes •532 views

SMA Real-time Software Attila Kovcs SAO SMA Advisory Committee Meeting Cambridge, 1718 July 2018 SMA Advisory Committee Meeting Cambridge, 1718 July 2018 Attila Kovcs Objectives 1.Maintain 2.Fix 3.Improve 4.Enhance SMA

slide-1
SLIDE 1

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

SMA Real-time Software

Attila Kovács SAO

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018

slide-2
SLIDE 2

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Objectives

1.Maintain 2.Fix 3.Improve 4.Enhance

slide-3
SLIDE 3

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Overview: Architecture

hal9000 acc1 acc8 ... hcn

  • bscon

m5 engdb SWARM newdds tenzing

slide-4
SLIDE 4

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Overview: Architecture - 2

hal9000 acc1 acc8 ... command

slide-5
SLIDE 5

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Overview: Architecture - 3

hal9000 acc1 acc8 ... hcn

  • bscon

SWARM command tenzing

slide-6
SLIDE 6

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Overview: Architecture - 4

hal9000 acc1 acc8 ... newdds hcn tenzing

  • bscon

m5 engdb MRG SWARM command monitor

slide-7
SLIDE 7

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Overview: Stats

Language LynxOS Linux all C 264k 73k 365k b[a]sh 3.7k 16k 18k perl 7.8k 4.2k 13k [t]csh 4.3k

  • 10k

python

  • 2.6k

2.6k

  • ther

15k 2k 20k all 295k 98k 429k

slide-8
SLIDE 8

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Real-time Software Team

Thomas Mac Cooper Paul Grimes Ryan Howie Garrett ‘Karto’ Keating Attila Kovács Scott Paine Nimesh Patel Ram Rampasao Taco Bob Wilson

slide-9
SLIDE 9

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

  • 1. Maintain

Maintain

Keep the SMA fully operational as computers and hardware components get replaced, and as OS and 3rd-party software are updated.

slide-10
SLIDE 10

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Maintain: git version control

CVS → git in Fall 2017

  • up-to-date code base
  • includes runtime configurations files
  • in-tree builds
  • better cross-referencing
slide-11
SLIDE 11

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Maintain: git version control

slide-12
SLIDE 12

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Maintain: LynxOS

hal9000 acc1 acc8 ... hcn

  • bscon

m5 engdb SWARM newdds tenzing

slide-13
SLIDE 13

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Maintain: LynxOS

hal9000 acc1 acc8 ... hcn

  • bscon

m5 engdb SWARM

PowerPC / LynxOS

newdds tenzing

slide-14
SLIDE 14

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Maintain: LynxOS → Linux

  • Affects a lot of computers & code
  • LynxOS vs. Linux differences, peculiarities, kludges
  • PowerPC hardware (e.g. reflective memory, timing, servos)
slide-15
SLIDE 15

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Maintain: LynxOS → Linux

  • hal9000 (2018)

– PowerPC/LynxOS → Linux/x86_64 VM. – reflective memory → global redis database with quasi-

equivalent API (sma-rm)

– timing-hardware → ntp – test with hangar antenna first – array deployment by end of 2018…

  • acc1 – acc8 (2019?)
slide-16
SLIDE 16

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Maintain: Documentation

inline git / markdown wiki

  • perations log
slide-17
SLIDE 17

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Maintain: Documentation (inline)

slide-18
SLIDE 18

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Maintain: Documentation (git / markdown)

slide-19
SLIDE 19

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Maintain: Documentation (wiki)

slide-20
SLIDE 20

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Maintain: Documentation (operations log)

slide-21
SLIDE 21

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

  • 2. Fix

Fix

Fix problems that arise, and preemptively resolve issues identified in software.

slide-22
SLIDE 22

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Fix: Tracking of Solar System sources

Ephemeris time lookup offset (TT vs UT)

(explains why planets, asteroids, comets were not properly phase centered before.)

2017 Dec. 15 2017 Dec. 16

Phaethon

slide-23
SLIDE 23

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Fix: Nutation correction (NOVAS)

NOVAS 3.1 has much improved nutation model, improving systematic pointing by up to 3”. NOVAS 2.0 (2017 Dec. 16) NOVAS 3.1 (2017 Dec. 24)

slide-24
SLIDE 24

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Fix: Flagging

  • Expose hidden flags
  • Detect relevant issues
  • Aggregate flags for scans
  • Receiver-wise flagging support

System tracks ~20 critical antenna-based conditions

slide-25
SLIDE 25

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

  • 3. Improve

Improve

Increase the observing efficiency and scientific throughput of the SMA.

slide-26
SLIDE 26

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Improve: Overheads

  • “Sleepless” programs and scripts
  • Much faster & more reliable interferometric pointing
  • Faster & better source position checking
  • Faster priming (tuning)
  • Faster calibration (bandpass)
slide-27
SLIDE 27

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Improve: design

hal9000 acc1 acc8 ... newdds hcn tenzing

  • bscon

m5 engdb MRG SWARM command monitor

slide-28
SLIDE 28

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Improve: design

  • Less is more…

– More bug-proof – Easier to follow – Easier to maintain – Easier to extend

slide-29
SLIDE 29

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Improve: VLBI / EHT

  • Support EHT observations
  • Seamless EHT operation

model simulated

  • bservation
slide-30
SLIDE 30

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Improve: Regression testing

  • Test before deployment on live system
  • Unit tests for functions where appropriate
  • Simulator for real-time behavior
  • … but needs more man-power
slide-31
SLIDE 31

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

  • 4. Enhance

Enhance

Develop new capabilities to keep the SMA at the forefront of (sub)millimeter interferometry.

slide-32
SLIDE 32

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Enhance: wSMA - IF bandwidth

  • Prepare for increased bandwidth

– combine (extra SWARM segments & speed) – transfer (REDIS → direct TCP/IP stream) – archive

(dataCatcher)

– analyze (dataCatcher) – monitor (corrPlotter)

slide-33
SLIDE 33

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Enhance: wSMA - New receivers & tuning

  • Overhaul Rx tuning

– Focus on operations vs. engineering – Modular low-level (C) controlled via ‘scripts’ (Python) – Faster, simpler, & more reliable (e.g. lookup tables)

slide-34
SLIDE 34

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Enhance: New observing modes

  • On-the-Fly (OTF) mapping
  • Hybrid (Total Power + Interferometric) Imaging Mode
  • High time-resolution mode
slide-35
SLIDE 35

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Enhance: Hybrid Mode

A B Observing sequence: 0.16s - on A 0.32s - on B ~3 Hz chop Observing sequence: 0.64s – Walsh steps 1-64 λ/d < L < chop λ/D < L < λ/d Single Dish Interferometric

slide-36
SLIDE 36

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Enhance: Hybrid Mode

A B Observing sequence: 0.16s - A1: Walsh 1-16 on A 0.32s - B1: Walsh 1-16 on B 0.48s - A2: Walsh 17-32 on A 0.64s - B2: Walsh 17-32 on B 0.80s - A3: Walsh 33-48 on A 0.96s - B3: Walsh 33-48 on B 1.12s - A4: Walsh 48-64 on A 1.28s - B4: Walsh 48-64 on B + switching time…. A1 A2 B1 B2 λ/D < L < chop 2 interferometric fields (A,B) + 1 differential primary beam (A-B) mosaic to cover field

slide-37
SLIDE 37

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Enhance: Array / subarray control

  • Command the ‘array’ vs. antennas
  • Add/remove project antennas on the fly
  • Split mode (e.g. 6 science + 2 engineering)
slide-38
SLIDE 38

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Enhance: Online monitoring tools

slide-39
SLIDE 39

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Enhance: Online monitoring tools

corrPlotter (X11)

slide-40
SLIDE 40

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Enhance: Online monitoring tools

  • Web-based tools (login / firewall / X11 forwarding)
  • client-side plotting (server CPU load)
  • Conserve network bandwith (plotting)
  • Focus on operations vs engineering
slide-41
SLIDE 41

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

Enhance: Diagnostics and alarms

  • Diagnose sooner
  • Diagnose more
  • Notify smarter (operators vs. engineers / errors vs. warnings)
slide-42
SLIDE 42

SMA Advisory Committee Meeting Cambridge, 17–18 July 2018 Attila Kovács

THE END