LLRF Embedded Development Brian Chase, Joshua Einstein-Curtis - - PowerPoint PPT Presentation

▶
llrf embedded development
SMART_READER_LITE
LIVE PREVIEW

LLRF Embedded Development Brian Chase, Joshua Einstein-Curtis - - PowerPoint PPT Presentation

Mar 30, 2024 151 likes •499 views

LLRF Embedded Development Brian Chase, Joshua Einstein-Curtis Controls Modernization Group 28 September 2018 Introduction LLRF is a high-speed, high-precision real time system. For the FNAL accelerator infrastructure, have relatively

slide-1
SLIDE 1

LLRF Embedded Development

Brian Chase, Joshua Einstein-Curtis Controls Modernization Group 28 September 2018

slide-2
SLIDE 2

Introduction

  • LLRF is a high-speed, high-precision real time system.
  • For the FNAL accelerator infrastructure, have relatively high

data rates

– Similar to beam instrumentation – Necessitated builds of a Labview client for commissioning, debug, and diagnostics (Backdoor/custom)

9/27/2018 LLRF Embedded Development 2

Example: CMTS has 4 ADC and 2 DAC channels per cavity. In addition to the raw data, there are also the baseband IQ and amplitude/phase signals, error, and controller intermediate stages 2 * (4*2 + 2 + 10) * 65 MSPS * 32-bit = 82 Gbps max 82/65 = 1.26 Gbps if 1 MSPS sampling 1.26 Gbps / 8 * 360 = 56 TB/hour 0.02 * 56 = 1.12 TB/hour if only 20 ms of 1MSPS data every second

slide-3
SLIDE 3

Real-time Needs

  • Hard Realtime  LLRF (timescale-dependent, of course)

– Failures not allowed

  • Bucket-resolution timestamping on data
  • Necessary to do extensive hardware testing or simulation

(Qemu, ARM simulator)

– Good debugging tools a necessity

  • Utilization and overloading

– Need for optimizations and good compilers

9/27/2018 LLRF Embedded Development 3

slide-4
SLIDE 4

vxWorks

  • MVME2400, MVME5500 with Kinetic Systems backplane
  • VxWorks 5.5, 6.1, 6.4, 6.9

– 5.5 running on MI, FAST, 6.4 on Recycler, 6.9 dev system provided to controls

  • Power PC 603e, 750 or 7457
  • Standardized communication between front ends enables faster
  • development. Shared libraries, etc.

– https://www-bd.fnal.gov/controls/micro_p/mooc_project/welcome.html

  • Protocol Compiler

– Custom for each device? Possibly leads to complexity and versioning issues – Each device has own receive stack, and each group develops their

  • wn

9/27/2018 LLRF Embedded Development 4

slide-5
SLIDE 5

Current Model

9/27/2018 LLRF Embedded Development 5

Spreadsheet LLRF Frontend sys_init.cpp Python Scripts-to-dabbel Modification Scripts Dennis’ Frontends Acnet and Front-end independently record and manage variables

slide-6
SLIDE 6

SoC MFC

9/27/2018 LLRF Embedded Development 6

slide-7
SLIDE 7

LLRF Control and Timing

  • VUCD

– Developed in mid-nineties – No replacement for current systems available – Timing and data decoding using backplane triggers – Multi-timer system – No planned replacement for current/deployed systems

  • Bucket resolution and accurate timestamping needed for

future diagnostics

9/27/2018 LLRF Embedded Development 7

slide-8
SLIDE 8

Build Environment

  • nova build environment (esd tools) well developed

– LLRF projects are all there, including configuration files for new SoC systems (spreadsheet register maps and defaults)

  • Need for a well-developed upgrade plan across departments
  • Need for supporting both legacy systems and new systems

using same tools?

  • Significant increases in performance with each controller, but

processor-specific programming can make porting difficult

9/27/2018 LLRF Embedded Development 8

slide-9
SLIDE 9

Revision Control

  • Currently part of build toolchains on nova
  • One of the largest issues we’ve run in to is revision and

versioning

– Separate versions for firmware, software, libraries – Storing git commits in f/w and software works well – Xilinx allows for same bitfile from same files

  • routing/placement hash/seed is based on files included in project
  • Enforcing this across all project levels is a challenge
  • PLC versioning? Software versioning? HDL versioning?

9/27/2018 LLRF Embedded Development 9

slide-10
SLIDE 10

Revision Control

  • LBL/LCLS-II and CERN have self-describing

firmware/software

– Register maps in control system and front-ends match

  • gzip’d JSON in memory on devices
  • CERN’s FESA and cheby
  • Acnet database
  • TFTP/remote image loading is crucial

– Limit wear on flash (limited write) devices

  • Remote/central logging is crucial

– See above

  • Verification and Continuous Integration are valuable tools

9/27/2018 LLRF Embedded Development 10

slide-11
SLIDE 11

What is plan for future?

  • Responsibility of each department?
  • High-speed data and diagnostics

– artDAQ? FTP? Something new? – DAQ analysis and storage – High-speed means what for each system?

  • Data lifetime?
  • Scope capabilities
  • Event identification
  • Design lifetime

– 20 years? 5 years? 1 year?

  • Tool availability and automation

– Build system for non-experts – Nova replacement? Command-line functionality?

9/27/2018 LLRF Embedded Development 11

slide-12
SLIDE 12

Looking Forward

  • What will be available in 2 years? 5 years? 10 years?
  • Need a long-term plan
  • Due to few resources at developer level, really need process

and procedure directed from managment

  • Hardware - should there be a standard developed across

groups?

– Shared FPGA/Processor boards? – Crate standards? – Physical interfaces?

9/27/2018 LLRF Embedded Development 12

slide-13
SLIDE 13
  • Designed for cell base

stations, radar, military

  • Single-chip includes all

necessary for an RF system, but might not meet performance requirements of modern accelerators.

  • Separate application and

real-time processors

  • Shared memory makes for

very powerful, high- bandwidth toolset

Xilinx Zynq Ultrascale+ RFSoC

9/27/2018 13 LLRF Embedded Development https://www.xilinx.com/products/silicon-devices/soc/rfsoc.html

slide-14
SLIDE 14

Conclusion

  • Concerns for future

– DAQ and timing/precision – Data retention and high-speed data management – Revision Control and Build Environment are significant concerns – Ease of integrating in to control system: shared protocols, development architectures, etc

  • Need a multi-year, division-wide path forward

– While still supporting legacy systems

  • Thanks!

9/27/2018 LLRF Embedded Development 14

slide-15
SLIDE 15

Extra Slides

9/27/2018 LLRF Embedded Development 15

slide-16
SLIDE 16

Switching to Fiber

  • Fiber benefits outweigh slightly increased cost
  • Decreased power
  • Higher bandwidth
  • Low cost per bandwidth
  • Relative ease of implementation given available tools

9/27/2018 LLRF Embedded Development 16

slide-17
SLIDE 17

Communication Proposals

  • 9.027 MHz RF synchronous frame rate

– Local clock source needed on each device

  • 1300 MHz harmonic as data rate
  • Need known frame size (ie 128-bit)
  • Previous State, Events, Beam pattern for next group
  • Single-source star
  • Send as timing/events -- Return line as MPS

– Data concentrators?

  • Throw out proposal
  • 3 links – DAQ, timing/MPS, control

9/27/2018 LLRF Embedded Development 17

slide-18
SLIDE 18

9/27/2018 LLRF Embedded Development 18

slide-19
SLIDE 19

Intel Cyclone V and Arria 10

9/27/2018 LLRF Embedded Development 19

slide-20
SLIDE 20

SoC MFC Test DAQ System

9/27/2018 LLRF Embedded Development 20

Courtesy Ed Cullerton

slide-21
SLIDE 21

Arm Product Line

9/27/2018 LLRF Embedded Development 21

https://www.silabs.com/documents/public/white- papers/Which-ARM-Cortex-Core-Is-Right-for-Your- Application.pdf

slide-22
SLIDE 22

Microsemi SmartFusion2

9/27/2018 LLRF Embedded Development 22

Single Event Upset Immune design

slide-23
SLIDE 23

LPC17xx Kinetis K60

NXP

9/27/2018 23 LLRF Embedded Development

i.MX

slide-24
SLIDE 24

Why not Pis?

  • The question is really: what does real-time mean?

– Bucket-level resolution and repeatability? – TCLK decoding accuracy? Event decoding accuracy?

  • What runs at each layer?

– Firmware/embedded RTOS – Software/application layer

  • Lifetime and long-term support

– Open Hardware

  • Interconnect standards

– SPI + I2C support? (1 SPI, 1 I2C, 1 UART) – Standard interconnects? – Bus/backplane protocols?

9/27/2018 LLRF Embedded Development 24

slide-25
SLIDE 25

DSP

  • LLRF requires higher-performance DSP and DAQ
  • Newer FPGAs are offering dedicated floating-point blocks

– Require use of special tools or overloading standard operators

  • Integrated simulation and verification environment being

more important

  • Collaboration requires the ability to transfer IP easily

9/27/2018 LLRF Embedded Development 25

slide-26
SLIDE 26

SoM Benefits

9/27/2018 LLRF Embedded Development 26

https://www.intel.com/content/www/us/en/programmable/products/soc/ecosystem/system-on-modules.html

slide-27
SLIDE 27

Linux (and variants)

  • Linux w/ preemptRT
  • RTAI (Linux as task)
  • NI Real-Time Linux (dual-schedulers)

9/27/2018 LLRF Embedded Development 27

slide-28
SLIDE 28

RTOS

  • FreeRTOS -- https://www.freertos.org/
  • Contiki (IoT) -- http://www.contiki-os.org/
  • MicroC/OS-IIx
  • Mbed (IoT)
  • RTEMS
  • vxWorks

– Now a Linux variant

  • MQX

9/27/2018 LLRF Embedded Development 28

slide-29
SLIDE 29

MPC603e

9/27/2018 LLRF Embedded Development 29

slide-30
SLIDE 30

MPC750

9/27/2018 LLRF Embedded Development 30

slide-31
SLIDE 31

MPC7457

9/27/2018 LLRF Embedded Development 31

slide-32
SLIDE 32

Processing Models

  • IOC+Epics

– “Dumb” front-end with managing IOC – Subscriber model

  • Crate processors

– Slot 0 manages all cards in crate. Advanced data processing happens ???

  • NADs

– Multiple models available

  • Acnet
  • Requirements on who does each portion is not defined

– Controls communication (MOOC(AN), libacnet, erlang, PC) – Debug communication (Labview, Python) – Front-end intelligence

9/27/2018 LLRF Embedded Development 32

slide-33
SLIDE 33