Deploying ASKAP's Phased Array Feeds Lessons from the Field Dr. Aaron Chippendale | 16 Sept. 2019 Australia’s National Science Agency
36 Reflector Antennas Australian Square 12 m diameter aperture Kilometre Array 22 m to 6 km baselines 3-axis mounts Pathfinder Phased Array Feeds World's fastest survey 188 receptors per reflector radio telescope 6,768 radio signals × 600 MHz bandwidth Radio frequency over fibre (...and most complex) Digital Signal Processing 6,912 direct sampled inputs 72 beams × 36 PAFs × (336 × 1 MHz) 3,690 DSP FPGAs, 130 Tbps raw IO, 210 kW Data rate of a megacity!
BYO water, electricity, hum
Key lessons Prototypes Scaling Things that work in Scaling to production prototypes cause less before prototype trouble. integration causes headaches. Expect new problems at full scale. Staging Flexibility … in building large, complex Plan a design refresh Flexibility of subsystems systems with major subsystems after deploying and can inhibit full-system operating a 10% system flexibility. Specify that are first of kind or risk a drastic pivot. standard modes and Plan and heed robust configs early or risk reviews. delays. Test Systems Automation Complete 1-antenna Continuously integrate, test and deploy and partial 3-antenna test platforms are everything. Script, log, indispensable. visualise and learn from everything.
Synchronisation challenges over 36 PAFs Massive distributed systems dictate asynchronous data transport. Advantages: • opportunity for COTS hardware • free running clocks don't correlate But data synchronisation is: • far more complex • harder as system scales Problems manifest as: • setup/hold violations on FIFO resets • buffer over/under-runs • clock-domain crossing issues • intermittent and distributed issues that don't show up until the system reaches a certain size We are working to reduce the likelihood of errors and automate their detection and correction with an on-dish noise source.
RF Link Problems for 36 PAFs (10 Apr 19) • 95% work fine! • 1.7% failed LNAs (lightning) • Replacing 2-channel receiver module (domino) fixes most issues
LNA Failures for 36 PAFs (10 Apr 19) 'Red' leg 4 (service) • LNA bias abnormally high (by 13 mA) • Distorted bandpass • Confirmed lightening event on worst affected antenna • Affects edge elements • Low impact on FOV • Letting it be for now
RFoF Transmitter Dropouts • Domino RF TX power suddenly high/low on many ports • Affects approximately 1 of 36 PAFs each day • Some antennas affected more frequently • Automatically flag affected antennas • Planning to automatically restart affected PAFs • Investigating root cause (power/comms glitches?)
Comms Issues COTS vs Bespoke PSU • Production/deployment • Designed bespoke linear PSUs commenced before for the PAFs and TECs, completing design of the PSU minimising transmission losses and TEC subsystems -> several and amount of equipment in modifications required antenna pedestals • Now losing comms to about • Backtracked to a COTS one PAF/PSU/TEC unit every switched-mode lab supply for few months the TEC subsystem • RFoF control link to PAF meets • A COTS PAF PSU would have original temperature spec but significantly reduced controlling rack temperatures debugging time at the cost of has been harder than higher ongoing electricity use expected
GAMA-23 Multi-beam Image interactive version Galaxy and Mass Assembly, 215 megapixels, 2 × 8 h, 36 antennas
3D Image of a Complex Field 50 TB raw data, 36 antennas, 10 h, 7 galaxies rich in H I ,
Thank you Astronomy and Space Science Dr. Aaron Chippendale Group Leader, Signal Processing Technologies +61 2 9372 4296 Aaron.Chippendale@csiro.au csiro.au/cass Australia’s National Science Agency
Recommend
More recommend
