There is No LSST Scheduler Peter Yoachim University of Washington, - - PowerPoint PPT Presentation

▶
there is no lsst scheduler
SMART_READER_LITE
LIVE PREVIEW

There is No LSST Scheduler Peter Yoachim University of Washington, - - PowerPoint PPT Presentation

Dec 23, 2022 184 likes •632 views

There is No LSST Scheduler Peter Yoachim University of Washington, LSST 99% recycled slides from DESC last month Traditional telescope scheduling Independent ranked observing proposals Discrete time blocks Typically over-subscribed

slide-1
SLIDE 1

There is No LSST Scheduler

Peter Yoachim University of Washington, LSST

99% recycled slides from DESC last month

slide-2
SLIDE 2

Traditional telescope scheduling Independent ranked observing proposals Discrete time blocks Typically over-subscribed

23:00-24:00 0:00 – 1:00 1:00-2:00 2:00-3:00 Proposal #1 Proposal #2 Proposal #3 Proposal #4 Proposal #5 Proposal #6

slide-3
SLIDE 3

23:00-24:00 0:00 – 1:00 1:00-2:00 2:00-3:00 Proposal #1 (not up) Scheduled Scheduled X Proposal #2 X X X Scheduled Proposal #3 (not dark) X X X Proposal #4 (not up) X X X Proposal #5 Scheduled X X X Proposal #6 X X X X

Given proposal constraints, try to fill in the table and maximize “TAC Happiness”. LCOGT, ZTF are solving this type of scheduling problem with Gyrobi Mixed Linear Programming

slide-4
SLIDE 4

23:00-24:00 0:00 – 1:00 1:00-2:00 2:00-3:00 Proposal #1 (not up) Scheduled Scheduled X Proposal #2 X X X Scheduled Proposal #3 (not dark) X X X Proposal #4 (not up) X X X Proposal #5 Scheduled X X X Proposal #6 X X X X

LSST does not lend itself to this kind of scheduling

  • We have no TAC, we’re not oversubscribed
  • Time blocks would be only 30s, 2 million cells is a lot
  • We need to minimize slew times
  • Need to respond to weather/clouds
slide-5
SLIDE 5

23:00-24:00 0:00 – 1:00 1:00-2:00 2:00-3:00 Proposal #1 (not up) Scheduled Scheduled X Proposal #2 X X X Scheduled Proposal #3 (not dark) X X X Proposal #4 (not up) X X X Proposal #5 Scheduled X X X Proposal #6 X X X X

We can’t really cast the LSST survey as a traditional telescope scheduling problem, so we have no scheduler

slide-6
SLIDE 6

We still need to point the telescope Given:

  • current conditions
  • future conditions
  • progress to date
  • how we want to take observations
  • what we want the final survey to look like

Decide where we should observe. If you see the word “predict”, you should think Machine Learning. If you see “decide”, you should think Artificial Intelligence.

slide-7
SLIDE 7

Scheduling LSST with Artificial Intelligence Resources (what we should have called it rather than “feature based scheduler”)

slide-8
SLIDE 8

Top level of the AI, a decision tree Observe a DD field? Observe a large block of sky (and repeat)? Observe a single pointing?

Pick the DD sequence that reports highest reward Pick block with highest reward Pick observation with highest reward

slide-9
SLIDE 9

The DD logic is currently simple:

  • Are the filters needed loaded?
  • Is the LMST within acceptable range?
  • Has twilight not started?
  • Is there enough time before the next twilight

starts?

  • Has the DD not exceeded it’s limit of total number
  • f observations fraction? (~2% limit)
  • Has it been N days since last observed?
  • Is it not cloudy?

If it checks out, returns a sequence to observe (currently no recovery for interruptions, could add that)

slide-10
SLIDE 10

For surveying large areas Markov Decision Process (MDP): Mathematical framework for decision making.

  • Given the state of the system, pick the maximum “reward”
  • With good basis functions, there are few free parameters
  • Easy to optimize behavior
  • Formalism of “features” -> “basis functions” -> “Decision function”

easy to map to OO-python implementation

slide-11
SLIDE 11

A quick example: I ran a 30-day simulation

  • DD fields on
  • Only r-band.
  • Observe blocks in pairs (if time)
  • Just greedy snake otherwise

Jupyter notebook can be seen here: http://ls.st/c6k

slide-12
SLIDE 12

1 0.46 0.15 A HEALpix map that defines the desired distribution of

  • bservations

Nside=32 1.8 degree resolution 12,288 pixels (5292 pointings in our usual tesselation)

slide-13
SLIDE 13

The actual distribution

  • f observations after 30

days

slide-14
SLIDE 14

Survey Progress Basis Function: Scaled goal map minus actual observation map.

slide-15
SLIDE 15

We can take HEALpix maps of airmass, seeing, and sky brightness to generate a map of the 5-sigma limiting depth.

slide-16
SLIDE 16

Depth Basis Function: Compare the current 5- sigma map to the depth when it’s dark time on the meridian with nominal seeing. Good seeing + dark time at this moment means the best is ~0.3 mags better. High airmass is 0.74 mags worse than the typical meridian dark time.

slide-17
SLIDE 17

Slewtime Basis Function

slide-18
SLIDE 18

Filter change basis function (not relevant here since we are only r-band, but usually have a basis function for staying in a filter)

slide-19
SLIDE 19

Final Reward Function linear combination of basis functions.

  • Survey progress (0.3)
  • 5-sigma depth (3)
  • Slewtime (3)
  • Filter change (3)

Only 4 free parameters control the behavior of the AI. Small parameter space to explore, intuitive to

  • ptimize.
slide-20
SLIDE 20

Once we have the reward map, a decision function uses it to take an action. Each night, generate a unique mapping of HEALpix to pointing. Decision function could take the top HEALpix, or a block in some azimuth range.

slide-21
SLIDE 21

If we are selecting a block to observe, run convex-hull traveling salesperson to reduce slewtime.

slide-22
SLIDE 22

Undithered OpSim fields OpSim with HexDither SLAIR full-sky dither

slide-23
SLIDE 23

Again, near the SCP WFD boundary.

slide-24
SLIDE 24

We can just add a dimension for each filter. Compute the basis functions and reward functions for each filter. Now decision function picks the highest reward in space and filter.

slide-25
SLIDE 25

Results from selecting large regions (and observing pairs) Observe a DD field? Observe a large block of sky for ~22 min (and repeat)? Observe a single observation?

DD1, DD2, DD3, DD4, … u, g+g, r+r, i+i, z, y Or u, g+r, r+i, i+g, z, y u, g, r, i, z, y

slide-26
SLIDE 26

Taking pairs in the same filter, 2.2 million visits

slide-27
SLIDE 27

Pairs in mixed filters, 2.1 million visits

slide-28
SLIDE 28

Filters used through each night. Moon phase plotted on the bottom. Note lots of y and z at full moon. Few filter changes through night.

slide-29
SLIDE 29

Taking pairs in different filters Depth basis function still forcing y and z at full moon, but getting lots of colors in night

  • therwise.
slide-30
SLIDE 30

Alt sched for comparison Not using red filters in bright time. Bug or feature?

slide-31
SLIDE 31

Rolling Cadence Experiment Before, define a goal target map

slide-32
SLIDE 32

Now, have two maps, one for even years, one for odd.

slide-33
SLIDE 33

Rolling cadence results (from Friday afternoon!):

  • 2.26 Million visits.
  • Looks good on SRD requirements
  • edge-effect so we end up with a always high

cadence strip (bug or feature?)

slide-34
SLIDE 34

How to change the AI behavior

  • Change goal survey footprint maps
  • Adjust weights on basis functions
  • Increase slewtime weight, get more observations
  • Add new basis function(s)
  • Add a basis function that prefers observations in places

that haven’t been observed for 2-3 days.

  • Update the survey progress basis function to use

different maps on even/odd years for rolling cadence

  • Add a new survey tier
  • If we are in twilight, go use some totally different logic to

pick observations

  • Have a top tier for ToO observations
slide-35
SLIDE 35

Conclusions:

  • Should have a new baseline based on

SLAIR soon, so folks can start hacking off

  • f that to put in their own strategies.
slide-36
SLIDE 36
slide-37
SLIDE 37
slide-38
SLIDE 38
slide-39
SLIDE 39

23:00-24:00 0:00 – 1:00 1:00-2:00 2:00-3:00 Proposal #1 (not up) Scheduled Scheduled X Proposal #2 X X X Scheduled Proposal #3 (not dark) X X X Proposal #4 (not up) X X X Proposal #5 Scheduled X X X Proposal #6 X X X X

Grid scheduling doesn’t make sense for most of LSST, but it might work well for Deep Drilling fields. If we had other specific cadence pointings, could classically schedule them all then use the AI to fill in gaps.

At 10 minute resolution, can break the 10 year survey into ~260k time steps. Could be a good Gurobi optimization problem.

slide-40
SLIDE 40

Summary

  • I have beta SLAIR simulations
  • Pairs in blocks
  • Pairs in different filters
  • Rolling cadence
  • Potential ToO interrupts

https://lsst-web.ncsa.illinois.edu/sim- data/beta_slair_surveys/ http://ls.st/sg5 For wide area surveys, try to think in terms of building and training a decision process, not

  • ptimizing a schedule grid.

Beta = I ignored cable wrap and have non-standard DB schema

slide-41
SLIDE 41

Let’s hack!

  • Rolling cadence
  • Desired cadence basis function (2.5-3.5 days?)
  • ToO simulation
  • Full sky template collection?
  • Twilight survey
  • DDF scheduling, dithering
  • Camera rotational dithering
  • ComCam surveys
  • Ubercal basis function (make sure we get enough
  • bservations per year to close sky)
  • Year 1 survey (build up difference image

templates)

slide-42
SLIDE 42

Let’s be ridiculous with fonts

slide-43
SLIDE 43