Nautilus Automates, You Analyze Reproducible Analysis of Sensor Data - - PowerPoint PPT Presentation

▶
nautilus automates you analyze
SMART_READER_LITE
LIVE PREVIEW

Nautilus Automates, You Analyze Reproducible Analysis of Sensor Data - - PowerPoint PPT Presentation

Feb 22, 2024 126 likes •474 views

Nautilus Automates, You Analyze Reproducible Analysis of Sensor Data in R Dr. Benjamin Ashwell Dr. Kevin Kirshenbaum 21 March, 2018 We need to know how well sensors that provide positional data work 1 We need to know how well sensors that

slide-1
SLIDE 1

Nautilus Automates, You Analyze

Reproducible Analysis of Sensor Data in R

  • Dr. Benjamin Ashwell
  • Dr. Kevin Kirshenbaum

21 March, 2018

slide-2
SLIDE 2

We need to know how well sensors that provide positional data work

1

slide-3
SLIDE 3

We need to know how well sensors that provide positional data work This apparently simple goal is actually very complicated

2

slide-4
SLIDE 4

3

Track-to-truth assignment is often tedious and ambiguous

Latitude Longitude

slide-5
SLIDE 5

4

Who is this track tracking? How should we evaluate its track accuracy?

Track-to-truth assignment is often tedious and ambiguous

Latitude Longitude

slide-6
SLIDE 6

5

Track-to-truth assignment is often tedious and ambiguous

When was this target detected? Latitude Longitude

slide-7
SLIDE 7

The things that matter for sensor performance change with the mission

6

Surveillance radars Fire control radars

slide-8
SLIDE 8

7

Target A Target B Track 1 Track 2 Target C Track 3 Target D

Nautilus takes tracks and truth…

slide-9
SLIDE 9

… and assigns tracks to the most likely targets

8

Target A Target B Track 2 Target C Target D Track 3 Track 1

slide-10
SLIDE 10

With track-to-truth assignments, we can start answering questions

9

Target Name Detection Range (km) Detection Time Avg Position Error (m) F-18 2.16 1/1/2015 0:15 46.36 EA-18G 1.58 1/1/2015 0:22 45.6 P-8A 2.58 1/1/2015 0:07 40.54 LCS 4 4.2 1/1/2015 1:13 31.46

slide-11
SLIDE 11

With track-to-truth assignments, we can start answering questions

10

USS Good Guy

slide-12
SLIDE 12

Nautilus produces a number of different plots

11

slide-13
SLIDE 13

Our approach, Nautilus, compares sensor reports to reality

12

Nautilus is designed for operational testing, so we’re interested in the correctness of actionable information provided by the sensor Nautilus is system agnostic

slide-14
SLIDE 14

What you need to use Nautilus

13

  • A basic understanding of R
  • Truth data for ownship (the sensor) and targets: time,

latitude, longitude, altitude

  • The sensor tracks: time, latitude, longitude, altitude, track

ID

  • Only 2 (or 3) user inputs to assign tracks to targets
  • Patience and a willingness to experiment
slide-15
SLIDE 15

Which target is each track following?

14

Target A Target B Track 1 Track 2 Target C Track 3 Target D

slide-16
SLIDE 16

Step 1: Nautilus assigns each sensor point to the closest target

15

Target A Track 2 Target C Track 3 Target D Track 1 Target B

slide-17
SLIDE 17

Applies a cutoff distance: any sensor point ‘wronger’ than δ is a false track

16

Target A Track 2 Target C Track 3 Target D Track 1

δ

Target B

slide-18
SLIDE 18

Step 2: Removes points for which we have no nearby truth data

17

Target A Track 2 Target C Track 3 Target D Track 1 Target B

δ

slide-19
SLIDE 19

A small cutoff may unfairly reduce error estimates…

18

Target A Track 2 Target C Track 3 Target D Track 1 Target B

δ

slide-20
SLIDE 20

But a large cutoff may include additional false track points

19

Target A Track 2 Target C Track 3 Target D Track 1 Target B

δ

slide-21
SLIDE 21

Changing the method to a window method will smooth the target assignment

20

Target A Target B Track 1 Track 2 Target C Track 3 Target D t = τ t = τ - α t = τ + α

slide-22
SLIDE 22

21

Target A Target B Track 1 Track 2 Target C Track 3 Target D

Changing the method to a window method will smooth the target assignment

slide-23
SLIDE 23

22

Target A Target B Track 1 Track 2 Target C Track 3 Target D

Changing the method to a window method will smooth the target assignment

slide-24
SLIDE 24

Adjusting window size may change results

23

Target A Track 2 Target C Track 3 Target D Track 1 Target B

slide-25
SLIDE 25

Adjusting window size may change results

24

Target A Track 2 Target C Track 3 Target D Track 1 Target B

slide-26
SLIDE 26

Adjusting window size may change results

25

Target A Track 2 Target C Track 3 Target D Track 1 Target B

slide-27
SLIDE 27

How do we know our assignments are right?

26

Because it is sensor system agnostic, Nautilus cannot judge the reasonability of its answers

Nautilus automates, you analyze

slide-28
SLIDE 28

R Demonstration

27

slide-29
SLIDE 29

Our code is modular – use the pieces that you find useful

28

  • Automatically calculate values like relative bearing,

target aspect, and slant range between targets

  • Useful interactive figures for event reconstruction
  • Import and export data from/to SIMDIS
slide-30
SLIDE 30

Backup

29

slide-31
SLIDE 31

What do we even mean by performance?

30

slide-32
SLIDE 32

What do we even mean by performance?

31

slide-33
SLIDE 33

Nautilus tries to do what your eyes do automatically

32

1. Calculate the distance from each track to each target (User specifies assignment method and cutoff distance)

  • 2. Assign track to closest target (caution: reasonable

assumption)

  • 3. Flag false tracks (distance > δ)
  • 4. Generate figures and diagnostics