EECS 192: Mechatronics Design Lab Discussion 7: Camera GSI: Justin - - PowerPoint PPT Presentation

EECS 192: Mechatronics Design Lab

Discussion 7: Camera GSI: Justin Yim 6 & 7 Mar 2019 (Week 7)

1 Line Sensing 2 Embedded Software 3 Summary

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 1 / 30

Line Sensing

Line Sensing

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 2 / 30

Line Sensing Problem Setup Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 3 / 30

Line Sensing Problem Setup

Let’s look at one specific frame

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 4 / 30

Line Sensing Problem Setup

frame[100] - mean(frame[100])

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 5 / 30

Line Sensing Camera Calibration

Which pixels correspond to which points in front of the car?

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 6 / 30

Line Sensing Camera Calibration

◮ Pinhole camera model ◮ Can calibrate centimeters-to-pixels gain for a fixed camera distance

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 7 / 30

Line Sensing Argmax Based Detection

How can we find the index of the line?

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 8 / 30

Line Sensing Argmax Based Detection

Take the argmax!

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 9 / 30

Line Sensing Argmax Based Detection

Any potential issues?

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 10 / 30

Line Sensing Argmax Based Detection

What about- noise, track crossings, other bright objects, etc.?

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 11 / 30

Line Sensing Argmax Based Detection

What about- noise, track crossings, other bright objects, multiple peaks, etc.?

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 12 / 30

Line Sensing Gradients and Smoothing

I(x,t) = Intensity at pixel x What if we look at the derivative dI(x)

dx ?

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 13 / 30

Line Sensing Gradients and Smoothing

dI(x) dx

≈ I(x+1)−I(x)

1

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 14 / 30

Line Sensing Gradients and Smoothing

Convolution! ˙ I(x) = I(x + 1) − I(x) = I ∗ [1, −1]

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 15 / 30

Line Sensing Gradients and Smoothing

Convolution! ˙ I(x) = I(x + 1) − I(x) = I ∗ [1, −1]

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 16 / 30

Line Sensing Gradients and Smoothing

Convolution! ˙ I(x) = I(x + 1) − I(x) = I ∗ [1, −1]

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 17 / 30

Line Sensing Gradients and Smoothing

Convolution! What about all that high frequency noise?

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 18 / 30

Line Sensing Gradients and Smoothing

Low Pass Filter! I(x) ∗ LPF ∗ [1, −1] = I(x) ∗ (LPF ∗ [1, −1])

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 19 / 30

Line Sensing Gradients and Smoothing

Low Pass Filter (Gaussian)! I(x) ∗ LPF ∗ [1, −1] = I(x) ∗ (LPF ∗ [1, −1])

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 20 / 30

Line Sensing Gradients and Smoothing

Low Pass Filter (Gaussian)! High frequency noise gone!

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 21 / 30

Line Sensing Gradients and Smoothing

Laplacian of Gaussian (low-passed second derivative) Can also be approximated with difference of Gaussians Detect a bright line of the expected width

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 22 / 30

Line Sensing Temporal Differencing

What about... Temporal Differencing?

dI(x) dt

≈ I(t)−I(t−1)

∆t

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 23 / 30

Line Sensing Temporal Differencing

What about... Temporal Differencing? Good for removing constant bias in parts of the image.

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 24 / 30

Line Sensing Temporal Differencing

More Ideas!

◮ Difference of Gaussians (Bandpass- not just Lowpass) ◮ Cross Correlation with known signal ◮ Convolutional Neural Networks??

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 25 / 30

Line Sensing Issues

Potential Issues?

◮ What to do if there are multiple track crossings? ◮ Can’t find the line ◮ Other Issues?

Solutions!

◮ Maintain a history (previous line pos, camera scan, etc.) ◮ Ignore the problem(s) and go fast! ◮ Anything else?

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 26 / 30

Embedded Software

Embedded Software

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 27 / 30

Embedded Software Modularity

Modularity

◮ Code is starting to get complicated

◮ Pthreads, PRU, UART, GPIO, ADC etc.

◮ Many tasks to run on a single core CPU

◮ Velocity Measurement ◮ Line Finding ◮ PID Controllers (steering and velocity) ◮ Telemetry ◮ Others? Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 28 / 30

Embedded Software Modularity

General Tips

◮ Remove unecessary threads. This will make debugging easier later on. ◮ Build modular code, test components in isolation ◮ Minimize dynamic memory allocation aka malloc & free (memory

fragmentation)

◮ Save logfiles ◮ Keep things simple!

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 29 / 30

Summary

Summary

◮ Many ways to do line detection (max, differencing, gaussian

smoothing, temporal differencing, etc.)

◮ Camera calibration ◮ Embedded Software tips

Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 30 / 30