EECS 192: Mechatronics Design Lab Discussion 7: Camera GSI: Justin - PowerPoint PPT Presentation

EECS 192: Mechatronics Design Lab Discussion 7: Camera GSI: Justin Yim 6 & 7 Mar 2019 (Week 7) 1 Line Sensing 2 Embedded Software 3 Summary Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 1 / 30

Line Sensing Line Sensing Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 2 / 30

Line Sensing Problem Setup Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 3 / 30

Line Sensing Problem Setup Let’s look at one specific frame Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 4 / 30

Line Sensing Problem Setup frame[100] - mean(frame[100]) Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 5 / 30

Line Sensing Camera Calibration Which pixels correspond to which points in front of the car? Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 6 / 30

Line Sensing Camera Calibration ◮ Pinhole camera model ◮ Can calibrate centimeters-to-pixels gain for a fixed camera distance Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 7 / 30

Line Sensing Argmax Based Detection How can we find the index of the line? Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 8 / 30

Line Sensing Argmax Based Detection Take the argmax! Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 9 / 30

Line Sensing Argmax Based Detection Any potential issues? Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 10 / 30

Line Sensing Argmax Based Detection What about- noise, track crossings, other bright objects, etc.? Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 11 / 30

Line Sensing Argmax Based Detection What about- noise, track crossings, other bright objects, multiple peaks, etc.? Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 12 / 30

Line Sensing Gradients and Smoothing I(x,t) = Intensity at pixel x What if we look at the derivative dI ( x ) dx ? Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 13 / 30

Line Sensing Gradients and Smoothing dI ( x ) ≈ I ( x +1) − I ( x ) dx 1 Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 14 / 30

Line Sensing Gradients and Smoothing Convolution! ˙ I ( x ) = I ( x + 1) − I ( x ) = I ∗ [1 , − 1] Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 15 / 30

Line Sensing Gradients and Smoothing Convolution! ˙ I ( x ) = I ( x + 1) − I ( x ) = I ∗ [1 , − 1] Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 16 / 30

Line Sensing Gradients and Smoothing Convolution! ˙ I ( x ) = I ( x + 1) − I ( x ) = I ∗ [1 , − 1] Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 17 / 30

Line Sensing Gradients and Smoothing Convolution! What about all that high frequency noise? Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 18 / 30

Line Sensing Gradients and Smoothing Low Pass Filter! I ( x ) ∗ LPF ∗ [1 , − 1] = I ( x ) ∗ ( LPF ∗ [1 , − 1]) Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 19 / 30

Line Sensing Gradients and Smoothing Low Pass Filter (Gaussian)! I ( x ) ∗ LPF ∗ [1 , − 1] = I ( x ) ∗ ( LPF ∗ [1 , − 1]) Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 20 / 30

Line Sensing Gradients and Smoothing Low Pass Filter (Gaussian)! High frequency noise gone! Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 21 / 30

Line Sensing Gradients and Smoothing Laplacian of Gaussian (low-passed second derivative) Can also be approximated with difference of Gaussians Detect a bright line of the expected width Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 22 / 30

Line Sensing Temporal Differencing What about... Temporal Differencing? dI ( x ) ≈ I ( t ) − I ( t − 1) dt ∆ t Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 23 / 30

Line Sensing Temporal Differencing What about... Temporal Differencing? Good for removing constant bias in parts of the image. Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 24 / 30

Line Sensing Temporal Differencing More Ideas! ◮ Difference of Gaussians (Bandpass- not just Lowpass) ◮ Cross Correlation with known signal ◮ Convolutional Neural Networks?? Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 25 / 30

Line Sensing Issues Potential Issues? ◮ What to do if there are multiple track crossings? ◮ Can’t find the line ◮ Other Issues? Solutions! ◮ Maintain a history (previous line pos, camera scan, etc.) ◮ Ignore the problem(s) and go fast! ◮ Anything else? Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 26 / 30

Embedded Software Embedded Software Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 27 / 30

Embedded Software Modularity Modularity ◮ Code is starting to get complicated ◮ Pthreads, PRU, UART, GPIO, ADC etc. ◮ Many tasks to run on a single core CPU ◮ Velocity Measurement ◮ Line Finding ◮ PID Controllers (steering and velocity) ◮ Telemetry ◮ Others? Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 28 / 30

Embedded Software Modularity General Tips ◮ Remove unecessary threads. This will make debugging easier later on. ◮ Build modular code, test components in isolation ◮ Minimize dynamic memory allocation aka malloc & free (memory fragmentation) ◮ Save logfiles ◮ Keep things simple! Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 29 / 30

Summary Summary ◮ Many ways to do line detection (max, differencing, gaussian smoothing, temporal differencing, etc.) ◮ Camera calibration ◮ Embedded Software tips Ducky (UCB EECS) Mechatronics Design Lab 6 & 7 Mar 2019 (Week 7) 30 / 30