EECS 192: Mechatronics Design Lab Discussion 8: Camera & MCUX - - PowerPoint PPT Presentation

▶

eecs 192 mechatronics design lab

EECS 192: Mechatronics Design Lab Discussion 8: Camera & MCUX - - PowerPoint PPT Presentation

Jun 27, 2023 271 likes •616 views

EECS 192: Mechatronics Design Lab Discussion 8: Camera & MCUX GSI: Varun Tolani 7 & 8 Mar 2018 (Week 8) 1 Line Sensing 2 Embedded Software/ MCUXpresso 3 Summary Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 1 /

slide-1

SLIDE 1

EECS 192: Mechatronics Design Lab

Discussion 8: Camera & MCUX GSI: Varun Tolani 7 & 8 Mar 2018 (Week 8)

1 Line Sensing 2 Embedded Software/ MCUXpresso 3 Summary

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 1 / 33

slide-2

SLIDE 2

Line Sensing

Line Sensing

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 2 / 33

slide-3

SLIDE 3

Line Sensing Problem Setup Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 3 / 33

slide-4

SLIDE 4

Line Sensing Problem Setup

Let’s look at one specific frame

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 4 / 33

slide-5

SLIDE 5

Line Sensing Problem Setup

frame[100] - mean(frame[100])

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 5 / 33

slide-6

SLIDE 6

Line Sensing Problem Setup

How can we find the index of the line?

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 6 / 33

slide-7

SLIDE 7

Line Sensing Argmax Based Detection

Take the argmax!

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 7 / 33

slide-8

SLIDE 8

Line Sensing Argmax Based Detection

Any potential issues?

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 8 / 33

slide-9

SLIDE 9

Line Sensing Argmax Based Detection

What about- noise, track crossings, other bright objects, etc.?

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 9 / 33

slide-10

SLIDE 10

Line Sensing Argmax Based Detection

What about- noise, track crossings, other bright objects, multiple peaks, etc.?

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 10 / 33

slide-11

SLIDE 11

Line Sensing Gradients and Smoothing

I(x,t) = Intensity at pixel x What if we look at the derivative dI(x)

dx ?

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 11 / 33

slide-12

SLIDE 12

Line Sensing Gradients and Smoothing

dI(x) dx

≈ I(x+1)−I(x)

1

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 12 / 33

slide-13

SLIDE 13

Line Sensing Gradients and Smoothing

Convolution! ˙ I(x) = I(x + 1) − I(x) = I ∗ [1, −1]

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 13 / 33

slide-14

SLIDE 14

Line Sensing Gradients and Smoothing

Convolution! ˙ I(x) = I(x + 1) − I(x) = I ∗ [1, −1]

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 14 / 33

slide-15

SLIDE 15

Line Sensing Gradients and Smoothing

Convolution! ˙ I(x) = I(x + 1) − I(x) = I ∗ [1, −1]

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 15 / 33

slide-16

SLIDE 16

Line Sensing Gradients and Smoothing

Convolution! What about all that high frequency noise?

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 16 / 33

slide-17

SLIDE 17

Line Sensing Gradients and Smoothing

Low Pass Filter! I(x) ∗ LPF ∗ [1, −1] = I(x) ∗ (LPF ∗ [1, −1])

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 17 / 33

slide-18

SLIDE 18

Line Sensing Gradients and Smoothing

Low Pass Filter (Gaussian)! I(x) ∗ LPF ∗ [1, −1] = I(x) ∗ (LPF ∗ [1, −1])

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 18 / 33

slide-19

SLIDE 19

Line Sensing Gradients and Smoothing

Low Pass Filter (Gaussian)! High frequency noise gone!

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 19 / 33

slide-20

SLIDE 20

Line Sensing Temporal Differencing

What about... Temporal Differencing?

dI(x) dt

≈ I(t)−I(t−1)

∆t

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 20 / 33

slide-21

SLIDE 21

Line Sensing Temporal Differencing

What about... Temporal Differencing? Good for removing constant bias in parts of the image.

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 21 / 33

slide-22

SLIDE 22

Line Sensing Temporal Differencing

More Ideas!

◮ Difference of Gaussians (Bandpass- not just Lowpass) ◮ Cross Correlation with known signal ◮ Convolutional Neural Networks??

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 22 / 33

slide-23

SLIDE 23

Line Sensing Issues

Potential Issues?

◮ What to do if there are multiple track crossings? ◮ Can’t find the line ◮ Other Issues?

Solutions!

◮ Maintain a history (previous line pos, camera scan, etc.) ◮ Ignore the problem(s) and go fast! ◮ Anything else?

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 23 / 33

slide-24

SLIDE 24

Embedded Software/ MCUXpresso

Embedded Software/ MCUXpresso

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 24 / 33

slide-25

SLIDE 25

Embedded Software/ MCUXpresso Modularity

Modularity

◮ Code is starting to get complicated

◮ Periodic Interrupt TImers (PIT), Flexible Timer Module (FTM),

UART, GPIO, ADC etc.

◮ Many tasks to run on a single core CPU

◮ Velocity Measurement ◮ Line Finding ◮ PID Controllers (steering and velocity) ◮ Telemetry ◮ Others?

◮ How do we make all of this work together???

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 25 / 33

slide-26

SLIDE 26

Embedded Software/ MCUXpresso Timing

How to delay?

// Why might this be a bad idea? void delay(void) { volatile uint32_t i = 0U; for (i = 0U; i < 80000U; ++i) { __asm("NOP"); /* delay */ } } // Where does get_curr_time_ms () come from ??? void delay_ms(uint32_t t){ uint32_t end_time = t + get_curr_time_ms (); while( get_curr_time_ms () < end_time ){ // Block until the delay is

ver

} } Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 26 / 33

slide-27

SLIDE 27

Embedded Software/ MCUXpresso Timing

Use a Timer!! (PIT or FTM)

// Where does get_curr_time_ms () come from ??? void delay_ms(uint32_t t){ uint32_t end_time = t + get_curr_time_ms (); while( get_curr_time_ms () < end_time ){ // Block until the delay is

ver

} } uint32_t get_curr_time_ms (){ return systime; } // Initialize this interrupt somewhere to run at 1Khz // Then systime is in units

f ms

void PIT0_IRQHandler (void) { /* Clear interrupt flag.*/ systime ++; /* hopefully atomic

peration

*/ PIT_ClearStatusFlags (PIT , kPIT_Chnl_0 , kPIT_TimerFlag ); pitIsrFlag0 = true; } Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 27 / 33

slide-28

SLIDE 28

Embedded Software/ MCUXpresso Timing

What about FTM’s???

/* Initialize the Flexible Timer Module to Produce PWM with init_duty_cycle at freq_hz */ void init_pwm(uint32_t freq_hz , uint8_t init_duty_cycle ) { duty_cycle = init_duty_cycle ; ftm_config_t ftmInfo; ftm_chnl_pwm_signal_param_t ftmParam; /* Configure ftm params with for pwm freq - freq_hz , duty cycle - init_duty_cycle */

ftmParam. chnlNumber = BOARD_FTM_CHANNEL ;

ftmParam.level = PWM_LEVEL;

ftmParam. dutyCyclePercent = init_duty_cycle ;
ftmParam. firstEdgeDelayPercent = 0U;

FTM_GetDefaultConfig (& ftmInfo ); /* Initialize FTM module */ FTM_Init(BOARD_FTM_BASEADDR , &ftmInfo ); FTM_SetupPwm (BOARD_FTM_BASEADDR , &ftmParam , 1U, kFTM_CenterAlignedPwm , freq_hz , FTM_SOURC /* Enable channel interrupt flag.*/ FTM_EnableInterrupts (BOARD_FTM_BASEADDR , FTM_CHANNEL_INTERRUPT_ENABLE ); EnableIRQ( FTM_INTERRUPT_NUMBER ); FTM_StartTimer (BOARD_FTM_BASEADDR , kFTM_SystemClock ); } Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 28 / 33

slide-29

SLIDE 29

Embedded Software/ MCUXpresso Timing

Can Get Rid of FTM Interrupts!!!

/* Initialize the Flexible Timer Module to Produce PWM with init_duty_cycle at freq_hz */ void init_pwm(uint32_t freq_hz , uint8_t init_duty_cycle ) { duty_cycle = init_duty_cycle ; ftm_config_t ftmInfo; ftm_chnl_pwm_signal_param_t ftmParam; /* Configure ftm params with for pwm freq - freq_hz , duty cycle - init_duty_cycle */

ftmParam. chnlNumber = BOARD_FTM_CHANNEL ;

ftmParam.level = PWM_LEVEL;

ftmParam. dutyCyclePercent = init_duty_cycle ;
ftmParam. firstEdgeDelayPercent = 0U;

FTM_GetDefaultConfig (& ftmInfo ); /* Initialize FTM module */ FTM_Init(BOARD_FTM_BASEADDR , &ftmInfo ); FTM_SetupPwm (BOARD_FTM_BASEADDR , &ftmParam , 1U, kFTM_CenterAlignedPwm , freq_hz , FTM_SOURC FTM_StartTimer (BOARD_FTM_BASEADDR , kFTM_SystemClock ); } Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 29 / 33

slide-30

SLIDE 30

Embedded Software/ MCUXpresso Timing

Can Get Rid of FTM Interrupts (also here)!!

void update_duty_cycle (uint8_t updated_duty_cycle ) { FTM_DisableInterrupts (BOARD_FTM_BASEADDR , FTM_CHANNEL_INTERRUPT_ENABLE ); /* Disable channel

utput

before updating the dutycycle */ FTM_UpdateChnlEdgeLevelSelect (BOARD_FTM_BASEADDR , BOARD_FTM_CHANNEL , 0U); /* Update PWM duty cycle */ FTM_UpdatePwmDutycycle (BOARD_FTM_BASEADDR , BOARD_FTM_CHANNEL , kFTM_CenterAlignedPwm , upda /* Software trigger to update registers */ FTM_SetSoftwareTrigger (BOARD_FTM_BASEADDR , true ); /* Start channel

utput

with updated dutycycle */ FTM_UpdateChnlEdgeLevelSelect (BOARD_FTM_BASEADDR , BOARD_FTM_CHANNEL , PWM_LEVEL ); /* Delay to view the updated PWM dutycycle */ delay (); // Can be removed when using PWM for realtime applications /* Enable interrupt flag to update PWM dutycycle */ FTM_EnableInterrupts (BOARD_FTM_BASEADDR , FTM_CHANNEL_INTERRUPT_ENABLE ); } Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 30 / 33

slide-31

SLIDE 31

Embedded Software/ MCUXpresso Timing

Can Get Rid of FTM Interrupts (also here)!!

void update_duty_cycle (uint8_t updated_duty_cycle ) { /* Disable channel

utput

before updating the dutycycle */ FTM_UpdateChnlEdgeLevelSelect (BOARD_FTM_BASEADDR , BOARD_FTM_CHANNEL , 0U); /* Update PWM duty cycle */ FTM_UpdatePwmDutycycle (BOARD_FTM_BASEADDR , BOARD_FTM_CHANNEL , kFTM_CenterAlignedPwm , upda /* Software trigger to update registers */ FTM_SetSoftwareTrigger (BOARD_FTM_BASEADDR , true ); /* Start channel

utput

with updated dutycycle */ FTM_UpdateChnlEdgeLevelSelect (BOARD_FTM_BASEADDR , BOARD_FTM_CHANNEL , PWM_LEVEL ); }

Don’t forget to remove the FTM interrupt handler!!

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 31 / 33

slide-32

SLIDE 32

Embedded Software/ MCUXpresso Timing

General Tips

◮ Remove unecessary interrupts (FTM’s doing PWM for example). This

will make debugging easier later on

◮ Build modular code, test components in isolation ◮ PRINTF can mess up your timing (especially the floating point

version) (see sprintf)

◮ Minimize dynamic memory allocation aka malloc & free (memory

fragmentation)

◮ Use the debugger tool!! ◮ See skeleton code (frdmk64f skeleton) ◮ Keep things simple!

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 32 / 33

slide-33

SLIDE 33

Summary

Summary

◮ Many ways to do line detection (max, differencing, gaussian

smoothing, temporal differencing, etc.)

◮ Embedded Software/ MCUX tips

Ducky (UCB EECS) Mechatronics Design Lab 7 & 8 Mar 2018 (Week 8) 33 / 33