Real-Time Note Digitizer! ECE532 Team A \ / \ / 350 / \ / \ 3 ! - - PowerPoint PPT Presentation

▶
real time note digitizer
SMART_READER_LITE
LIVE PREVIEW

Real-Time Note Digitizer! ECE532 Team A \ / \ / 350 / \ / \ 3 ! - - PowerPoint PPT Presentation

Feb 08, 2023 675 likes •1.01k views

Overview Implementation Segmentation History Frame Buffering Results DEMO Real-Time Note Digitizer! ECE532 Team A \ / \ / 350 / \ / \ 3 ! Sean Bell, Sanae Rosen, Konstantine Tsotsos University of Toronto, Engineering Science

slide-1
SLIDE 1

Overview Implementation Segmentation History Frame Buffering Results DEMO

Real-Time Note Digitizer!

– ECE532 Team A\/\/350/\/\3!– Sean Bell, Sanae Rosen, Konstantine Tsotsos

University of Toronto, Engineering Science

slide-2
SLIDE 2

Overview Implementation Segmentation History Frame Buffering Results DEMO

We’ve all been in class...

Projectors and tablets used for delivering notes Hard to record content, lots of wasted data or no effective way of replaying!

slide-3
SLIDE 3

Overview Implementation Segmentation History Frame Buffering Results DEMO

Goal

Easy to record notes Efficient storage Without loss of written content

slide-4
SLIDE 4

Overview Implementation Segmentation History Frame Buffering Results DEMO

Goal

Easy to record notes Efficient storage Without loss of written content

slide-5
SLIDE 5

Overview Implementation Segmentation History Frame Buffering Results DEMO

Goal

Easy to record notes Efficient storage Without loss of written content

slide-6
SLIDE 6

Overview Implementation Segmentation History Frame Buffering Results DEMO

Initial system for real-time performance

High-performance, real-time video processing Entirely in hardware

MPMC DDR Memory VideoToRam WriteHistory WriteFrame Video Out Microprocessor Port A Port B Port C Port D Port E pixel histories frame for display Bus Master TFT Controller 8-bit Buf V0 8-bit Buf V1 Decode & Threshold Relative DDR Mem Addr 1-bit Thresh Delay Delayed Addr Delayed Addr New 32-bit History Delay 32-bit Buf T0 32-bit Buf T1 32-bit History Bus Master Write Enable Logic Bus Master VGA Bus Master YCbCr

slide-7
SLIDE 7

Overview Implementation Segmentation History Frame Buffering Results DEMO

Initial system for real-time performance

High-performance, real-time video processing Entirely in hardware

MPMC DDR Memory VideoToRam WriteHistory WriteFrame Video Out Microprocessor Port A Port B Port C Port D Port E pixel histories frame for display Bus Master TFT Controller 8-bit Buf V0 8-bit Buf V1 Decode & Threshold Relative DDR Mem Addr 1-bit Thresh Delay Delayed Addr Delayed Addr New 32-bit History Delay 32-bit Buf T0 32-bit Buf T1 32-bit History Bus Master Write Enable Logic Bus Master VGA Bus Master YCbCr

slide-8
SLIDE 8

Overview Implementation Segmentation History Frame Buffering Results DEMO

Revised System

!"#"$%& '()& '*%"+,+-./& !*$"&012"3& !*$"&012"3& 4"/.$56& 4"#/"$7.5+$& 8"6+%*$#& !+#*6& 9:;:3& 01<&=.<7"3& ,>,& :+$73+??"3& =@=:&

  • 4ABA&

C0& D)-,& :1<7+/&E@& FG*<5$#&E@& FG7H&@+37& @!0& I3*5$#& '"3*J6.5+$& K17L17& ("$"3.5+$& D<"3&E$7"3M.6"& =.$.#"/"$7&

slide-9
SLIDE 9

Overview Implementation Segmentation History Frame Buffering Results DEMO

Revised System

!"# $%"&'(("%# )'*+,-&# )'.*',/+-",# 0121%# 3'&"4'%# 5','%+/'# 67/87/# 9,+:".# $;<':# =;(/"%># ?59#

slide-10
SLIDE 10

Overview Implementation Segmentation History Frame Buffering Results DEMO

Qualitative Approach

Given an input image, would like to classify each pixel: Ink Not Ink

slide-11
SLIDE 11

Overview Implementation Segmentation History Frame Buffering Results DEMO

Decision Tree Classifier

1

Paper and ink alone is easy

2

YCbCr is decorelated

3

Human skin profile Build decision tree to classify!

Cr > 140? Yes not ink No Y > 115? Yes not ink No ink

slide-12
SLIDE 12

Overview Implementation Segmentation History Frame Buffering Results DEMO

Decision Tree Classifier

1

Paper and ink alone is easy

2

YCbCr is decorelated

3

Human skin profile Build decision tree to classify!

Cr > 140? Yes not ink No Y > 115? Yes not ink No ink

slide-13
SLIDE 13

Overview Implementation Segmentation History Frame Buffering Results DEMO

Calibration

Base Image Cr segmentation

slide-14
SLIDE 14

Overview Implementation Segmentation History Frame Buffering Results DEMO

Calibration

Base Image Y Segmentation

slide-15
SLIDE 15

Overview Implementation Segmentation History Frame Buffering Results DEMO

Pixel History

Classification: History Buffer Output: Time … … … … … … … … … …

slide-16
SLIDE 16

Overview Implementation Segmentation History Frame Buffering Results DEMO

Pixel History

Classification: History Buffer Output: Time … … … … … … … … … …

slide-17
SLIDE 17

Overview Implementation Segmentation History Frame Buffering Results DEMO

Pixel History

Classification: History Buffer Output: Time … … … … … … … … … …

slide-18
SLIDE 18

Overview Implementation Segmentation History Frame Buffering Results DEMO

Pixel History

Classification: History Buffer Output: Time … … … … … … … … … …

slide-19
SLIDE 19

Overview Implementation Segmentation History Frame Buffering Results DEMO

Pixel History

Classification: History Buffer Output: Time … … … … … … … … … …

slide-20
SLIDE 20

Overview Implementation Segmentation History Frame Buffering Results DEMO

Pixel History

Classification: History Buffer Output: Time … … … … … … … … … …

slide-21
SLIDE 21

Overview Implementation Segmentation History Frame Buffering Results DEMO

Pixel History

Classification: History Buffer Output: Time … … … … … … … … … …

slide-22
SLIDE 22

Overview Implementation Segmentation History Frame Buffering Results DEMO

Pixel History

Classification: History Buffer Output: Time … … … … … … … … … …

slide-23
SLIDE 23

Overview Implementation Segmentation History Frame Buffering Results DEMO

Pixel History

Classification: History Buffer Output: Time … … … … … … … … … …

slide-24
SLIDE 24

Overview Implementation Segmentation History Frame Buffering Results DEMO

Pixel History

Classification: History Buffer Output: Time … … … … … … … … … …

slide-25
SLIDE 25

Overview Implementation Segmentation History Frame Buffering Results DEMO

Double Buffering

!"#$%& '$()*&+& '$()*&,&

  • $%#$%&
slide-26
SLIDE 26

Overview Implementation Segmentation History Frame Buffering Results DEMO

Circular Buffering

!"#$%& '$()*&+& '$()*&,&

  • $%#$%&

'$()*&./0"%)*& '$()*&1& '$()*&2&

slide-27
SLIDE 27

Overview Implementation Segmentation History Frame Buffering Results DEMO

Successes

Near real-time recording of handwriting Rejection of user’s hand and writing utensil from image stream Rich writing interface: erasing, multiple colours Playback of previous writing

slide-28
SLIDE 28

Overview Implementation Segmentation History Frame Buffering Results DEMO

Future Improvements

Recording Save to PC

slide-29
SLIDE 29

Overview Implementation Segmentation History Frame Buffering Results DEMO

Future Improvements

Recording Save to PC

slide-30
SLIDE 30

Overview Implementation Segmentation History Frame Buffering Results DEMO

Design Philosophy

Our philosophy:

1

No integration stage: rotate tasks

2

Explore alternatives concurrently and choose best

3

Adapt design quickly

slide-31
SLIDE 31

Overview Implementation Segmentation History Frame Buffering Results DEMO

Lessons Learned

Design complexity and tough performance criteria don’t always lead to most effective system Solve as much as possible in software

slide-32
SLIDE 32

Overview Implementation Segmentation History Frame Buffering Results DEMO

DEMO