Position Detection For a Camera Pen Using LLAH and Dot Patterns - - PowerPoint PPT Presentation
Position Detection For a Camera Pen Using LLAH and Dot Patterns Matthias Sperber German Research Center for Artificial Intelligence Osaka Prefecture University Outline n Introduction n Locally Likely Arrangement Hashing n The Dot
Matthias Sperber German Research Center for Artificial Intelligence Osaka Prefecture University
n Introduction n Locally Likely Arrangement Hashing n The Dot Pattern n Overview: Retrieval Steps n Dot Extraction from the Camera Image n Position Calculation n Examples & Results n Outlook & Conclusion
n Paper documents
n Higher readability n Handwriting
applicable
n Digital documents
n Flexible organizing n Convenient editing n Searchable
notes from class notes from class
n Example Use Case OCR handwriting recognition Full-text search for “class”
n Capture Pen
Position
n Reconstruct
Handwriting
n Distinguish documents n Inexpensive n Portable n Require no special
paper
+ Write using stylus or
+ Fairly inexpensive (no running costs)
not distinguished
+ Works on any writing surface + Inexpensive
calibrated
distinguished
+ Can distinguish documents + Highly portable
be bought
running costs)
apparently visible
n Ordinary pen
equipped with cheap, low- resolution camera
n Yellow Dot
Pattern
n Print on
Print yellow dots on paper
Print document foreground (or leave empty)
Write using camera pen Reconstruct handwriting
Pen tablet Ultra- sonic Anoto Proposed pen Low Cost ü üü û ü Normal paper (ü) ü û ü Portable û (û) ü ü Distinguish documents û û ü ü
n Introduction n Locally Likely Arrangement Hashing n The Dot Pattern n Overview: Retrieval Steps n Dot Extraction from the Camera Image n Position Calculation n Examples & Results n Outlook & Conclusion
Locally Likely Arrangement Hashing
n Retrieve document
images
n Feature points from
document foreground
n Retrieval by
matching individual points
n Determine position
Captured camera image
Locally Likely Arrangement Hashing
Calculation of Indices LLAH Database Feature points for every document Feature points from partial document image # documents & dots affect accuracy! Storage Retrieval
n Local arrangements n Geometric invariant:
Area Ratio
P(A,C,D) P(A,B,C)
A B C D
LLAH With Dot Patterns
No feature points!
è Print yellow dots in background
n Introduction n Locally Likely Arrangement Hashing n The Dot Pattern n Overview: Retrieval Steps n Dot Extraction from the Camera Image n Position Calculation n Examples & Results n Outlook & Conclusion
n Yellow
n Almost invisible to
human eye
n Can still be extracted
by computer
n Randomized n Start: Regular Grid n Gauss distribution n Bounding box
n Avoid “holes”
n Dot spacing: 2.7mm n Diameter: 0.2mm n (Anoto’s dot spacing: 0.3mm)
n Introduction n Locally Likely Arrangement Hashing n The Dot Pattern n Overview: Retrieval Steps n Dot Extraction from the Camera Image n Position Calculation n Examples & Results n Outlook & Conclusion
LLAH (Doc ID, Coordinates) Calculate pen position Feature points Camera image
n Introduction n Locally Likely Arrangement Hashing n The Dot Pattern n Overview: Retrieval Steps n Dot Extraction from the Camera Image n Position Calculation n Examples & Results n Outlook & Conclusion
LLAH (Doc ID, Coordinates) Calculate pen position Feature points Camera image
n Distance-Image è Adaptive Thresholding
n “Distance Image”: For each pixel, determine
how close its color is to the color yellow
n Distance-Image è Adaptive Thresholding
n Distance-Image è Adaptive Thresholding
n Distance-Image è Adaptive Thresholding
n Introduction n Locally Likely Arrangement Hashing n The Dot Pattern n Overview: Retrieval Steps n Dot Extraction from the Camera Image n Position Calculation n Examples & Results n Outlook & Conclusion
LLAH (Doc ID, Coordinates) Calculate pen position Feature points Camera image
LLAH (Doc ID, Coordinates) Calculate pen position Feature points Camera image
n Estimate geometric transformation n Reconstruct handwriting by drawing lines
between consecutively determined positions
n Introduction n Locally Likely Arrangement Hashing n The Dot Pattern n Overview: Retrieval Steps n Dot Extraction from the Camera Image n Position Calculation n Examples & Results n Outlook & Conclusion
Little foreground Much foreground No foreground
100 100.0% 94.0% 74.4% 1000 96.2% 80.4% 59.3%
DB size Retrieval accuracy (% of correctly determined positions) Bounding box
n Smaller document DB (100 docs)
n Larger document DB (1000 docs)
n Performance Measurements
n Intel Core CPU @ 2.13GHz, 3GB RAM
DB size 1 document 18.5ms 23.8ms 100 documents 22.0ms 30.6ms 1,000 documents 49.4ms 53.5ms
3.1× 2.3cm2 2.2 ×1.6cm2
Area of captured image
n Introduction n Locally Likely Arrangement Hashing n The Dot Pattern n Overview: Retrieval Steps n Dot Extraction from the Camera Image n Position Calculation n Examples & Results n Outlook & Conclusion
n Problem: yellow dots
hidden by too much document foreground
(using large Document DB)
n Problem: yellow dots
hidden by too much document foreground
n Solution: Use feature points from both
background (yellow dots) and foreground (characters)
n Combination of techniques:
n Proposed method to establish absolute
position and current document
n Tracking method to measure relative
movement and reconstruct handwriting
n Low-cost camera pen
n Used cheap USB camera n Yellow dots printable on ordinary hardware
n Support of 1,000+ documents n Reasonably fast retrieval speed n Future work:
n Make more stable when too much document
foreground: Incorporate features from foreground!