Bachelor Thesis Presentation Implementation of an Android app for - - PowerPoint PPT Presentation

▶
bachelor thesis presentation
SMART_READER_LITE
LIVE PREVIEW

Bachelor Thesis Presentation Implementation of an Android app for - - PowerPoint PPT Presentation

Jan 07, 2023 738 likes •1.12k views

Bachelor Thesis Presentation Implementation of an Android app for the recording of Go games by tracking their state Tilman A DLER June 1, 2015 Pattern Recognition Lab (CS 5) Motivation Detector Pipeline Overview Preprocessing Detection of

slide-1
SLIDE 1

Bachelor Thesis Presentation

Implementation of an Android app for the recording of Go games by tracking their state Tilman ADLER June 1, 2015 Pattern Recognition Lab (CS 5)

slide-2
SLIDE 2

Motivation Detector Pipeline Overview Preprocessing Detection of Lines and Pieces Extrapolating the Board Postprocessing Evaluation Image Set Detection Quality Speed Conclusion Our Contribution Future Work Live Presentation

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 2

slide-3
SLIDE 3

A Really Short Introduction to Go

  • This is a 9x9 Go board (there’s also

11x11, 13x13 and 19x19)

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 3

slide-4
SLIDE 4

A Really Short Introduction to Go

  • This is a 9x9 Go board (there’s also

11x11, 13x13 and 19x19)

  • Players alternately put pieces on the

intersections

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 3

slide-5
SLIDE 5

A Really Short Introduction to Go

  • This is a 9x9 Go board (there’s also

11x11, 13x13 and 19x19)

  • Players alternately put pieces on the

intersections

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 3

slide-6
SLIDE 6

A Really Short Introduction to Go

  • This is a 9x9 Go board (there’s also

11x11, 13x13 and 19x19)

  • Players alternately put pieces on the

intersections

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 3

slide-7
SLIDE 7

A Really Short Introduction to Go

  • This is a 9x9 Go board (there’s also

11x11, 13x13 and 19x19)

  • Players alternately put pieces on the

intersections

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 3

slide-8
SLIDE 8

A Really Short Introduction to Go

  • This is a 9x9 Go board (there’s also

11x11, 13x13 and 19x19)

  • Players alternately put pieces on the

intersections

  • The player with the most area wins

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 3

slide-9
SLIDE 9

Reasons to Record Go Games

The famous Ear-reddening Game of 1846

  • This is the game after the 25th

move,...

Image CC-BY-SA 3.0 by Wikipedia http://en.wikipedia.org/w/index.php? title=Ear-reddening_game&oldid=640966258

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 4

slide-10
SLIDE 10

Reasons to Record Go Games

The famous Ear-reddening Game of 1846

  • This is the game after the 25th

move,...

  • when black made a mistake, that put

white in the lead.

Image CC-BY-SA 3.0 by Wikipedia http://en.wikipedia.org/w/index.php? title=Ear-reddening_game&oldid=640966258

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 4

slide-11
SLIDE 11

Reasons to Record Go Games

The famous Ear-reddening Game of 1846

  • This is the game after the 25th

move,...

  • when black made a mistake, that put

white in the lead.

  • The game went on to move 126...

Image CC-BY-SA 3.0 by Wikipedia http://en.wikipedia.org/w/index.php? title=Ear-reddening_game&oldid=640966258

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 4

slide-12
SLIDE 12

Reasons to Record Go Games

The famous Ear-reddening Game of 1846

  • This is the game after the 25th

move,...

  • when black made a mistake, that put

white in the lead.

  • The game went on to move 126...
  • when this ingenious move turned the

game and let the white player’s ears

  • blush. No one but the players noticed

what a good move it was.

Image CC-BY-SA 3.0 by Wikipedia http://en.wikipedia.org/w/index.php? title=Ear-reddening_game&oldid=640966258

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 4

slide-13
SLIDE 13

Motivation Detector Pipeline Overview Preprocessing Detection of Lines and Pieces Extrapolating the Board Postprocessing Evaluation Image Set Detection Quality Speed Conclusion Our Contribution Future Work Live Presentation

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 5

slide-14
SLIDE 14

Overview

Most of the related work uses only line detection to find the board

⇒ This is problematic for mobile applications

Our approach is to

  • use line detection for finding intersections
  • detect pieces, too, for intersections
  • extrapolate the board from both
  • classify intersections

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 6

slide-15
SLIDE 15

Preprocessing

Segmenting the board from the background

Segmenting the board from the background is useful for

  • removal of noisy backgrounds
  • improving detection speed

We use an adaptive threshold and connected-component analysis for this task

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 7

slide-16
SLIDE 16

Preprocessing

Segmenting the board from the background

Segmenting the board from the background is useful for

  • removal of noisy backgrounds
  • improving detection speed

We use an adaptive threshold and connected-component analysis for this task

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 7

slide-17
SLIDE 17

Preprocessing

Segmenting the board from the background

Segmenting the board from the background is useful for

  • removal of noisy backgrounds
  • improving detection speed

We use an adaptive threshold and connected-component analysis for this task

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 7

slide-18
SLIDE 18

Detection of Lines

using Hough Lines Transformation

This approach is pretty straight forward:

  • detect lines
  • classify as horizontal/vertical
  • intersect each horizontal with

each vertical

  • remove duplicates

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 8

slide-19
SLIDE 19

Detection of Lines

using the Line Segment Detector

Additional steps for LSD detector

  • filter short lines
  • filter lines without parallels
  • stitch lines

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 9

slide-20
SLIDE 20

Detection of Pieces

Thresholding

To segment the tokens from the background we use thresholding White pieces are a problem due to color aberration

⇒ Threshold in HSV space, use

combined results from saturation and hue channel

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 10

slide-21
SLIDE 21

Detection of Pieces

Thresholding

To segment the tokens from the background we use thresholding White pieces are a problem due to color aberration

⇒ Threshold in HSV space, use

combined results from saturation and hue channel

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 10

slide-22
SLIDE 22

Detection of Pieces

Thresholding

To segment the tokens from the background we use thresholding White pieces are a problem due to color aberration

⇒ Threshold in HSV space, use

combined results from saturation and hue channel To detect the pieces we use Hough Transformation or fit rectangles around the blobs and filter for squares

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 10

slide-23
SLIDE 23

Extrapolating the Board

We finally fill the intersections by

  • rotating intersections to be horizontal
  • selecting 20 around the center
  • building a model from them
  • detecting orientation of model with

RANSAC

  • using orientation to extrapolate board
  • using detected intersections to refine

extrapolation

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 11

slide-24
SLIDE 24

Postprocessing

We try to filter out invalid detection results. We consider results invalid if one intersection is

  • outside the image
  • or closer than 5px to each other.

We undo the cropping from the preprocessing step. We smooth results over time: only if a token was detected in 5 of the last 10 frames we count it as accurate.

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 12

slide-25
SLIDE 25

Motivation Detector Pipeline Overview Preprocessing Detection of Lines and Pieces Extrapolating the Board Postprocessing Evaluation Image Set Detection Quality Speed Conclusion Our Contribution Future Work Live Presentation

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 13

slide-26
SLIDE 26

Image Set

  • We took 101 photos with different lighting and backgrounds
  • 70 were used for empirical parameter optimization; 31 for testing

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 14

slide-27
SLIDE 27

Detection Quality

Line detection algorithms

Hough LSD FAST Empty board Sensitivity 100% 30.3% 0.685% Precision 99.4% 94.2% 83.3% Some pieces (7-13) Sensitivity 95.6% 11.4% 1.97% Precision 98.6% 91.2% 53.3% Many pieces (27-34) Sensitivity 58.6% 9.02% 3.80% Precision 95.5% 94.0% 68.8% Sensitivity and precision on our test set. We evaluated correct (≤ 15px

  • ff) intersections.

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 15

slide-28
SLIDE 28

Detection Quality

Line detection algorithms

0.2 0.4 0.6 0.8 1 Tokens on the board Sensitivity & Precision 0.2 0.4 0.6 0.8 1 Tokens on the board 0.2 0.4 0.6 0.8 1 Tokens on the board

Sensitivity (light color filled circles) and precision (dark color filled circles) per image in our test set.

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 16

slide-29
SLIDE 29

Detection Quality

Piece detection algorithms

Contour HOUGH No pieces Sensitivity N/A N/A Precision 1 false positive N/A Some pieces (7-13) Sensitivity 86.7% 75.90% Precision 98.6% 100% Many pieces (27-34) Sensitivity 40.0% 70.6% Precision 99.2% 99.2% Total Sensitivity 71.6% 71.6% Precision 97.8% 99.3% Sensitivity and precision on our test set. We evaluated correct (≤ 15px

  • ff) piece location and roughly correct piece size.

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 17

slide-30
SLIDE 30

Detection Quality

Piece detection algorithms

0.2 0.4 0.6 0.8 1 Tokens on the board Sensitivity & Precision 0.2 0.4 0.6 0.8 1 Tokens on the board

Sensitivity (light color filled circles) and precision (dark color filled circles) per image in our test set.

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 18

slide-31
SLIDE 31

Speed

100 200 300 400 Hough lines LSD lines Hough pieces Contour pieces Preprocessing Extrapolate board Detect colors Framework Create output 64.21 51.33 26.39 83.16 208.04 249.69 322.58 168.46 345.24 Time in ms Average duration of parts of the pipeline over 100 iterations on a Nexus 4

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 19

slide-32
SLIDE 32

Overall Quality

Accuracy of our final detector on our image sets

Trainingset Testset Number of images 70 31 Discarded results 5 2 Accuracy localizing 99.1% (5217/5265) 100% (2349/2349) Accuracy classifying 98.0% (5112/5217) 98.6% (2317/2349) Accuracy for both 97.1% (5112/5265) 98.6% (2317/2349) Similar quality is shown in tests where real games were played and recorded.

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 20

slide-33
SLIDE 33

Our Contribution

  • We present a reliable method

for the detection of Go boards

  • Our method is fast enough to

be used on smartphones

  • We can detect boards with

dense token distribution

  • We showcase a possible

approach for an end user product

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 21

slide-34
SLIDE 34

Future Work

Possible areas of future work could be:

  • extending the approach to a 19x19 grid or grids of arbitrary size
  • autonomously detecting the center of the board
  • improving correction of specular highlights

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 22

slide-35
SLIDE 35

Live presentation

Because seeing is believing

If there is still time...

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 23

slide-36
SLIDE 36

Thank you for your attention. Questions?

June 1, 2015 | Tilman ADLER | Pattern Recognition Lab (CS 5) | BT Presentation 24