Applications of Subword Spotting Brian Davis A common scenario... - - PowerPoint PPT Presentation

▶
applications of subword spotting
SMART_READER_LITE
LIVE PREVIEW

Applications of Subword Spotting Brian Davis A common scenario... - - PowerPoint PPT Presentation

Mar 11, 2024 310 likes •543 views

Applications of Subword Spotting Brian Davis A common scenario... A common scenario... A common scenario... A common scenario... Wouldnt it be nice if something could scan automatically for you? Were going to do this and other things

slide-1
SLIDE 1

Applications of Subword Spotting

Brian Davis

slide-2
SLIDE 2

A common scenario...

slide-3
SLIDE 3

A common scenario...

slide-4
SLIDE 4

A common scenario...

slide-5
SLIDE 5

A common scenario...

Wouldn’t it be nice if something could scan automatically for you?

slide-6
SLIDE 6

We’re going to do this and other things with subword spotting

slide-7
SLIDE 7

Outline

  • Review word spotting
  • Subword spotting
  • Our implementation
  • Performance
  • Applications
  • Suffix spotting
  • Transcription assistant demo
slide-8
SLIDE 8

Word Spotting

  • Goal is to search corpus of images directly
  • Query-by-string (QbS): search with text
  • Query-by-example (QbE): search with an example word image

Search for “pay” Search for “payment”

slide-9
SLIDE 9

Subword Spotting

  • We now allow spottings within words

Search for “pa” Search for “pay”

slide-10
SLIDE 10

Subword Spotting Implementation

  • Converted Sudholt et al’s word spotting

method PHOCNet to perform sliding window

  • Changed PHOC used and comparison

method for better QbS results

  • Less resolution for PHOC
  • Similarity based on cross-entropy instead cosine

distance

  • Original PHOC and cosine similarity better for QbE
  • Found optimal window width for each

subword of interest

PHOC (descriptive vector) CNN

slide-11
SLIDE 11

Datasets

Bentham US 1930 Census Names

slide-12
SLIDE 12

Subord Spotting Results

Unigrams Bigrams Trigrams Unigrams Bigrams Trigrams QbS 67.7% 68.2% 70.5% 49.7% 40.2% 36.3% QbE 51.1% 56.9% 57.1% 34.0% 29.5% 28.5%

Bentham US 1930 Census Names

  • Unigrams: all letters of alphabet
  • Bigrams: 100 most frequent in English
  • Trigrams: 300 most frequent in English

Reported as Mean Average Precision

slide-13
SLIDE 13

Bentham

slide-14
SLIDE 14

A searching task

What if I wanted to find all the towns in a set of German documents?

  • What about automatically finding all words

ending in “-burg”?

slide-15
SLIDE 15

Suffix Spotting

  • Find all words with a given suffix
  • Constraint on original subword spotting problem
  • Could be extended to handle regular-expression-like queries
slide-16
SLIDE 16

Suffix Spotting

  • IAM results
slide-17
SLIDE 17

Suffix Spotting

  • Census Names

results

slide-18
SLIDE 18

Transcription Assistant Demo

  • Using ground truth word segmentations
  • Both embedding and PHOCs for windows are precomputed.
  • Selection snapped to closest window
  • ~10 second delay to compute PHOC for all windows of a single size, depending on size,

using GPU

slide-19
SLIDE 19

Thank you

Questions?

slide-20
SLIDE 20

Transcription Assistant Demo

(Images in case of technical difficulties)

slide-21
SLIDE 21

Transcription Assistant Demo

(Images in case of technical difficulties)

slide-22
SLIDE 22

Transcription Assistant Demo

(Images in case of technical difficulties)

slide-23
SLIDE 23

Subord Spotting Implementation

  • Network architecture