Outline 1. Introduction 1. Motivation 2. Portuguese Sign Language - - PowerPoint PPT Presentation

▶
outline
SMART_READER_LITE
LIVE PREVIEW

Outline 1. Introduction 1. Motivation 2. Portuguese Sign Language - - PowerPoint PPT Presentation

Feb 03, 2024 231 likes •648 views

Outline 1. Introduction 1. Motivation 2. Portuguese Sign Language 3. Goals 2. Architecture 1. Gloss Translate 2. Lookup 3. Animate 3. Implementation 4. Evaluation 5. Conclusions 6. Demo Motivation the system would be helpful in improving

slide-1
SLIDE 1
slide-2
SLIDE 2

Outline

  • 1. Introduction 1. Motivation
  • 2. Portuguese Sign Language
  • 3. Goals
  • 2. Architecture 1. Gloss Translate
  • 2. Lookup
  • 3. Animate
  • 3. Implementation
  • 4. Evaluation
  • 5. Conclusions
  • 6. Demo
slide-3
SLIDE 3
slide-4
SLIDE 4

Motivation

the system would be helpful in improving communication and reducing the isolation of the deaf community higher rate of illiteracy and learning problems hard to access content, aggravated by the rise and speed

  • f new technologies
slide-5
SLIDE 5

Sign Languages and LGP

slide-6
SLIDE 6

Knowledge Sources

slide-7
SLIDE 7

Different Sentence Structure

OSV (Object-Subject-Verb) vs. SVO (Subject-Verb-Object) strong topicalization

slide-8
SLIDE 8

Different Grammar

classes No distinction between nouns and adjectives Articles? (o, os,.. um, uns) inflexion: gender, number, aspect, time... spatial agreement

slide-9
SLIDE 9

Gesture Subunits

gestures are built from parameters hand configuration hand orientation hand placement hand movement non-manual

slide-10
SLIDE 10

Writing Systems

slide-11
SLIDE 11

Writing Systems

slide-12
SLIDE 12
slide-13
SLIDE 13
slide-14
SLIDE 14

Goals

basic architecture and implementation that others can build upon study existing research and list open problems

slide-15
SLIDE 15

Architecture

slide-16
SLIDE 16

Gloss Translate

Statistical Machine Translation language model bilingual lexicon alignment model Rule-Based Machine Translation lexicon and bilingual dictionary rules

slide-17
SLIDE 17

Gloss Translate

Rule-Based Machine Translation pipeline

slide-18
SLIDE 18

Lookup

slide-19
SLIDE 19

Animate - Rig

Inverse Kinematics Definition of spatial marks Facial Controls and Mouth Shapes

slide-20
SLIDE 20

Animate - Procedural Gestures

slide-21
SLIDE 21

Animate - Avatar variance

slide-22
SLIDE 22

Animate - Procedural Gestures

inter- and intra-gesture blending transitions and timing acquiring data

slide-23
SLIDE 23

Implementation

slide-24
SLIDE 24

Blender

slide-25
SLIDE 25

Blender

Python Addon Interface Rendering

slide-26
SLIDE 26

Translation and Natural Language Processing

Python + NLTK - Natural Language Toolkit

slide-27
SLIDE 27

Lookup

JSON database

slide-28
SLIDE 28

Rigify

slide-29
SLIDE 29

Spatial marks - mesh

slide-30
SLIDE 30

Spatial marks - grid

slide-31
SLIDE 31

Base configurations

57 base configurations

slide-32
SLIDE 32

Action Layering and Blending

slide-33
SLIDE 33

Gesture building and Transitions

slide-34
SLIDE 34

Gesture building and Transitions

Distinction between: regular glosses numerals fingerspelling

slide-35
SLIDE 35

Evaluation

6 deaf persons / interpreters + collecting feedback, observation in informal environment

slide-36
SLIDE 36

Evaluation

usefulness user interface translation quality avatar perceptibility

slide-37
SLIDE 37

Conclusions

definitely possible, definitely a challenge Sign Languages and LGP have particularities that pose additional problems more work from a multidisciplinary team is needed more linguistic studies

slide-38
SLIDE 38

Contributions

  • 1. summary of related work and background
  • 2. overview of pipeline and architecture
  • 3. open implementation as proof of concept
  • 4. list of problems and future work
slide-39
SLIDE 39

Contributions

slide-40
SLIDE 40

Demo