Spoken and Sign Languages Spoken and Sign Languages A Cross Modal - - PowerPoint PPT Presentation

▶
spoken and sign languages spoken and sign languages a
SMART_READER_LITE
LIVE PREVIEW

Spoken and Sign Languages Spoken and Sign Languages A Cross Modal - - PowerPoint PPT Presentation

Nov 26, 2023 260 likes •495 views

Spoken and Sign Languages Spoken and Sign Languages A Cross Modal Study Purushottam Kar Achla M. Raina Amitabha Mukerjee Indian Institute of Technology Kanpur 28th All India Conference of Linguists, Banaras Hindu University, 2006 1 Overview

slide-1
SLIDE 1

Spoken and Sign Languages Spoken and Sign Languages A Cross Modal Study

28th All India Conference of Linguists, Banaras Hindu University, 2006 1

Purushottam Kar Achla M. Raina Amitabha Mukerjee Indian Institute of Technology Kanpur

slide-2
SLIDE 2

Overview

  • Introduction - Research in SLs
  • Framework Adopted
  • The Indian Sign Language
  • Cross-modal Systems - Challenges
  • INGIT – Issues and Solutions

28th All India Conference of Linguists, Banaras Hindu University, 2006 2

  • INGIT – Issues and Solutions
  • Summing Up
  • Future Directions
slide-3
SLIDE 3

Introduction

  • Language – a multimodal construct
  • Evolution of Sign Languages
  • Origin – ‘Deaf’ communities
  • Evolution – linguistic aspects
  • Sociolinguistic Aspects - ISL
  • Geographical Variation

28th All India Conference of Linguists, Banaras Hindu University, 2006 3

  • Geographical Variation
  • Status as a Minority Language
  • ISL in Education [Deshmukh96]
slide-4
SLIDE 4

Introduction

  • Past Research in SLs
  • Neuro-physiological
  • [Damasio86], [Gordon04]
  • Multi-modal [Petitto04]
  • Descriptive
  • [Stokoe60], [Sexton99]

28th All India Conference of Linguists, Banaras Hindu University, 2006 4

  • Cross-modal
  • [Wray et al 04], [Zhao et al 00], [Speers02]
  • The case of ISL
  • [Vasishta et al 86], [Zeshan00]
slide-5
SLIDE 5

The Present Project

  • Issues in cross-modality
  • Representations
  • Mappings
  • Part of a larger endeavour
  • A description of ISL

28th All India Conference of Linguists, Banaras Hindu University, 2006 5

  • A description of ISL
  • A cross-modal translation system
slide-6
SLIDE 6

Framework Adopted

  • Construction Grammars [Kay, 2002]
  • The fly is buzzing.
  • What is the fly doing in my soup?
  • . . . fly in the ointment. . .
  • Constructions
  • form – meaning maps at various levels

28th All India Conference of Linguists, Banaras Hindu University, 2006 6

  • form – meaning maps at various levels
  • morphological, lexical, syntactic
  • Unification based approach
slide-7
SLIDE 7

Framework Adopted

  • Fluid Construction Grammar (FCG)
  • Computational model of CG [De Beule and

Steels, 2006]

  • Paired syntactic and Semantic

Structures

  • Bidirectional rules, unification based

approach

28th All India Conference of Linguists, Banaras Hindu University, 2006 7

approach

  • Why Construction Grammars?
slide-8
SLIDE 8

The Indian Sign Language

  • Spatial Modality
  • Extensive use of space
  • Iconic signs
  • Role play
  • Use of person and space deixis
  • Directional verbs

28th All India Conference of Linguists, Banaras Hindu University, 2006 8

  • Directional verbs
  • Non-manual markers
slide-9
SLIDE 9

Ram gave Sita a cat

28th All India Conference of Linguists, Banaras Hindu University, 2006 9

slide-10
SLIDE 10

Does Ram love Sita ?

28th All India Conference of Linguists, Banaras Hindu University, 2006 10

slide-11
SLIDE 11

Cross-Modal Issues

  • Levels of Mapping
  • Constituent Level
  • Complete
  • गाड सात बजे जाएगी

गाड सात बजे जाएगी गाड सात बजे जाएगी गाड सात बजे जाएगी

  • {train time seven go}
  • Partial
  • Constituent Deletion
  • राजधानी रात म चलती है

राजधानी रात म चलती है राजधानी रात म चलती है राजधानी रात म चलती है

  • {rAjdhAni night go}

28th All India Conference of Linguists, Banaras Hindu University, 2006 11

  • राजधानी रात म चलती है

राजधानी रात म चलती है राजधानी रात म चलती है राजधानी रात म चलती है

  • {rAjdhAni night go}
  • Constituent Insertion
  • आप दस पये दजए

आप दस पये दजए आप दस पये दजए आप दस पये दजए

  • {money ten give {you me}}
slide-12
SLIDE 12

Cross-Modal Issues

  • Levels of Mapping
  • Levels of Mapping
  • Construction Level
  • Compositional
  • टकट नहं िमलेगा !य"क वेटंग है

टकट नहं िमलेगा !य"क वेटंग है टकट नहं िमलेगा !य"क वेटंग है टकट नहं िमलेगा !य"क वेटंग है

  • @neg {ticket get

neg} {Q-why} {waiting-list}

  • Non-Compositional
  • X म

म म म Y वेटंग है वेटंग है वेटंग है वेटंग है

  • {x waiting-list y}

28th All India Conference of Linguists, Banaras Hindu University, 2006 12

  • X म

म म म Y वेटंग है वेटंग है वेटंग है वेटंग है

  • {x waiting-list y}
slide-13
SLIDE 13

Cross-Modal Issues

  • Visuo-spatial nature of ISL
  • Yes-no Question vs. corresponding

affirmative

  • Spatial Morphology – aspect, classifier
  • Polysemous Expressions
  • attributive and existential senses of ‘है

है है है’

28th All India Conference of Linguists, Banaras Hindu University, 2006 13

है है है है

  • alienable vs. inalienable possession as in

मेर कताब मेर कताब मेर कताब मेर कताब and मेरा भाई मेरा भाई मेरा भाई मेरा भाई

  • transactional and non-transactional senses
  • f the verb ‘ले

ले ले ले’

slide-14
SLIDE 14

Cross-Modal Issues

  • Elided Expressions
  • Elided Expressions
  • दस पये दजए

दस पये दजए दस पये दजए दस पये दजए

  • {ten rupees give {you

me}}

  • Event semantics in ISL
  • Anaphoric expressions
  • May be resolved to saturate event

28th All India Conference of Linguists, Banaras Hindu University, 2006 14

  • May be resolved to saturate event

semantics

  • May be replaced with deictic signs
slide-15
SLIDE 15

INGIT – Issues and Solutions

  • Elided Expressions
  • Semantically mediated route
  • HPSG, CG …
  • Mapping Levels
  • Morphological, lexical, sentential
  • Hybrid formulaic expressions

28th All India Conference of Linguists, Banaras Hindu University, 2006 15

  • Hybrid formulaic expressions
  • … Construction Grammars
slide-16
SLIDE 16

INGIT – Issues and Solutions

  • Ellipsis and Anaphora
  • Ellipsis resolution module
  • Participants in transactional events
  • Subject in monadic/dyadic events
  • Some instances passed on to ISL using

unit constructions

28th All India Conference of Linguists, Banaras Hindu University, 2006 16

unit constructions

  • Polysemous expressions: unit

constructions

slide-17
SLIDE 17

INGIT – Issues and Solutions

  • Mapping Level problem
  • Complete constituent mapping: word-

constituent mappings

  • Partial constituent mapping
  • Deletion: word-constituent mapping
  • Insertion: indeterminate process, unit

28th All India Conference of Linguists, Banaras Hindu University, 2006 17

  • Insertion: indeterminate process, unit

constructions used

  • Constructional Mapping: unit and

compositional constructions

slide-18
SLIDE 18

INGIT – Issues and Solutions

  • Visuo-Spatial Nature of ISL
  • Output formatted in a manner so as to

enable the rendering module to bring

  • ut directionality
  • Output String tagged for visual markers

and deictics

28th All India Conference of Linguists, Banaras Hindu University, 2006 18

slide-19
SLIDE 19

Summing Up

  • Semantically mediated procedure
  • CG adapted to specific objectives
  • Working implementation developed *

28th All India Conference of Linguists, Banaras Hindu University, 2006 19

* Purushottam Kar, Madhusudan Reddy, Amitabha Mukerjee and

Achla M Raina, “INGIT: Limited Domain Formulaic Translation from Hindi to Indian Sign Language”, Proceedings of ICON-07, Hyderabad (To appear).

slide-20
SLIDE 20

Future Directions

  • Describe ISL in terms of a framework

allowing parallel processing

  • Define such a framework and develop

formalisms for the same

  • Further explore representational and

mapping issues

28th All India Conference of Linguists, Banaras Hindu University, 2006 20

mapping issues

  • Develop robust graphical front end
  • Add support to take speech as an

input

slide-21
SLIDE 21

References

  • Damasio, A. et al, “Sign language aphasia during left-

hemisphere amytal injection”, Nature, 322: 363–5, 1986.

  • Deshmukh, D., “Sign Language and Bilingualism in Deaf
  • Deshmukh, D., “Sign Language and Bilingualism in Deaf

Education”, India: Deaf Foundation, Ichalkaranji, 1996.

  • Gordon, N. “The neurology of sign language", Brain and

Development, 26: 146-150, 2004.

  • Kay, P. “An informal sketch of a formal architecture for

construction grammar”, Grammars, 5: 1–19, 2002.

  • Petitto, L. A. et al, “Baby hands that move to the rhythm of

language: hearing babies acquiring sign languages babble

28th All India Conference of Linguists, Banaras Hindu University, 2006 21

language: hearing babies acquiring sign languages babble silently on the hands”, Cognition, 93: 43-73, 2004.

  • Sexton, A. L., “Grammaticalization in American Sign

Language”, Language Sciences, 21: 105-141, 1999.

slide-22
SLIDE 22

References

  • Speers d’Armond, “Representation of American Sign

Language for Machine Translation”, Ph.D. Dissertation. Department of Linguistics, Georgetown University, 2002. Department of Linguistics, Georgetown University, 2002.

  • Stokoe, W. C., “Sign Language structure: an outline of the

visual communication systems of the American Deaf”, Studies in Linguistics: Occasional Papers, 8, University of Buffalo, Buffalo, 1960.

  • Vasishta, M. et al, An Introduction to Indian Sign Language,

All India Federation of the Deaf (Third Edition), 1998.

  • Wray, A. et al, “A formulaic approach to translation at the

28th All India Conference of Linguists, Banaras Hindu University, 2006 22

  • Wray, A. et al, “A formulaic approach to translation at the

post

  • ffice:

reading the signs", Language and Communication, 24: 59-75, 2004.

  • Zeshan, U. Sign Language in Indopakistan: A Description of

a Signed Language, Amsterdam: John Benjamins, 2000.