Constructing a Novel Blend Through Gesture Robert F. Williams - - PowerPoint PPT Presentation

Constructing a Novel Blend Through Gesture

11th Conference on Conceptual Structure, Discourse, and Language University of British Columbia, 17-20 May 2012 Theme: Language and the Creative Mind

Robert F. Williams

Appleton, Wisconsin

Introduction

Creativity & Conceptual Blending

Creativity

• Mechanism: Combining mental representations

– bisociation of conceptual matrices (Koestler 1964) – conceptual blending (Fauconnier & Turner 1998, 2002) – neural convolution (Thagard & Stewart 2011)

• Criteria: Combinations are creative if they are…

– new, surprising, & valuable (Boden 2004)

A Novel Blend

(Fauconnier & Turner 1998: 146-9; 2002: 270-4)

Real Numbers Geometric Space Complex Numbers Commutative Ring Question How do novel blends emerge in human activity?

Constructing Blends in situ

Conventional blend:

Reading 9:30

Novel blend:

Reading o’clock times by angles

First Example

Conventional Blend for Reading 9:30

Conventional Blend for 9:30: Video Clip

Conventional Blend: Hour Hand

Consequence Can read time from portion of path traveled (halfway to next number)

One Hour (9 to 10)

12 6 9 3 1 2 4 5 7 8 10 11

Emergent property Tip of hour hand is trajector moving along path from source to goal

days hours minutes

... ... ... ... ... ...

9 6 3 12 1 2 4 5 7 8 10 11 ½

1

S G

TR

x x

Conventional Blend: Minute Hand

Consequence Can read time from portion of path traveled (= # of minutes)

Minute Hand Cycle

Emergent property Tip of minute hand is trajector moving along path from source to goal

days hours minutes

... ... ... ... ... ...

9 6 3 12 1 2 4 5 7 8 10 11

30 60 0

S G

TR

12 6 9 3 1 2 4 5 7 8 10 11

Constructing the Blend

a series of mapping gestures

12 6 9 3 1 2 4 5 7 8 10 11

The minute hand is down … Down is halfway from … (2x)

12 6 9 3 1 2 4 5 7 8 10 11

the top back to itself … x L R

12 6 9 3 1 2 4 5 7 8 10 11

So it’s half an hour … x R L

12 6 9 3 1 2 4 5 7 8 10 11

30 minutes x R L

Minute Hand Cycle

days hours minutes

... ... ... ... ... ...

9 6 3 12 1 2 4 5 7 8 10 11

30 60 0

S G

TR

12 6 9 3 1 2 4 5 7 8 10 11

Second Example

Novel Blend for Reading O’Clock Times

Novel Blend for O’Clock Times: Video Clip

“How would you read the time without seeing the numbers?”

Novel Blend

12 6 9 3 1 2 4 5 7 8 10 11

0º 180º 90º 270º

Consequence Can read “o’clock” times from angle:

• 90º is three o’clock
• 180º is six o’clock
• 270º is nine o’clock

Clock Angles

12 6 9 3 1 2 4 5 7 8 10 11

Emergent property Hour hand defines angle relative to minute hand

0º

θ

θ

Phase 1: Constructing the Blend

a series of mapping gestures

12 6 9 3 1 2 4 5 7 8 10 11

O we’ll put our …

• ur zero angle up here

(straight-up-and-down) 12 6 9 3 1 2 4 5 7 8 10 11

ninety degrees is three

12 6 9 3 1 2 4 5 7 8 10 11

a hundred ’n eighty degrees is six

12 6 9 3 1 2 4 5 7 8 10 11

straight up … straight-up-and-down 1 2

12 6 9 3 1 2 4 5 7 8 10 11

two hundred and seventy

(or ninety-degrees-the-other-way)

two seventy … is nine 1 2 3 x 4

(it’s a…)

Clock Angles Blend

12 6 9 3 1 2 4 5 7 8 10 11

0º 180º 90º 270º

12 6 9 3 1 2 4 5 7 8 10 11 0º

θ

θ

Phase 2: Running the Blend

manipulations (to run) & gestures (to profile)

So you can see it…

12 6 9 3 1 2 4 5 7 8 10 11

that’s obviously three

(now-we-know-it-without-the-numbers)

3 turns of minute hand

12 6 9 3 1 2 4 5 7 8 10 11

It’s … pointing up and down R L

12 6 9 3 1 2 4 5 7 8 10 11

3 turns of minute hand that’s six …

12 6 9 3 1 2 4 5 7 8 10 11

3 turns of minute hand that’s … nine

Discussion

Enaction, Imagination, and Insight

(Hutchins 2010)

• Two key ideas:

– Embodiment = “the premise that the particular bodies we have influence how we think” (428) – Enaction = “the idea that organisms create their own experience through their actions” (428)

• Enactments make material patterns into representations

– “Humans make material patterns into representations by enacting their meanings” (434). – “To apprehend a material pattern as a representation of something is to engage in specific culturally shaped perceptual processes” (429-30). – ‘Seeing-as’ is enacted representation (434).

Gestural Enactment vs. Actual Manipulation

• Gesturing over objects to:

– explore possibilities (act in “hypothetical mode” [Murphy 2004]) – construct an anchored blend for others or interpret its state (“guided conceptualization” [Williams 2008b])

• Manipulating objects to:

– prepare a material anchor for a blend (to make a representation) – run a blend to generate an inference

How Creative is the Clock Angles Blend?

• Using Boden’s (2004) criteria:

– New? – Surprising? – Valuable?

For time-telling, the hand configurations are useful, but the angle measures are not (esp. for children). The clock angles may have other uses: q relating clock hands, the sun, and compass direction? q estimating angles by relating them to clock hand configurations? Yes Yes Not very

Conclusion

• Continuity between conventional and novel blends

– Common conceptual mechanisms – Inherent in everyday activity but not noted unless novel (new, surprising) and useful (valuable)

• Emergence through embodied interaction with world

– Perception: ‘situated seeing’ or seeing-as – Action: manipulation of objects (incl. adventitiously) – Enaction: gesturing in hypothetical mode

• Propagation through multimodal discourse

– Gestures map conceptual entities and relations onto material structures, building anchored blends – Manipulation of the material structures ‘runs’ the blend to generate inferences

References

Boden, M. A. (2004), The Creative Mind: Myths and Mechanisms, 2nd Ed. London: Routledge. Fauconnier, G. & Turner, M. (1998). Conceptual integration networks. Cognitive Science, 22(2): 133-187. Fauconnier, G. & Turner, M. (2002). The Way We Think: Conceptual Blending and the Mind’s Hidden

• Complexities. New York: Basic Books.

Hutchins, E. (1995). Cognition in the Wild. Cambridge, MA: MIT Press. Hutchins, E. (2005). Material anchors for conceptual blends. Journal of Pragmatics, 37(10): 1555-1577. Hutchins, E. (2010). Enaction, imagination, and insight. In J. Stewart, O. Gapenne, & E. A. Di Paolo (eds.), Enaction: Towards a New Paradigm for Cognitive Science (pp. 425-450). Cambridge, MA: MIT Press. Koestler, A. (1964). The Act of Creation. New York: Macmillan. Murphy, K. (2004). Imagination as joint activity: The case of architectural interaction. Mind, Culture, and Activity, 11(4): 267-278. Thagard, P., & Stewart, T.C. The AHA! experience: Creativity through emergent binding in neural

• networks. Cognitive Science, 35: 1-33

Williams, R. F. (2008a). Gesture as a conceptual mapping tool. In A. Cienki & C. Müller (eds.), Metaphor and Gesture [Gesture Studies 3] (pp. 55-92). Amsterdam: John Benjamins. Williams, R. F. (2008b). Guided conceptualization: Mental spaces in instructional discourse. In T. Oakley & A. Hougaard (eds.), Mental Spaces in Discourse and Interaction (pp. 209-234). Amsterdam: John Benjamins.