Gesture, self-repair and reasoning in schizophrenia Christine Howes - - PowerPoint PPT Presentation

Gesture, self-repair and reasoning in schizophrenia

Christine Howes

with Mary Lavelle, Patrick G.T. Healey, Ellen Breitholtz, Julian Hough, Rose McCabe

24th June 2016

Christine Howes2015

1

Background

2

Method

3

Results

4

Conclusions

Christine Howes2015

1

Background

2

Method

3

Results

4

Conclusions

Christine Howes2015

Background

Successful social encounters require mutual understanding between people People must monitor their own and others’ behaviour for potential misunderstandings Problems can be signalled non-verbally with interactive gestures (Bavelas et al., 1992) Or addressed verbally using repair (Schegloff et al., 1977)

Christine Howes2015

Background

For non-clinical participants:

repair can aid comprehension (Brennan and Schober, 2001) people compensate for verbal difficulties using head nods and hand gestures (Seyfeddinipur and Kita, 2014; Healey et al., 2013, 2015)

Is this also true for patients with schizophrenia?

Christine Howes2015

Background

For non-clinical participants:

repair can aid comprehension (Brennan and Schober, 2001) people compensate for verbal difficulties using head nods and hand gestures (Seyfeddinipur and Kita, 2014; Healey et al., 2013, 2015)

Is this also true for patients with schizophrenia? What can this tell us about “normal” interaction?

Christine Howes2015

Background

Patients with schizophrenia experience difficulties in social interaction. They:

use fewer self-repairs in their talk (Leudar et al., 1992; Caplan et al., 1996) have difficulty monitoring their own behaviour (Johns et al., 2001) display fewer hand gestures when speaking (Lavelle et al., 2013) have mismatched gesture use and speech (Millman et al., 2014) have difficulty interpreting figurative language and inferring others’ mental states (Gavil´ an and Garc´ ıa-Albea, 2011) jump to conclusions (Dudley and Over, 2003)

Christine Howes2015

Background

Most studies of patients’ nonverbal, verbal and reasoning behaviour relies on

non-interactive data (e.g. monologues, offline tasks) or dialogues with known interlocutors (e.g. therapist, family members)

Do these findings hold in social interactions, with unknown partners?

Christine Howes2015

Background

The presence of a patient with schizophrenia influences the nonverbal behaviour of their interacting partners:

in clinical contexts (Lavelle et al., 2015) and during first meetings with healthy controls, when the patient’s diagnosis is undisclosed (Lavelle et al., 2013, 2014)

Does interaction itself play a crucial role in patients’ deficits in dialogue? Do their healthy partners modify their own behaviours?

Christine Howes2015

Research questions

Compared to conversations without a patient, and patients’ healthy conversational partners:

1

Do patients with schizophrenia use less self-repair and gesture during conversation?

2

Is their use of self-repair associated with their use of gesture?

3

Do patients reason differently in dialogue?

Christine Howes2015

1

Background

2

Method

3

Results

4

Conclusions

Christine Howes2015

Data

Triadic conversations of approximately 5 minutes 20 patient interactions

• ne patient

two healthy controls who were unaware of the patient’s diagnosis

20 control interactions

three healthy participants

Christine Howes2015

Who would you throw out?

William Harris: Balloon pilot Susanne Harris: 7 months pregnant wife Robert Lewis: Cancer research scientist Heather Sloan: Musical child prodigy

Christine Howes2015

Data

(1) Video (2) Motion capture

Christine Howes2015

(3) Transcriptions

A The lady’s, the lady’s pregnant, so we’re saying we don’t want to throw her out C [Well, I don’t know, we can chuck her] A [And the little child], the little child, I don’t think she’s any use C Yeah, [who let a nine year old child go on a hot air balloon?] A [But, it it’s just because she’s young] B [Yeah]. A [But, sh-] even if they did throw her out, she’d be quite light B Yeah A So she wouldn’t really [make a difference] C [So you] should throw out the pregnant woman, [because like, she’s probably gonna be the heaviest, and]= A [She’s two people] C =I’m guessing they’re all like normal weight A Ummm, I think the husband B Yeah, but he’s he’s the only one with any flying experience

Christine Howes2015

Analysis

Self-repair (Hough and Purver, 2014)

STIR (STrongly Incremental Repair detection) state-of-the-art incremental disfluency detector (accuracy 0.81 repairs, 0.94 edit terms) self-repair rate per word calculated for each individual

Gesture (Lavelle et al., 2012)

derived from MoCap hand movements data through automatic gesture extraction. hand movement speeds > 1 sd above an individual’s mean hand movement speed

Christine Howes2015

Self-repair

A The lady’s, the lady’s pregnant, so we’re saying we don’t want to throw her out C [Well, I don’t know, we can chuck her] A [And the little child], the little child, I don’t think she’s any use C Yeah, [who let a nine year old child go on a hot air balloon?] A [But, it it’s just because she’s young] B [Yeah]. A [But, sh-] even if they did throw her out, she’d be quite light B Yeah A So she wouldn’t really [make a difference] C [So you] should throw out the pregnant woman, [because like, she’s probably gonna be the heaviest, and]= A [She’s sh- sh- two people] C =I’m guessing they’re all like normal weight A Ummm, I think the husband B Yeah, but he’s he’s the only one with any flying experience

Christine Howes2015

Analysis

Reasoning (Breitholtz et al., 2015)

Manual annotation Does the current turn contain a reason for throwing or saving one of the balloon’s

• ccupants?

Christine Howes2015

Reasoning

A The lady’s, the lady’s pregnant, so we’re saying we don’t want to throw her out C [Well, I don’t know, we can chuck her] A [And the little child], the little child, I don’t think she’s any use C Yeah, [who let a nine year old child go on a hot air balloon?] A [But, it it’s just because she’s young] B [Yeah]. A [But, sh-] even if they did throw her out, she’d be quite light B Yeah A So she wouldn’t really [make a difference] C [So you] should throw out the pregnant woman, [because like, she’s probably gonna be the heaviest, and]= A [She’s sh- sh- two people] C =I’m guessing they’re all like normal weight A Ummm, I think the husband B Yeah, but he’s he’s the only one with any flying experience

Christine Howes2015

1

Background

2

Method

3

Results

4

Conclusions

Christine Howes2015

Self-repair rate

N M (sd) β SE χ2 p Patient 19 0.01 (0.01)

• 0.02

0.004 12.59 <0.001 HP partner 38 0.02 (0.02)

• 0.01

0.004 2.78 0.1 Controls 57 0.03 (0.02)

Healthy participants in control groups used more self-repair than schizophrenia patients

Christine Howes2015

Overall gesture rate

N M (sd) β SE χ2 p Patient 19 7.2 (2.8) 0.21 0.84 0.06 0.8 HP partner 38 7.7 (3.0) 0.55 0.63 0.77 0.38 Controls 57 7.2 (3.0)

No significant differences in overall rates of gesture

Christine Howes2015

Speaker gesture rate

N M (sd) β SE χ2 p Patient 19 12.5 (2.8)

• 5.84

2.91 4.01 0.05 HP partner 38 13.1 (3.0)

• 3.29

1.99 2.78 0.1 Controls 57 16.5 (3.0)

Patients use significantly fewer hand gestures when speaking

Christine Howes2015

Gesture and self-repair

In control groups, self-repair is positively correlated with gesture

r = 0.35, p = 0.02

Christine Howes2015

Reasoning

Control Patient groups Total groups Control Patient Total Conversations 18 18 36 Participants 54 36 18 54 108 Turns per person 46.7 39.7 29.3 36.2 41.5 Words per person 391.1 367.0 210.9 315.0 353.0 Words per turn 8.4 9.3 7.2 8.7 8.5 Arguments per person 8.1 7.3 4.4 6.3 7.2 Arguments per turn 0.17 0.18 0.15 0.17 0.17

Christine Howes2015

Reasoning

Control Patient groups Total groups Control Patient Total Conversations 18 18 36 Participants 54 36 18 54 108 Turns per person 46.7 39.7 29.3 36.2 41.5 Words per person 391.1 367.0 210.9 315.0 353.0 Words per turn 8.4 9.3 7.2 8.7 8.5 Arguments per person 8.1 7.3 4.4 6.3 7.2 Arguments per turn 0.17 0.18 0.15 0.17 0.17

Patients come up with fewer arguments regarding who to throw out of the balloon

Christine Howes2015

Reasoning

Control Patient groups Total groups Control Patient Total Conversations 18 18 36 Participants 54 36 18 54 108 Turns per person 46.7 39.7 29.3 36.2 41.5 Words per person 391.1 367.0 210.9 315.0 353.0 Words per turn 8.4 9.3 7.2 8.7 8.5 Arguments per person 8.1 7.3 4.4 6.3 7.2 Arguments per turn 0.17 0.18 0.15 0.17 0.17

Patients come up with fewer arguments regarding who to throw out of the balloon Patients make fewer dialogue contributions

Christine Howes2015

Reasoning

Control Patient groups Total groups Control Patient Total Conversations 18 18 36 Participants 54 36 18 54 108 Turns per person 46.7 39.7 29.3 36.2 41.5 Words per person 391.1 367.0 210.9 315.0 353.0 Words per turn 8.4 9.3 7.2 8.7 8.5 Arguments per person 8.1 7.3 6.3 7.2 Arguments per turn 0.17 0.18 0.15 0.17 0.17

Patients’ partners use more words per turn

Christine Howes2015

Reasoning

Patients’ reasoning may be less likely to change over the course of a conversation

despite 200 intervening turns including different arguments put forward by the patient’s interlocutors

4: but I’m gonna say that I would go for <unclear> Sue ’cause like she’s got her baby so is extra weight . . . 209: I’d still go with Sue though you know what I mean she’s carrying extra weight like

Christine Howes2015

1

Background

2

Method

3

Results

4

Conclusions

Christine Howes2015

During social interaction

Patients with schizophrenia . . .

repair their own speech less gesture less when speaking make less use of hand gesture during repair reason less in interaction

Patients’ healthy interlocutors also . . .

display reduced association between self-repair and gesture use longer turns

Christine Howes2015

During social interaction

Patients’ behaviours do not follow typical patterns

which may be (unconsciously?) picked up on by their interlocutors who may modify their behaviours (to compensate?)

This contributes to the social exclusion experienced by patients and can also offer insights into how “normal” interaction works

Christine Howes2015

Future directions

This is a really broad brush approach

by participant

• ver the whole conversation

but suggests we should look more closely at gesture, repair and reasoning

and the usefulness of automatically derivable data from speech and MoCap (potentially live)

(the start of) a fruitful line of research in investigating the difficulties experienced by patients in social interaction . . .

Christine Howes2015

Questions?

Thanks to all my many co-authors! And many others...

Christine Howes2015

References I

Bavelas, J. B., Chovil, N., Lawrie, D., and Wade, L. (1992). Interactive gestures. Discourse Processes, 15:469–489. Breitholtz, E., Howes, C., and Lavelle, M. (2015). Enthymematic reasoning in a moral dilemma - do patients with schizophrenia reason differently? In (In)coherence of discourse 3, Loria Inria Nancy Grand-Est, Nancy. Brennan, S. and Schober, M. (2001). How listeners compensate for disfluencies in spontaneous speech. Journal of Memory and Language, 44(2):274–296. Caplan, R., Guthrie, D., and Komo, S. (1996). Conversational repair in schizophrenic and normal children. Journal of the American Academy of Child & Adolescent Psychiatry, 35(7):950 – 958. Dudley, R. and Over, D. (2003). People with delusions jump to conclusions: a theoretical account of research findings on the reasoning of people with delusions. Clinical Psychology & Psychotherapy, 10(5):263–274. Gavil´ an, J. M. and Garc´ ıa-Albea, J. E. (2011). Theory of mind and language comprehension in schizophrenia: Poor mind-reading affects figurative language comprehension beyond intelligence deficits. Journal of Neurolinguistics, 24(1):54–69. Healey, P . G. T., Lavelle, M., Howes, C., Battersby, S., and McCabe, R. (2013). How listeners respond to speaker’s troubles. In Proceedings of the 35th Annual Conference of the Cognitive Science Society, Berlin.

Christine Howes2015

References II

Healey, P . G. T., Plant, N., Howes, C., and Lavelle, M. (2015). When words fail: Collaborative gestures during clarification dialogues. In 2015 AAAI Spring Symposium Series: Turn-Taking and Coordination in Human-Machine Interaction. Hough, J. and Purver, M. (2014). Strongly incremental repair detection. In Proceedings

• f the 2014 Conference on Empirical Methods in Natural Language Processing

(EMNLP), Doha, Qatar. Association for Computational Linguistics. Johns, L. C., Rossell, S., Frith, C., Ahmad, F., Hemsley, D., Kuipers, E., and McGuire, P . (2001). Verbal self-monitoring and auditory verbal hallucinations in patients with

• schizophrenia. Psychological medicine, 31(04):705–715.

Lavelle, M., Dimic, S., Wildgrube, C., McCabe, R., and Priebe, S. (2015). Non-verbal communication in meetings of psychiatrists and patients with schizophrenia. Acta Psychiatrica Scandinavica, 131(3):197–205. Lavelle, M., Healey, P . G., and McCabe, R. (2013). Is nonverbal communication disrupted in interactions involving patients with schizophrenia? Schizophrenia bulletin, 39(5):1150–1158. Lavelle, M., Healey, P . G., and McCabe, R. (2014). Participation during first social encounters in schizophrenia. PloS one, 9(1). Lavelle, M., Healey, P . G. T., and McCabe, R. (2012). Is nonverbal communication disrupted in interactions involving patients with schizophrenia? Schizophrenia Bulletin.

Christine Howes2015

References III

Leudar, I., Thomas, P ., and Johnston, M. (1992). Self-repair in dialogues of schizophrenics: Effects of hallucinations and negative symptoms. Brain and Language, 43(3):487 – 511. Millman, Z. B., Goss, J., Schiffman, J., Mejias, J., Gupta, T., and Mittal, V. A. (2014). Mismatch and lexical retrieval gestures are associated with visual information processing, verbal production, and symptomatology in youth at high risk for

• psychosis. Schizophrenia Research, 158(1-3):64 – 68.

Schegloff, E., Jefferson, G., and Sacks, H. (1977). The preference for self-correction in the organization of repair in conversation. Language, 53(2):361–382. Seyfeddinipur, M. and Kita, S. (2014). Gestures and self-monitoring in speech

• production. In Annual Meeting of the Berkeley Linguistics Society, volume 27,

pages 457–464.

Christine Howes2015