[PPT] - Carolyn Penstein Ros Language Technologies Institute Human-Computer PowerPoint Presentation

SLIDE 1

Carolyn Penstein Rosé

Language Technologies Institute Human-Computer Interaction Institute School of Computer Science

With funding from the National Science Foundation and the Office of Naval Research And in partnership with the Cancer Care Community and the American Cancer Association

1

SLIDE 2

2

Special Thanks to My Collaborators

Miaomiao Wen
Dong Nguyen
Elijah Mayfield
Robert Kraut
John Levine
Yi-Chia Wang

* 2012 CMDA BCO team: Miaomiao Wen, Zeyu Zheng, Kenneth Huang, William Wang * 2013 CMDA BCO team: Zhou Yu, Shou-I Yu, Yajie Miao, Yuchen Zhang, Lu Jiang * Special thanks to David Adamson and Philip Gianfortoni for work on Stretchy Patterns

SLIDE 3

3

Developing technology capable of shaping conversation and supporting effective participation in conversation to achieve positive impact on…

SLIDE 4

4

Developing technology capable of shaping conversation and supporting effective participation in conversation to achieve positive impact on…

Human learning

SLIDE 5

5

Developing technology capable of shaping conversation and supporting effective participation in conversation to achieve positive impact on…

Human learning Health

SLIDE 6

6

Developing technology capable of shaping conversation and supporting effective participation in conversation to achieve positive impact on…

Human learning Health Wellbeing

SLIDE 7

7

Developing technology capable of shaping conversation and supporting effective participation in conversation to achieve positive impact on…

SLIDE 8

8

Automatic Analysis Of Conversation Conversational Interventions Positive Psycho-Social Outcomes

SLIDE 9

Many cancer patients and survivors participate in
nline health support groups (Chou et al., 2009; Chou

et al., 2011)

Online support groups decrease depression and

increase self-efficacy and quality of life (Rains & Young, 2009)

One benefit is just-in-time interaction
Participants don’t need to wait until their next doctor’s appointment or

weekly support-group meeting to get the support they need

Our goal: Understand how online health support

communities work and what the current problems are so we can design interventions to make them better

9

SLIDE 10

10

SLIDE 11

Defined a new event extraction task
Interesting because of informal temporal references
Created an annotated corpus
Coding manual and annotated data will be made publically

available

Developed a technical approach to event extraction

with informal temporal references

Builds on earlier work on structured feature induction

(Gianfortoni, Adamson, & Rosé, 2011)

Results: Temporal resolution beats state of the art

(Ji et al., 2011; McClosky & Manning, 2012; Garrido et al., 2012)

11

SLIDE 12

12

Cancer trajectory

Kuang-Yi Wen et. al. 2011

SLIDE 13

13

Extracting Cancer Histories

(Wen & Rosé, 2012)

SLIDE 14

Can we increase the extent to which participants receive the targeted support they need when they need it?

SLIDE 15

Emotional support = 5.2/ Informational support = 3.3
Hello! First off I want to say that I am sorry you have been diagnosed with

breast cancer and I know how you feel today. I know how scared and how

ver whelming it all is. We are here to help walk you through…
Lumpectomy vs. mastectomy is a very personal decision and not one that

can be made by anyone but you. Of course, you may get advice from your medical team. In my case I had a 4 cm. tumor so lumpectomy was not an

ption for me BUT that being said, I would have chosen mastectomy anyway.

For a variety of reasons but those reasons were my reasons…God comfort you and lead you as you go......

15

SLIDE 16

Socialization begins with
bserving community norms
Users may choose to adopt

these norms if they share community values and goals

Level of adoption reflects

commitment and identification

But what if goals are

changing over time?

Commmunities of practice

(Lave and Wenger, 1991 )

SLIDE 17

17

Technical Contribution: Getting Higher Resolution on

Participation Trajectories by Extracting Cancer Histories

Modeling the Problem: General Decline in Offered Support

Over Time

Understanding Effect of Experience: Modeling Changes in

Posting Behavior over Time Relative to Cancer Events

Looking to the Future: Interventions for Increasing Support

Exchange

Outline

SLIDE 18

18

Technical Contribution: Getting Higher Resolution on

Participation Trajectories by Extracting Cancer Histories

Modeling the Problem: General Decline in Offered Support

Over Time

Understanding Effect of Experience: Modeling Changes in

Posting Behavior over Time Relative to Cancer Events

Looking to the Future: Interventions for Increasing Support

Exchange

Outline

SLIDE 19

Cancer Event Extraction Task

Forum: Breastcancer.org
Task: For each user, identify which events have occurred in their

history and associate them with a month and year

Related prior tasks include the TAC-KBO 2011 (Ji et al., 2011) and the

timelining task (McClosky & Manning, 2012)

3 types of events
Status change event: breast cancer Diagnosis, Metastasis and Recurrence.
One-day event: Mastectomy, Lumpectomy, Reconstruction.
Event with a temporal bound: Chemotherapy and Radiation.
Annotated Corpus
Developed a reliable coding scheme (Kappas at least .9)
601 extracted temporal expressions from one post each of 1000 users
Complete posting history of 300 users
509 cancer events paired with event dates

SLIDE 20

<11/15/2008> I only have a mast on my left side, but I have noticed some pulling recently and I

won't start rads until March[Underspecified].

<11/20/2008> It is sloowwwly healing, so slowly, in fact, that she said she HOPES it will be healed

by March[Underspecified]. when I am supposed to start rads.

<1/13/2009> I still have one last chemo to go on the 19th[Underspecified] and then start rads in

5 wks[Relative].

<1/31/2009> I go for my first meeting with the rad onc on 2/10[Underspecified](my 50th

birthday[User-specific]!).

<2/23/2009> I had my first rad today[Relative].
<3/31/2009> Tomorrow[Relative] will be my last full rads, then I will start the boosts.
<4/2/2009> I started rads in Feb[Underspecified]. I just did #29 today[Relative].
<4/8/2009> The rad onc wants to see me again next week[Relative] for a skin check and I am on

antibiotics since I am at risk of getting cellulitis since I have had it twice since August[Underspecified].

<6/21/2010> My friend Lisa had her port put in last week[Relative] and will begin 2

weeks[Relative] of radiation on Tuesday[Underspecified].

20

SLIDE 21

<11/15/2008> I only have a mast on my left side, but I have noticed some pulling recently and I

won't start rads until March[Underspecified].

<11/20/2008> It is sloowwwly healing, so slowly, in fact, that she said she HOPES it will be healed

by March[Underspecified]. when I am supposed to start rads.

<1/13/2009> I still have one last chemo to go on the 19th[Underspecified] and then start rads in

5 wks[Relative].

<1/31/2009> I go for my first meeting with the rad onc on 2/10[Underspecified](my 50th

birthday[User-specific]!).

<2/23/2009> I had my first rad today[Relative].
<3/31/2009> Tomorrow[Relative] will be my last full rads, then I will start the boosts.
<4/2/2009> I started rads in Feb[Underspecified]. I just did #29 today[Relative].
<4/8/2009> The rad onc wants to see me again next week[Relative] for a skin check and I am on

antibiotics since I am at risk of getting cellulitis since I have had it twice since August[Underspecified].

<6/21/2010> My friend Lisa had her port put in last week[Relative] and will begin 2

weeks[Relative] of radiation on Tuesday[Underspecified].

21

SLIDE 22

<11/15/2008> I only have a mast on my left side, but I have noticed some pulling recently and I

won't start rads until March[Underspecified].

<11/20/2008> It is sloowwwly healing, so slowly, in fact, that she said she HOPES it will be healed

by March[Underspecified]. when I am supposed to start rads.

<1/13/2009> I still have one last chemo to go on the 19th[Underspecified] and then start rads in

5 wks[Relative].

<1/31/2009> I go for my first meeting with the rad onc on 2/10[Underspecified](my 50th

birthday[User-specific]!).

<2/23/2009> I had my first rad today[Relative].
<3/31/2009> Tomorrow[Relative] will be my last full rads, then I will start the boosts.
<4/2/2009> I started rads in Feb[Underspecified]. I just did #29 today[Relative].
<4/8/2009> The rad onc wants to see me again next week[Relative] for a skin check and I am on

antibiotics since I am at risk of getting cellulitis since I have had it twice since August[Underspecified].

<6/21/2010> My friend Lisa had her port put in last week[Relative] and will begin 2

weeks[Relative] of radiation on Tuesday[Underspecified].

22

SLIDE 23

<11/15/2008> I only have a mast on my left side, but I have noticed some pulling recently and I

won't start rads until March[Underspecified].

<11/20/2008> It is sloowwwly healing, so slowly, in fact, that she said she HOPES it will be healed

by March[Underspecified]. when I am supposed to start rads.

<1/13/2009> I still have one last chemo to go on the 19th[Underspecified] and then start rads in

5 wks[Relative].

<1/31/2009> I go for my first meeting with the rad onc on 2/10[Underspecified](my 50th

birthday[User-specific]!).

<2/23/2009> I had my first rad today[Relative].
<3/31/2009> Tomorrow[Relative] will be my last full rads, then I will start the boosts.
<4/2/2009> I started rads in Feb[Underspecified]. I just did #29 today[Relative].
<4/8/2009> The rad onc wants to see me again next week[Relative] for a skin check and I am on

antibiotics since I am at risk of getting cellulitis since I have had it twice since August[Underspecified].

<6/21/2010> My friend Lisa had her port put in last week[Relative] and will begin 2

weeks[Relative] of radiation on Tuesday[Underspecified].

23

SLIDE 24

KBP task provides the event, and the task is to find the

date

We have 8 potential events, but we do not know which ones

have occurred for which users

Prior work leverages temporal ordering constraints

between events (Talukdar et al., 2012; McClosky & Manning, 2012)

Looser ordering constraints on cancer events (Zhou &

Hripcsak, 2007)

Prior work assumes events expressed as verbs and

leverages event structure (tense and adverbials)

Cancer events frequently expressed as nouns, with contextual

evidence further removed from the event mention

24

SLIDE 25

Opensource temporal taggers:
HeidelTime (Strotgen & Gertz, 2010)
SUTime (Chang & Manning, 2012)
Mainly evaluated on newswire texts and

wikipedia

SUTime is the best state-of-the-art system

available

On our task: 85 F1 on extent recognition and

88.3 F1 on Normalization

25

SLIDE 26

Evaluated on 601 manually extracted

temporal references

26

SLIDE 27

Regular expressions are used

to extract a list of event keyword sentences

Temporal tagger extracts date

sentences

Extracted sentences are

filtered

Constraints on events are

extracted

Potential dates are aggregated
A final event date is selected

27

SLIDE 28

Similar approach

taken by McClosky and Manning

MaxEnt classifier to

identify sentences that express true event dates

Is the event connected

with the date?

Is it the author’s event

28

SLIDE 29

MaxEnt classifier to identify

evidence of an event

ccurring or not before or

after the post

Evidence that an event

happened before:

I’m officially off rads!
Evidence that an event never

happened

Radiation is not a choice for me

29

SLIDE 30

Rule based: Heuristics that codify typical

practices for informal temporal reference

Relate extracted temporal reference to post date
Rules about related times
Relate extracted temporal reference to other

extracted information about the user

Their birthday
Their cancer anniversary
When other events took place

30

SLIDE 31

Train on 250 users, test on 50
Baseline 1: pick temporal reference rated most likely
Baseline 2: Combine probabilities across alternative temporal references
Date Classifier: First filter references that are not predicted to refer to the

event date

Joint model: Maximize the joint likelihood of a temporal reference based

both on the Date classifier and Constraint classifier

Oracle: Treat result as correct if any of the extracted potential dates was the

correct one (upper bound)

31

SLIDE 32

32

SLIDE 33

33

Technical Contribution: Getting Higher Resolution on

Participation Trajectories by Extracting Cancer Histories

Modeling the Problem: General Decline in Offered

Support Over Time

Understanding Effect of Experience: Modeling Changes in

Posting Behavior over Time Relative to Cancer Events

Looking to the Future: Interventions for Increasing Support

Exchange

Outline

SLIDE 34

34

Growth Models

SLIDE 35

Community

35

Growth Models

Each user’s trajectory is a combination of:

1. Their personal trajectory (intercept and slope)
2. The trajectory effect of their community

(intercept and slope)

User

SLIDE 36

36

Growth Models

Each user’s trajectory is a combination of:

1. Their personal trajectory (intercept and slope)
2. The trajectory of their subcommunity (intercept

and slope)

3. The trajectory effect of their community

(intercept and slope)

Community Subcommunity User

SLIDE 37

37

Growth Models

Each user’s trajectory is a combination of:

1. Their personal trajectory (intercept and slope)
2. The trajectory effect of their community

(intercept and slope) First calculate a general trend:

SLIDE 38

38

Growth Models

First calculate a general trend: Next, adjust for each user’s intercept:

SLIDE 39

39

Growth Models

First calculate a general trend: Next, adjust for each user’s intercept… and slope: (each path below is a single user’s trajectory)

SLIDE 40

Modeling the Problem with Growth Models

40

Variant on two level growth model: Latent Trajectory

Analysis

Jointly estimates user trajectories and user clusters

(based on similarity of slopes and intercepts)

91% 7% 2% .1%

SLIDE 41

41

SLIDE 42

42

Technical Contribution: Getting Higher Resolution on

Participation Trajectories by Extracting Cancer Histories

Modeling the Problem: General Decline in Offered Support

Over Time

Understanding Effect of Experience: Modeling Changes

in Posting Behavior over Time Relative to Cancer Events

Looking to the Future: Interventions for Increasing Support

Exchange

Outline

SLIDE 43

43

Effect of Cancer Event Experience on Offering of Support

Before Metastasis After Metastasis

SLIDE 44

44

Effect Sizes of Increase in Support

Event

Inf. Support

Specific to Event

Inf. Support

Overall Emo. Support Specific to Event Emo. Support Overall Diagnosis .28 .48 .27 .12 Chemo-therapy .55 .20 .58 .18 Radiation .69 .34 .62 .24 Metastasis .88 .28 .83 .31 Recon-struction .71 .26 .61 .11 Mastect-omy .47 .20 .29 .14 Recur-rence .73 .42 .61 .26 Lumpec-tomy .58 .35 .49 .17 * In most cases the effect on targeted support was bigger than for all support.

SLIDE 45

45

Event Talk Growth Models

Dependent Measure: # messages mentioning event
Two level model:
Person Event-span slope and intercept
Population slope and intercept
Group variable: Before event/After event

Before Metastasis After Metastasis

SLIDE 46

46

Event Talk Growth Models

Dependent Measure: # messages mentioning event
Two level model:
Person Event-span slope and intercept
Population slope and intercept
Group variable: Before event/After event
E[ESevent|Mon] = Intmodel,event+ Mon*Slmodel,event

Before Metastasis After Metastasis

The expected amount of event targeted support offered by users for an event given the number of months it has been since their initial posting month

SLIDE 47

47

Event Talk Growth Models

Dependent Measure: # messages mentioning event
Two level model:
Person Event-span slope and intercept
Population slope and intercept
Group variable: Before event/After event
E[ESevent|Mon] = Intmodel,event+ Mon*Slmodel,event

Before Metastasis After Metastasis

The expected amount of event targeted support for an event for users in their initial posting month

SLIDE 48

48

Event Talk Growth Models

Dependent Measure: # messages mentioning event
Two level model:
Person Event-span slope and intercept
Population slope and intercept
Group variable: Before event/After event
E[ESevent|Mon] = Intmodel,event+ Mon*Slmodel,event

Before Metastasis After Metastasis

The number of months since the first posting month times the expected change per month in amount of event targeted support for an event

SLIDE 49

49

Event Talk Growth Models

Dependent Measure: # messages mentioning event
Two level model:
Person Event-span slope and intercept
Population slope and intercept
Group variable: Before event/After event
E[ESevent| EMonevent] =

Intmodel,event + Intevent + EMon*(Slmodel,event + Slevent)

Before Metastasis After Metastasis

The expected amount of event targeted support for an event offered by users who have experienced the event given the number of months it has been since experiencing the event

SLIDE 50

50

Event Talk Growth Models

Dependent Measure: # messages mentioning event
Two level model:
Person Event-span slope and intercept
Population slope and intercept
Group variable: Before event/After event
E[ESevent| EMonevent] =

Intmodel,event + Intevent + EMon*(Slmodel,event + Slevent)

Before Metastasis After Metastasis

The expected amount of event targeted support for an event offered by users who have experienced the event in the month when they experienced it

SLIDE 51

51

Event Talk Growth Models

Dependent Measure: # messages mentioning event
Two level model:
Person Event-span slope and intercept
Population slope and intercept
Group variable: Before event/After event
E[ESevent| EMonevent] =

Intmodel,event + Intevent + EMon*(Slmodel,event + Slevent)

Before Metastasis After Metastasis

The number of months since experiencing an event times the expected change per month in amount of event targeted support for an event in the post event model

SLIDE 52

52

Event Talk Growth Models

Dependent Measure: # messages mentioning event
Two level model:
Person Event-span slope and intercept
Population slope and intercept
Group variable: Before event/After event
If E[ESevent| EMonevent] > E[ESevent|Mon], then user is in an

elevated state of offering event targeted support for the event

Before Metastasis After Metastasis

SLIDE 53

53

Example User 1

Stage 1 breast cancer. Mastectomy and Reconstruction in second

posting month.

SLIDE 54

54

Example User 2

Stage 4 breast cancer. Diagnosis of Metastasis and Chemotherapy

in 21st month.

SLIDE 55

55

Technical Contribution: Getting Higher Resolution on

Participation Trajectories by Extracting Cancer Histories

Modeling the Problem: General Decline in Offered Support

Over Time

Understanding Effect of Experience: Modeling Changes in

Posting Behavior over Time Relative to Cancer Events

Looking to the Future: Interventions for Increasing

Support Exchange

Outline

SLIDE 56

56

SLIDE 57

Prior interventions for providing peer support have involved group-based support

(either face-to-face, over the phone, or in online communities) or matched one-

n-one support (either face-to-face or over the phone)
Negative effects and null results occur most often in matched phone interventions with meetings at

pre-specified times.

State-of-the-art: participants fill in applications providing information about their

condition, the matching is done manually, and the match is expected to persist

ver time.
Our interventions will match support seekers with available others who have a history of offering

support and personal characteristics indicating they have resources that support seekers need.

Our approach is based on automatic analysis of both seekers and potential suppliers of support and

can be tailored to the changing needs of participants.

We can offer matching experiences to a very large population of cancer patients.

57

SLIDE 58

58

SLIDE 59

59

Long time participation and adherence to community norms (Nguyen &

Rosé, 2011) does not imply that users become model citizens

Possible explanation: Goals change as disease state changes
Computational Models of Discourse Analysis provide needed insight
Novel temporal extraction methods enable summarization of cancer

histories

Growth modeling techniques provide insight into typical participation

trajectories of users

Understanding Effect of Experience: Modeling Changes in Posting

Behavior over Time Relative to Cancer Events Experience with cancer events may be able to be leveraged for increasing support offered

Important Future Work: Evaluate potential interventions building on

cancer history extraction

Conclusions

SLIDE 60

60