Sensemaking in Dual Artefact Tasks The Case of Business Process - - PowerPoint PPT Presentation

Sensemaking in Dual Artefact Tasks – The Case of Business Process Models and Business Rules

By Tianwa Chen, Shazia Sadiq and Marta Indulska

ER 2020 - 39th International Conference on Conceptual Modeling November 3-6, 2020 in Vienna, Austria

Sensemaking in Dual Artefact Tasks – The Case of Business Process Models and Business Rules

Research Background Aim of Study Study Design Scanning and attention Task Specific Information Processing Summary

1

2

Research Background

• The widespread problem of

information silos in organizations.

• Knowledge workers have to navigate

multiple information artefacts.

• Understanding of a task will be based
• n both the business process model

and any related business rules [1].

References: [1] Wang, W., Indulska, M., Sadiq, S.W.: Cognitive efforts in using integrated models of business processes and rules. In: CAiSE

• Forum. pp. 33–40 (2016)

Business Process and Rule Integration [2, 3]

• Text annotation
• Diagrammatic integration
• Link integration

3

References: [2] Knolmayer, G., Endl, R., Pfahrer, M.: Modeling processes and workflows by business rules. In: Business Process Management, pp. 16–29. Springer (2000) [3] Chen, T., Wang, W., Indulska, M., Sadiq, S.: Business process and rule integration approaches-an empirical analysis. In: International Conference on Business Process Management. pp. 37–52. Springer (2018)

Research Background

• Fig. 1. Business rules integration approaches

Sensemaking

4

References: [4] Russell, D.M., Stefik, M.J., Pirolli, P., Card, S.K.: The cost structure of sensemaking. In: Proceedings of the INTERACT’93 and CHI’93 conference on Human factors in computing systems. pp. 269–276 (1993) [5] Klein, G., Moon, B., Hoffman, R.R.: Making sense of sensemaking 2: A macrocognitive model. IEEE Intelligent systems 21(5), 88–92 (2006)

Research Background

• Defined as “the process of searching for a representation and

encoding data in that representation to answer task-specific questions” [4]

• Individual cognitive processes: information foraging and task

specific information processing [5].

• Foundational sensemaking constructs of attention (search and

encoding) and memory (performance on task-specific questions).

• Limited knowledge on how knowledge workers make sense of the

various representations in the context of business process and business rule integration

5

Aim of Study

How knowledge workers make sense of dual artefacts (case of business process models and business rules)? What effect does integration approaches have on the efficacy of accomplishing a task that required dual artefacts, including quality of the task performance, time and effort efficiency?

6

Study Design

• Controlled lab experiment
• Experiment instruments: a tutorial, the treatments and a questionnaire
• Participants: 75 university students with foundational knowledge in

conceptual modeling (such as flowcharts, BPMN, UML or ER)

• Experiment data: a pre-experiment questionnaire, eye tracking log data

and task performance data

• Tobii Pro TX300 eye tracker: captures data on fixations, gaze, saccades,

etc, with timestamps

• No limit on the experiment duration nor a word count limit on participants’

answers

7

Study Design

• Fig. 2. Visual experiment design

Areas of Interest (AOI) to capture eye movements

8

Study Design

• Treatment: Informationally equivalent models for three integration

approaches with 25 participants per treatment group

• Diversity in terms of constructs and coverage

Questi

• n

Model Constructs Model Coverage Q1 Sequence, AND gateways Local area Q2 Sequence, AND gateways Local area Q3 Sequence, AND gateways, XOR gateways Global and local areas Table 1. Comparison of questions

9

Study Design

Searching and Encoding Phase (Understanding phase) Task specific information processing phase (Answering phase) Capture sensemaking behaviour: fixation durations, frequencies, task performance data, measurements related to AOI specific fixations, and transitions between AOIs. commences when the participant first fixates on the experiment screen. commences when the participant starts to type the answer in the question area for the first time.

Time

10

Scanning and attention

• Fig. 3. Mean fixation duration of each question for all participants

11

Scanning and attention

• Fig. 4. Heat maps and AOI measures in phase 1 for best performers

12

Scanning and attention

• Measured through mean fixation duration.
• Link representation requires less attention.
• For all participants this is observed in the initial question (Q1) and

again as task complexity increases in the global question (Q3).

• For best performers, the lower level of attention required is again noted

as task complexity increases, reflected through global question (Q3).

• We also note that the transition loops are diverse in the text and link

group compared with the diagrammatic group, which has the highest transition frequency between relevant area and question area.

When question complexity increases

13

Task Specific Information Processing

Number of correct answers 1 2 3 Text 9 9 6 Diagrammatic 9 7 6 2 Link 4 13 7 (a) Number of correct answers

• Fig. 5. Task performance

(b) Understanding accuracy

14

Task Specific Information Processing

• Fig. 6. Sequence of fixations in answering phase for best performers

15

Task Specific Information Processing

• Link representation requires the least attention on global question Q3,

indicating favorable performance as task complexity increases.

• For all groups: reduced transitions (proportion of transition frequency

count) in the answering phase as compared to the understanding phase, between relevant and other area, and between question and

• ther area.
• Link group: All questions show reduced transitions between rule and
• ther area and reduced transitions between rule and relevant area.
• After the understanding phase is complete, participants still engage in

deep processing (number of long fixations above 500 milliseconds) of information in the answering phase.

16

By using a sensemaking lens, we investigated how user behavior occurs in dual artefact tasks when the form of integrated representation of the artefacts (namely business process models and business rules) and task complexity changes. Link representation shows better task performance in terms of accuracy as well as efficiency, especially as task complexity increases. Our results provide some evidence that diagrammatic integration has better task performance on local questions in terms of accuracy, but also requires the most effort in the initial information foraging (understanding) phase. Limitation: basic constructs in business process models; the limitation of the eye tracking software limits the granularity of the AOI; We have mostly analyzed and presented the results of performers who answered the questions correctly. Our results contribute to a better understanding of the sense making processes in various settings and inform modeling practice. Our study provides a methodological contribution by offering an approach to visualize the different behaviors inherent in the two phases of sensemaking.

Summary

Thank you!

• Tianwa Chen

tianwa.chen@uq.edu.au

• Shazia Sadiq

shazia@itee.uq.edu.au

• Marta Indulska

m.indulska@business.uq.edu.au Sensemaking in Dual Artefact Tasks – The Case of Business Process Models and Business Rules

ER 2020 - 39th International Conference on Conceptual Modeling November 3-6, 2020 in Vienna, Austria