Promoting Flexible Idea Generation During Audit Planning Elizabeth - - PowerPoint PPT Presentation

Promoting Flexible Idea Generation During Audit Planning

Elizabeth (E.B.) Altiero University of Central Florida altiero@ucf.edu

Research Question

• In the face of increased regulatory scrutiny, how can we help

promote flexible thinking during audit procedure design?

– Specifically: idea generation regarding available audit evidence

• Publicized failures lead to emphasis on error reduction
• Non-audit example: The Central Intelligence Agency

Errors Insights Performance Improvements (Graphic adapted from Klein 2013)

Motivation & Contribution

• Prior research focuses primarily on error reduction:

Errors Insights Performance Improvements  Expertise  Effort  Accountability  Bias reduction  Skepticism  Fraud brainstorming  Pattern recognition

Motivation & Contribution

• Why do we want flexible idea generation during audit

planning? Error reduction is not enough!

– Keep up with the pace of business change (Chipman 2013;

Arning 2013)

– Respond to unique situations encountered in the field

(Altiero 2016)

– Provide unpredictability in audit procedures (AICPA 2006;

PCAOB 2007; IASB 2009; Bowlin 2011; Budescu et al. 2012)

– Improve audit effectiveness and efficiency of existing procedures (Altiero 2016)

Unfamiliar Tasks Familiar Tasks

Rethinking Audit Evidence

• Auditors need to be able to consider new types of audit

evidence to achieve these goals

• “New” might be:

– New to an audit specifically – New to auditing in general

Rethinking Audit Evidence

• Auditors need to be able to consider new types of audit

evidence to achieve these goals

• “New” might be:

– New to an audit specifically – New to auditing in general

Rethinking Audit Evidence

• Auditors need to be able to consider new types of audit

evidence to achieve these goals

• “New” might be:

– New to an audit specifically – New to auditing in general

Promoting Idea Generation

• Prior research shows auditors have difficulty modifying audit

plans used in prior years (Wright 1988; Hammersley et al. 2011)

• How to improve idea generation depends on why auditors are

having difficulty

– Adherence to industry norms – Lack of incentives (Peecher et al. 2013) – Time pressure (see Bonner 2008 for a review) – Psychological barriers, such as anchoring and production blocking (Joyce and Biddle 1981; Wright 1988; Asare and Wright 2004; Brazel et al.

2004; Carpenter 2007)

– Necessary changes are unclear due to changing business environment

Search for Ideas in Associative Memory (SIAM)

Problem Feature Problem Feature Problem Feature Problem Feature Problem Feature Search Cue Relevant Knowledge Relevant Knowledge Relevant Knowledge Other Knowledge Other Knowledge Idea

Ideas cannot be created out of nothing, must be based on existing knowledge.

(Model adapted from Nijstad and Stoebe 2006)

Idea

The more relevant knowledge activated, the more viable ideas generated.

Counterfactual Mindset

• Potential to activate larger set of relevant knowledge
• Counterfactual thinking – consideration of what could have been

– Increases consideration of disconfirmatory evidence (Kray and Galinsky 2003) – Increases information sharing in groups (Galinsky and Kray 2004) – Decreases effect of curse of knowledge (Kennedy 1995) – Increases skepticism of management’s explanations (Koonce 1992)

• Counterfactual mindset – counterfactual thinking can carry over to

subsequent, unrelated tasks (Galinsky et al. 2000; Galinsky and Moskowitz 2000)

– Helps overcome functional fixedness (Galinsky and Moskowitz 2000) Auditors are already trying to think of alternatives, so conscious prompt is not enough.

Task Characteristics Matter

• Task characteristics impact the ability to process problem

features into search cues

• Familiarity of task particularly important to idea generation

– Ease with which task cues can be linked to relevant knowledge

(input clarity in Bonner 1994 and Chen et al. 2014)

– Degree to which contextual cues highlight or obscure audit content (Tversky 1977; Einhorn and Hogarth 1981)

Search for Ideas: Familiar Task

Problem Feature Problem Feature Problem Feature Problem Feature Problem Feature Search Cue Relevant Knowledge Relevant Knowledge Relevant Knowledge Other Knowledge Other Knowledge Idea

PROBLEM Strong cue-idea link is hard to break. Suppresses

• ther knowledge and related ideas.

SOLUTION Prompt that broadens the relevant knowledge activated by any given cue.

Search for Ideas: Unfamiliar Task

Problem Feature Problem Feature Problem Feature Problem Feature Problem Feature Ill-formed Search Cue Relevant Knowledge Relevant Knowledge Relevant Knowledge Other Knowledge Other Knowledge Non-viable Idea

PROBLEM Poorly directed knowledge search due to difficulty interpreting problem and forming search cue. SOLUTION Prompt that improves search cue formation and directs knowledge search to relevant areas.

H1: Counterfactual Mindset x Task

Unfamiliar Familiar Audit Evidence Idea Generation Range of Theory Consistent Effects Strong Directional Prediction No Counterfactual Mindset Counterfactual Mindset Beneficial Counterfactual Mindset Detrimental

Examples

• So how to better direct knowledge

search (to help auditors with unfamiliar tasks)?

– Improve search cue formation – Examples are well suited to task and highly present in audit setting

• From brainstorming and creativity

literature

– Ideas of others can act as cognitive stimulants (Carpenter 2007; Chen et al. 2012) – Cognitive stimulants act as, or add to, a search cue to stimulate relevant knowledge

But auditors have difficulty generating alternatives! (Heiman 1990) So maybe examples will decrease idea generation… Effectiveness depends on similarity to own ideas and timing of introduction (Nijstad et al. 2003).

Counterfactual Mindset and Examples

• Counterfactual mindset promotes relational processing

style

– Increased consideration of relationships between task embedded cues (Kray et al. 2006)

• Counterfactual mindset + Examples

– Relational processing fixates cognition on relationship between the problem features and the example as a solution – Skips search cue formation  delays search for relevant knowledge (non-task embedded information)

H2: Counterfactual x Example

Unfamiliar Task (H2a)

No Counterfactual Counterfactual Audit Evidence Idea Generation Counterfactual Mindset Absent Present Example

• 1
• 1, -1

Familiar Task (H2b)

No Counterfactual Counterfactual Audit Evidence Idea Generation Counterfactual Mindset Absent Present Example

• 3
• 1

+2, +2 +3

Design

• 2 x 2 x 2 between subjects experiment

– Counterfactual mindset (no counterfactual, counterfactual) – Example (absent, present) – Task (lower familiarity, higher familiarity)

• Task

– List potential audit evidence items and audit procedures for hypothetical audit engagement – DV: Flexible idea generation

• Coded for originality, fluency, flexibility and elaboration (Guilford 1967)
• 138 experienced audit professional participants

Task

Lower Familiarity

• Identify audit evidence to gain

assurance over number of golf balls that fit in a school bus

• High audit content – audit

knowledge is relevant

• Low audit context – setting
• bscures relevance of audit

knowledge

Higher Familiarity

• Identify audit evidence to gain

assurance over the revenues from a toll booth’s operations

• High audit content – audit

knowledge is relevant

• High audit context – setting

provides clear cues to relevance of audit knowledge

Counterfactual Mindset

No Counterfactual Counterfactual

• Read story prior to primary task •

Read story prior to primary task

• Jane switches seats upon

arriving at a concert

• Jane wins a trip to Hawaii

based on her seat at the concert

• Jane wins a trip to Hawaii

based on her seat at the concert

• Participants list thoughts Jane

might be having

• Participants list thoughts Jane

might be having

Example

Absent

• Sentence of task that contains the

example is omitted

Present

• Example audit evidence that

could be obtained

• Both cases – data for use in

volume related analytical procedure

• Lower familiarity– school bus

dimensions

• Higher familiarity– average daily

traffic flow

Flexible Idea Generation

• Fluency (number of ideas): 89.0% coder agreement

– 1 point per evidence item listed

• Breadth (variety of ideas): 86.1% coder agreement

– 1 point per category: inspection of documents, inspection of tangible items, observation, inquiry, etc. (AU 326)

• Elaboration (detail specification of ideas): 76.6% coder agreement

– 1 point per modifier or additional detail

• Originality (uniqueness of ideas)

– Point scale based on number of participants who gave each response (1 participant = 8 points; 2 participants = 7 points; etc.) – Final score equal to highest scoring response

Results – H1

• 0.3
• 0.2
• 0.1

0.0 0.1 0.2 0.3

No Counterfactual Counterfactual Audit Evidence Idea Generation Lower Familiarity Higher Familiarity

• Predicted interaction of

Counterfactual and Task is significant (F1,128 = 3.23, p = 0.038, one-tailed)

• Simple effect of Counterfactual

given Higher Familiarity is marginally significant (F1,128 = 2.54, p = 0.057, one-tailed)

Results support H1.

Results - H2

Lower Familiarity (H2a)

• 0.2
• 0.1

0.0 0.1 0.2 0.3 0.4 0.5 0.6

No Counterfactual Counterfactual Audit Evidence Idea Generation Absent Present Example

• 1, -1

Higher Familiarity (H2b)

• 0.6
• 0.5
• 0.4
• 0.3
• 0.2
• 0.1

0.0 0.1 0.2

No Counterfactual Counterfactual Audit Evidence Idea Generation Absent Present Example +2, +2

• 1
• 3

Contrast significant (p = 0.046).

Note: p-values one-tailed. +3

• 1

Contrast significant (p = 0.032).

Results support H2a and H2b.

Supplemental Analysis: Audit Efficiency

• Fewer estimated hours required

in conditions with improved idea generation

• Promoting flexible idea

generation has potential to improve audit efficiency

30 40 50 60 70 80 90

No Counterfactual Counterfactual Estimated Hours to Complete Audit

Higher Familiarity

Absent Present Example No intervention

Conclusions

• Flexible idea generation has potential to improve audit processes

– Unfamiliar tasks:

• Use examples
• Avoid counterfactual mindset (How? Future research!)

– Familiar tasks:

• Prime a counterfactual mindset
• Recommendations to audit practice

– Design audit planning system to follow above recommendations and aid auditors in overcoming subconscious psychological barriers – Use system as a control that can help generate insights rather than avoid errors

Contributions

• Identify example presence and familiarity as new moderators of

counterfactual thinking in idea generation tasks

– Familiarity is closely linked to expertise (activation of relevant knowledge) – Highlights need for further research using professional subjects performing professional tasks

• Potential downstream benefit of improved audit planning on audit

efficiency

THA HANK Y K YOU!

Elizabeth (E.B.) Altiero University of Central Florida altiero@ucf.edu