A critical review of qualitative research outcomes in executive - - PowerPoint PPT Presentation

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice

A critical review of qualitative research outcomes in executive coaching and mentoring

What do we know from qualitative research in coaching and mentoring about client attributes that may be important to effectiveness?

The Jazz of Research Ethics (e.g. Aristotle; Denzin 1995; Mays &

Pope, 1997; Savin-Baden & Major, 2010; Niekerk & Savin-Baden, 2010)

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

AGENDA

Introduction to the presentation – What are your expectations? The Anchoring Paradigm – What’s the context? Coaching science – What‘s the contribution? Problematisation – What do we get wrong? Research design– Which step leads to which result? Conclusions – What do know now & What do I intend to do next? Open group dialogue

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

• A. The Anchoring Paradigm – What‘s my space?

‚The road to Dodoville needs paving with more than good assumptions‘ (Lowman, 2005)

Scientific Anchoring Paradigm, Tuende Erdoes, 2017

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

• B. Combining Lenses – Building theory (Okhuysen & Bonardi, 2011)

At best, the researcher builds an integration of the tacit and explicit into a trustworthy and defensible account (Polanyi, 1962, 1967).

Combining Lenses – Building Theory, Tuende Erdoes, 2017

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Coaching science – What‘s my contribution?

1.) Conceptual contribution

This review proposes an approach to research synthesis as interpretation and translation of

• evidence. This perspective represents an understanding of research synthesis that is

different to the ones emerging from the basic paradigmatic and positivist positions inherent in organizational research.

2.) Methodological contribution

This review illustrates the research process of building theory via meta-synthesizing through CIS and QRS of a multiplicity of studies. Going beyond conventional or systematic literature reviews (e.g., Rousseau et al., 2008; Tranfield, Denyer, & Smart, 2003), it seeks to synthesize the key independent variables and underlying relationships across a multiplicity of published qualitative studies to arrive at a refined, extended, or even new theory.

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Motivation for this literature research – Why do that?

A taken-for-granted assumption MEETS a genuine doubt What do we not understand? What would go wrong if we failed to look into the research outcome conducted in the domain of qualitative research in relation to the research question? For whom is coaching most suited? If personality does not matter, how come that Alphas are not coachable? Or, are they? And if so, how? What is this lack of apparent consistency about? Conger (1998; quoted in Sztucinski, 2002) asserts that although qualitative studies are underutilized, they must play a central role in leadership research since they serve to illuminate in radically new ways phenomenon as complex as leadership. Since the current research on executive coaching is scarce, particularly from the executive’s perspective, my purpose was to gain a deeper and more comprehensive understanding of the phenomenon. Therefore, a qualitative framework was ideal for this research, since it allowed me to probe and understand the meanings that are constructed and attached to the coaching process by the executive. I want to create an evaluation model that provides a better opportunity for the systematic selection of relevant criteria.

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Problematisation – What do we get wrong?

Personality does not directly predict outcome in coaching – Debate in science It might: rather than describing personality through assessment models we should rather ask the question: ‚Why people behave the way they do‘ (McDowell, 2016, Hampson, 2012) Literature review findings support Hampson’s (2012) purported view that personality processes are bound up with emotion. What I find is that emotion is not being sufficiently studied. Trigger was professional wisdom: I noticed that Alpha type personalities are (not) coachable: personality matters. I cannot change behaviour unless client’s attitude changes and unless his basic needs are met (emotional stability, safety is ensured). I need an idiosynchratic approach to each client. Can we use random measurement methods to understand idiosnychratic mechanisms? What is the force of generalizability about in a field that is barely generalizable and quantifiable?

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Qualitative Literature Synthesis – What‘s the fit?

To address themes for which ‚little or no previous theory exists ........ or else they represent new phenomena in the world.‘ (Edmondson & McManus, 2007: 1161-2) For developing theoretical ideas through the active mobilisation and problematisation

• f existing frameworks (Alvesson & Kärreman,

2007:1265) Some phenomena can only be explored and

understood through a qualitatie approach;

e.g. research question:‚to what extent do client attributes influence coaching outcomes‘ can be best answered via qualitative synthesis

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

What are the steps in the QRS?

Formulating the reseasrch question Deciding what will count as data Searching for studies Selecting the sample of original studies (inclusion & exclusion criteria, sampling process, ensuring quality/plausibility, quality checklist) Treating data (analysis: first-order themes; synthesis: developing second-order themes; interpretation: developing third-order themes) Establishing plausibility (locating realities: what seems to be true given the context and the studies included in the investigation; what do stakeholders consider as ‚reality‘) Presenting the synthesis

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 1 – Initial sampling

• Database search

strategies (Shaw, et al., 2004)

• Cochrane review
• Handbook
• ABRI – Web of Science
• Other methods for

locating papers (Barroso, et al., 2003)

• Inclusion & Exclusion

criteria & Sampling steps Patton (2002)

• Issues I encountered

123

Table 1: Title and abstract screening

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Evidence - Deciding what will count as data

What is the level of formulation (is it well-founded, how rigorous) of science base and how relevant is it for practice (socially robust?)

Adopted from Grant, 2017, Hodgkinson, 2001; Dixon-Woods, et al., 2006); see slide about Hierarchy of Evidence

• Combinging Lenses

Mentoring & Training

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 2 – Final sampling

• Further

criteria as

• ntolog.

& epistemol. starting points that coloured the methods and enthusiasm with which I was engaging with evidence (Nutley & Smith, 2000)

• Visibility

crucial for ensuring an accurate audit trail (Houghton et

• al. 2013)
• Saturation level

105

Table 2: Detailed descriptive analysis

Goal: maximise relevance and theoretical contribution of included papers

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 3 – Clustering & Quality Criteria (Lincoln & Cuba, 1985)

Table 3: Hierarchy of evidence – Credibility of studies

Thematic Analysis (Thomas & Harden, 2008)

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 4 – Approach to qualitative literature review

CIS does not aim to offer a series of pre-specified procedures for the conduct of review. It explicitly acknowledges the "authorial voice"; that some aspects of its production of the account of the evidence will not be visible or auditable; and that its account may not be strictly

• reproducible. Its aim is to offer a theoretically sound and useful account that is

demonstrably grounded in the evidence. (Noblit & Hare, 1998; Dixon-Woods, 2008) CIS demanded constant reflexivity on my part as the author of the review. Authors are charged with making conscientious and thorough searches, with making fair and appropriate selections of materials, with seeking disconfirming evidence and other challenges to the emergent theory, and with ensuring that the theory they generate is, while critically informed, plausible given the available evidence. I used the interpretative and translation synthesis methods (Hoon, 2013) for the critical synthesis of data. QRS - complements the interpretative critical synthesis approach in that it enables researchers to summarize existing studies in ways that are informative to policy makers and practitioners, and also enables the knowledge gained through such studies to be more widely available to others. It is an approach that is methodologically grounded and rigorous too since it seeks to answer a specific research question through combining qualitative studies that use thick description and that are located in broadly the same tradition. (Savin-Baden & Major, 2010)

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Pros & Cons of QRS

PROS: § methodologically grounded approach to making sense of existing research § can help scholars make connection between studies § can help scholars see the gap and omissions in the literature base § can allow for theory building § can put research findings into the hands of those who can make use of them CONS: § context is stripped through the process of synthesis § limited to questions that have already been asked and answered § synthesists have not had input into the study design § synthesists do not have access to original or primary data

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 5 – Identifying first-order themes

Table 4: First-order themes

• Open coding (Glaser &

Strauss, 1978; Strauss & Corbin, 1990)

• Categories shape up
• Constant reflexivity
• Constant comparison

with categories

• Comparison across

studies

• Relationship between

study types

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 6 – Identifying second-order themes I

Synthesis: developing second-order themes

Table 5: Second-order themes

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 6 – Identifying second-order themes II

• What is behaviour?
• What is attitude
• What is emotion?
• What is the direct evidence?

Conceptual dissonance

Table 6: Second-order themes - Details

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 7 – Third-order themes & Translation Synthesis I

Translation synthesis – Thematic translation

Purpose: translate studies into one another with the aim of drawing cross-case conclusions (Noblit & Hare, 1988)

Analogical reasoning

Table 7: Third-order themes

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 8 – Translation Synthesis II

Relationship translation: Client-Coach Impact

Purpose: to understand what goes on in client-coach relationship / process

The benefits of a leadership program and executive coaching for new nursing academic administrators: One college’s experience (Smith Glasgow, M., Weinstock, B., Lachman, V., Dunphy Suplee, P., & Dreher, H. (2009)

• Which client attribute

impacts the coaching process positively, negatively, or not at all?

• Whom does it impact:

e.g. what is the coach’s impact on client attributes? What is the client‘s impact on the coaching process?

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 8 – Translation Synthesis II – Illustration

Results of Client-Coach Impact – Relationship Translation

10 20 30 40 50 60 70 EMOTIONS ATTITUDE BEHAVIOUR

POSITIVE IMPACT - ALL STUDIES

COACH CLIENT

10 20 30 40 50 60 70 EMOTIONS ATTITUDE BEHAVIOUR

NEGATIVE IMPACT - ALL STUDIES

COACH CLIENT

IMPACT

COACH CLIENT INDIRECT

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 9 – Translation Synthesis III

Category Dynamics Translation – Client internal processes

Purpose: to identify &explain how client internal processes translate into coaching relationship

Adjusting the mirror: Strategies for coaching executives with narcissistic personality features (Hughes J.L., 2003)

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 9 – Translation Synthesis III - Illustration

Figure 1: Interrelationships of categories

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 10 – Translation Synthesis IV

Client-Coach External Process – Impact on Coaching Outcome

Purpose: to make meaning of how categories influence coaching outcome in the WA

Keeping our heads above water: Applying Kegan's orders of consciousness theory in coaching (Pinkavova, E. 2010)

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 10 – Translation Synthesis IV - Illustration

Table 8: Client attributes & Impact on coaching outcome

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Step 10 – Translation Synthesis IV - Conclusion

Figure 2: Client attributes & Impact on coaching outcome

Direct impact / strong influencer Indirect impact / weak influencer Direct impact / strong influencer Indirect impact / weak influencer

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Fact VS Fiction – What‘s my contribution?

‘Quantitative inquiry [in this sense] cannot be true, it can only be correct, whereas qualitative research when done properly can be true’ (Stenner, 1998. quoting Heidegger) Stegner W. claims: “For a work of fiction to be great it has to be true”. It does not, however, have to be correct (if it were, it would not be fiction but fact). „Anthropology, sociology, education, cultural studies, media research and many other fields have long considered qualitative methods as integral to the development of new

• knowledge. Psychology has been legging behind.“ (Camic, et al., 2003)

Maracek J. in a chapter ‚Dancing Through Minefields, argues that we should give up the phrase “the scientific method”, since it is false that science is unitary (and it is likewise false that it ought to be so). Qualitative research does not develop a correct map of the world but rather a useful one (Strean, W.B. (1998).

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Conclusions

Onotologically Epistemologically Rhetorical structure Axiology Methodologically Analogical reasoning Gap filling vs. Conceptual leaping We need a different model of how to define client attributes -> wild west of terminology when it comes to client attributes - I must do the definition -> conceptualisation of client attributes on the basis of my model -> we cannot explain why we got so many failures: ‘Andrew Day – how do we explain why certain coaches can work with certain clients and fail with others?’/ we do not focus on distal outcomes / include coaching definition Data in the synthesis about client attributes and their impact on coaching outcome show what the initial clustering and first-order theme analysis reveals (behaviour has been studied extensively and impacts behaviour as outcome) Difficult to find the fitting literature Even if change is recognizable and quantifiable, we only make suppositions about how effectively change occurs (Thompson & Hunt, 1996; Ratiju, 2012).

14-15 June 2017 Greenwich University, London

7th International Coaching & Mentoring Research Conference

Putting Research at the heart of practice 14 - 15 June 2017 Greenwich University, London

Bibliography

Savin-Baden & Major (2010) New approaches to qualitative research Hoon (2013)