The Meta-Narrative Review Systematic Reviewing Across Different Paradigms Henry W. W. Potts Centre for Health Informatics & Multiprofessional Education (CHIME), Institute of Epidemiology & Health Care, UCL With thanks to Trish Greenhalgh, Geoff Wong & others
• Systematic reviewing has evolved over time • Meta-analysis for quantitative outcomes • Some degree of methodological heterogeneity can be handled with sub-group analyses • Various ‘mixed methods’ approaches developed to combine qualitative and quantitative studies From Taylor & Potts (2008), cancer detection rate Eur J Cancer 44 (6):798-807
Not just heterogeneity, not just mixed methods, but incommensurability
Problems of heterogeneity multiply with more complex questions, with multiple outcomes, varying systems and different methodologies – different paradigms Various approaches developed to review broad methods… Moran-Ellis et al. ( Qual Res 2006; 6 (1):45-59): “Researchers who advocate the use of multiple methods often write interchangeably about ‘integrating’, ‘combining’ and ‘mixing’ methods […] [This] obscures the difference between (a) the processes by which methods (or data) are brought into relationship with each other (combined, integrated, mixed) and (b) the claims made for the epistemological status of the resulting knowledge.” Yardley & Bishop (In The SAGE Handbook of Qualitative Research in Psychology , 2007: pp. 352-67): ‘Composite analysis’: retain integrity of each method – integrate findings rather than ‘mixing methods’ Noblit & Hare ( Meta-ethnography: Synthesising Qualitative Studies , 1988): Distinction between integrative and interpretive reviews Lewis & Grimes ( Acad Manage Rev 1999; 24 :672-90): Meta-triangulation: building theory from multiple paradigms
Meta-narrative review – key citations 1 st : Greenhalgh, Robert, Macfarlane et al. , Milbank Q 2004; 82 :581-629 / expanded as Diffusion of Innovations in Health Service Organisations: A Systematic Literature Review , Blackwell BMJ Books Methods paper: Greenhalgh, Robert, Macfarlane et al. , Soc Sci Med 2005; 61 :417-30 2 nd (ish): Greenhalgh, Potts, Wong et al. , Milbank Q 2009; 87: 729-88. Publication standards: Wong, Greenhalgh, Westhorp et al. , BMC Med 2013; 11 :20
Meta-narrative review – key principles Use a historical and philosophical perspective as a pragmatic way of making sense of a diverse literature • Pragmatism • Pluralism • Historicity • Contestation • Peer review
Key questions (from Kuhn, “The structure of scientific revolutions”) • What research teams have researched this area? • How did they CONCEPTUALISE the problem? • What THEORIES did they use to link problem with potential causes and impacts • What METHODS did they define as ‘rigorous’ and ‘valid’? Application more post-Kuhnian than Kuhnian
Open-ended question Explore the literature Research tradition B Research tradition C Research tradition A Theoretical Quality Theoretical Quality Theoretical Quality basis criteria basis criteria basis criteria Evaluate, summarise Evaluate, summarise Evaluate, summarise Meta-narrative map of underpinning traditions Meta-narrative review (how to get started)
Research Disciplinary Definition & General format EPR EPR user Context tradition roots scope of research conceptualised conceptualised conceptualised question as... as... as... (Evidence- Study of What is impact Container for Rational Potential Health based) storage, of technology X information decision-maker confounder information medicine, computation (EPR, DSS, about patient; whose which can be systems computer & transmission etc. ) on tool for cognitive ability ‘controlled science of clinical data. process Y ( e.g. aggregating sets limits to for’ if right study Focus often on clinician clinical data what can be design used benefits of performance) for secondary achieved EPRs and and outcome uses without how to achieve Z? computers them (Evidence- Study of How can we Innovation that, ‘Resistant’ External milieu Change based) achieving improve if implemented agent who of interacting manage- medicine, organisation- delivery of widely and must be trained variables that ment social level change in healthcare and consistently, and serve as barriers psychology, Healthcare sustain will improve incentivised to or facilitators to (within management improvement? process and adopt new change efforts health outcome of technologies services care and ways of working research) Business Study of how What factors Unwelcome Potential External milieu Information studies, organisations (independent change, adopter who of interacting systems psychology, do or do nor variables) likely to be may engage variables that (positivist) computer adopt & account for resisted, and with or resist mediate or science assimilate success or which may change; moderate the information failure fit poorly with member of relationship systems (dependent organisational group whose between input variable) of structures & power base and output information systems may be variables system X in enhanced or organisation Y? threatened
Research Disciplinary Definition & General format EPR EPR user Context tradition roots scope of research conceptualised conceptualised conceptualised question as... as... as... Management, Study of how What meanings Socio-technical Stakeholder Scene & setting Information sociology, organisational does change that whose ‘framing’ for an unfolding systems social members make information holds different of the EPR is story; webs of (interpret- psychology, sense of system X hold meanings for crucial to its meaning in anthropology information for members of different assimilation. which ivist) systems & organization Y? individuals and Agent whose organisational thereby How to achieve groups creativity can actors are assimilate them accommoda- be drawn upon suspended tion between in this effort different views? Organizational Study of how What is the Itinerary and Knowledgeable Generated & Information sociology, social relationship organiser creative agent regenerated systems social structures between whose physical for whom social through interplay (technology psychology, recursively organisational & technical structures both of action & philosophy shape & are actors, properties create structure. Does -in- shaped by technology X, structure & possibilities & not study practice) human agency, and the support limit the ‘technologies’ & & role of organisation – collaborative possible ‘contexts’ technology in and how does clinical separately but this this change work technologies-in- over time? use Computer Study of how How can Contextualized Agent who External milieu Computer science, groups of technologies artefact works to local or emergent supported software people work support the goals in property of cooperative engineering, collaboratively, work of multiple collaboration action psychology, supported by interacting with others & (constituted by work sociology information people? creatively & inextricable technology overcomes from an activity limitations of involving people formal tools & technologies)
Research Disciplinary Definition & General format EPR EPR user Context tradition roots scope of research conceptualised conceptualised conceptualised question as... as... as... Sociology, Study of What social Implicated in Constrained by Social & material Critical philosophy relationship structures micro & macro dominant social conditions into sociology between & power power Structures, which the people & social imbalances are dynamics which may be unequal social order, & role of embedded in (because of link built into order is technologies in technology X, & between technologies by inscribed; this what impact knowledge designers more or less does this have & power) stable structure on social roles/ of macro social relationships? relations Philosophy, Study of How has Actor in a Actor in a EPR & its Empirical sociology, sociotechnical network, network network context philosophy linguistics networks: with its various together form (actor considers how relationships, the network; the relationships & work practices one cannot be network power shift & risks, studied without case within network changed as a the other studies ) result of technology X? Systems & Systems What role does Component of Component of Complex, Systems management perspective the EPR play complex socio- complex socio- changing approaches research, within a technical technical environment drawing on complex system whose system whose cognitive healthcare features & features & psychology, system? properties may properties may CSCW & ANT come together come together in in unpredictable unpredictable ways ways
Synthesis phase Highlight similarities and differences in the findings from different traditions Contestation between the disciplines is data (and leads to higher order constructs) Offer conclusions of the general format “in circumstances such as X, don’t forget to think about Y”
Recommend
More recommend
