What do you really do in a literature review? David N. Lopez, PhD - - PowerPoint PPT Presentation

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

What do you really do in a literature review? Studying the Comparative Politics of Public Education

David N. Lopez, PhD Student

Department of Political Science University of Washington QUAL Initiative Winter Series 2016

January 14, 2016

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Outline

◮ Getting started ◮ A Quick Note on Software ◮ I. The Working Bibliography ◮ II. Begin a New Project ◮ III. Create a Coding System ◮ IV: The Annotated Bibliography ◮ V. Final Comments

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Getting started with the Lit Review

What is a literature review? “A literature review is designed to identify related research, to set the current research project within a conceptual and theoretical context.” (Trochim 2006)

◮ Humbling (e.g. nothing new under the sun) ◮ Focused on scholarly research ◮ Should be done early (and well) ◮ Will help identify feasible approaches to question

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Getting started with the Lit Review

Given puzzle and questions of concern: What is wrong (and right) with the extant literature?

◮ Theoretically:

◮ Relax or alter assumptions entirely? ◮ Weak logic in extant approaches? ◮ Sound logic, but mechanisms wrong? ◮ Refine scope conditions?

◮ Empirically:

◮ Lack of rigorous and sound analysis? ◮ Causal mechanisms tested well, but not effects? ◮ Selection or omitted variable bias? ◮ Adequate qualitative or quantitative evidence?

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Getting started with the Lit Review

Two approaches:

◮ Soak-and-Poke

◮ “Have question; will Google”

◮ Systematized Review

◮ ...Not so fast

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Soak-and-Poke Review

Advantages:

◮ Let the literature speak to you ◮ Can lead to interesting insights ◮ Minimally labor-intensive on the front-end

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Soak-and-Poke Review

Potential pitfalls:

◮ Can lead to Wikipedia-like spiral into

click-and-download of articles

◮ Very labor-intensive on the back-end ◮ Re-reading and skimming for important annotations and

quotes

◮ Can lose sight of research priorities and waste time

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Systematized Review

Advantages and potential pitfalls:

◮ Need to have a general idea of where to start review

beforehand

◮ Discipline + flexibility is key ◮ Not as fun as a Google Scholar-spiral ◮ Labor intensive on the front-end

◮ Need to map out journey first

◮ Minimally labor-intensive on back-end ◮ Might need to “tech-up” a bit and learn how to

integrate new tools

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Getting started with the Lit Review

Happy medium can be fruitful - lead with a system to increase your likelihood of finding what you need

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

A Quick Note on Software

Useful tools for Lit Review and more

◮ Zotero - for reference management (syncs with some

mobile apps)

◮ ATLAS.ti - for qualitative data analysis and literature

review (collection, organization, and coding)

◮ Evernote - for memos, outlines, and quick notes (syncs

across devices)

◮ Adobe Acrobat Pro or Prizmo (Mac) - for cleaning and

creating searchable PDFs

◮ Excel - for grabbing relevant descriptive statistics ◮ Scrivener - for writing papers with added workflow

functionality

◮ L AT

EX- typesetting language for publication-ready drafts with graphs, figures, or equations

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

The Working Bibliography

Step 1: Organize a working bibliography

◮ Download the Zotero plug-in for Firefox or other

citation manager

◮ Create master working bibliography document in Word,

Evernote, or other preferred word processing format

◮ Initial map to situate yourself in “the literature” ◮ It will change, but start with a manageable reading list ◮ First question to ask yourself: What are the key

readings that I can’t avoid addressing in my research?

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

The Working Bibliography

Suggestions on organizing readings:

◮ By broad themes in main field(s) - situate project within

key debates and themes in relevant disciplines

◮ By related sub-themes - research related to broad

themes that more directly speaks to question

◮ By specialized scholarship - research from specialized

journals focused

◮ Other ideas:

◮ By causal mechanisms (microfoundational, structural,

ideational)

◮ By units of analysis (households, organizations,

country-year)

◮ Sub-divide and tag by theoretical and/or empirical

• rientation

◮ Tip: Align working bibliography to folders in reference

manager software

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Example: The Comparative Politics of Public Education

Central Puzzle: Why is “education reform” more politically contentious in some countries and not others?

◮ Broad themes: Domination, state-building, nationalism,

and democracy

◮ Sub-themes: Comparative politics, institutions, welfare

states, redistribution

◮ Specialized scholarship: comparative education,

sociology of education, and education economics

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Starting a New Project in ATLAS.ti

General suggestions:

◮ Keep all articles, readings, etc. in one project ◮ File names consistent with reference software (i.e.

Zotero “name - year - short title”)

◮ Paste bibliographic citation in memo section after

importing document

◮ Searchable PDFs ideal (OCR scan of printed material) ◮ Keep text/audio/visual evidence for qualitative analysis

in separate project from literature review

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Importing and Grouping Documents

Key point: Mirror organization of working bibliography

◮ Create document “groups” that match threads of

interest

◮ After importing, sort documents into respective groups ◮ Document can be a part of more than one group!

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Example of document grouping

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Coding for Literature Review

Key takeaway: Create a coding system that works for you!

◮ Code generously, systematically, and consistently ◮ My approach (after trail and error):

◮ First, create “code groups” to organize thoughts and

coding strategy

◮ Second, make detailed notes of code groups to refer

back to

◮ Third, revise code and code group labels as needed

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Example of code grouping

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Example of coding

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Mapping the literature

Grouping, coding, reading = thinking critically about the pieces of your puzzle

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Generate an Annotated Bibliography

The memo function in ATLAS.ti is your friend

◮ Remember: paste bibliographic citation in document

memo

◮ Create annotated bibliography format that fits your

needs

◮ One suggestion:

◮ Discipline (political science, sociology, economics) ◮ Subfield (political economy, structuralism, economic

history)

◮ Category in bibliography (Broad themes, sub-themes,

specialized lit)

◮ Main thesis (abstract-length summary of central

argument)

◮ Contribution to research question

(theory/empirics/assumptions)

◮ Limitations (why it falls short)

What do you really do in a literature review? Studying the Comparative Politics of Public Education David N. Lopez, PhD Student Getting started with a Lit Review A Quick Note on Software I: The Working Bibliography II: Using ATLAS.ti for Literature Review III: Coding for Literature Review IV: The Annotated Bibliography V: Final Comments

Final Comments

There is no one way to conduct a literature review:

◮ ATLAS.ti and other software makes it easier to

re-interrogate literature regularly

◮ Mix and match between digital and analog strategies as

needed

◮ Keep question in mind but keep eyes open for new

puzzles during review