Graziano and Raulin Research Methods: Chapter 8 Ideas lead to - - PowerPoint PPT Presentation

Graziano and Raulin Research Methods: Chapter 8

Ideas lead to

• bservations

library research

Statement of problem Problem statements

become research hypotheses when constructs are

• perationalized

“ “FINER FINER” ”

F Feasible I Interesting N Novel E Ethical R Relevant

Actually testing three sets of hypotheses

The null hypothesis The confounding variable hypotheses The causal hypothesis

Accept causal hypothesis only if you

reject null hypothesis (statistical analysis) rule out each potential confounding variable hypothesis

(based on appropriate controls)

Correlation (also called covariation)

Relationship found between variables

Time order

Cause must occur before result

Nonspuriousness

Alternative explanations must

be eliminated from possibility

Experiments are intended to

reduce or rule out alternative explanations and confounding variables

The PICO format: P Population I Intervention or Interest area C Comparison intervention or status O Outcome

“What is the usefulness or accuracy of the current 1-10 pain scale assessment in treating a patient’s pain, and what are other options that may prove more useful?” Does a 10 point pain Visual Analog Scale (____, ____) accurately assess pain in the first day postop abdominal total hysterectomy patient when compared with the Faces Pain Scale (Pasaro, 1997)?

Statistical Validity – carrying out the

actual statistical analysis properly

Construct Validity refer most often to

a characteristic of an instrument but also to the whole study

External Validity refers to the

generalizability of study findings

Internal Validity refers to a

characteristic of a study’s design

Are the statistical tests

accurate?

Threatened by

Unreliable measures Violations of statistical assumptions How do we detect these problems???

Strengthened by

Using well validated measures Having approximately equal sample sizes in each

group)

Is our theory the best explanation for the results? Threatened by

Any alternative explanation for the results HOW do we locate these alternative explanations?

Strengthened by

Using well-validated constructs to build the theoretical

predictions for the study

Do the results apply to the broader population? Threatened by

Unrepresentative samples Generalizing beyond the limits of the sample HOW do we know when this problem is present???

Strengthened by

Gathering a representative sample (if possible) Clearly describing sample, so that other researchers will

know the limits of generalization

Is the independent variable responsible for the

• bserved changes in the dependent variable?

Threatened by

Confounding variables HOW do we detect the presence of

confounding variables????

Strengthened by

Adding adequate controls to reduce or eliminate

confounding

Confounding and internal validity

Many sources for confounding (covered next) With proper controls, confounding can be virtually

eliminated (see Chapter 9) Confounding and construct validity

Make sure that you have considered alternative

theoretical explanations for the anticipated phenomenon

HOW????

Single-group, pretest-posttest design

compares pre-treatment and post-treatment scores to determine improvement

Fails to control most sources of confounding

• Historical

Historical events may occur during the course of the experiment.

Remember Pygmalion effect & its story

• Maturation

Maturation of the subjects.

• Testing

Testing and retesting can influence awareness of variables or behavior

Learn Hawthorne effect & its story.

• Instrumentation

Instrumentation – measurement methods or procedures may not be equivalent

• Statistical regression

Statistical regression of subjects starting out in extreme positions.

• Selection

Selection biases (we will see several types)

• Experimental mortality

Experimental mortality (a.k.a. attrition) (a.k.a. attrition) – subjects drop out of the study before it's finished.

• Sequence effects

Sequence effects – Performance on one measure is related to previous experience with other measures. Outcome depends on the sequence of measures.

• Demoralization

Demoralization subjects in control group find out, lose interest in study, stop trying

• Diffusion

Diffusion of treatment (those who get the experimental stimulus spread it to controls)

• Rivalry

Rivalry (controls change behavior to try to beat the experimental group)

• Equalization

Equalization of treatment (researcher compensates controls for not getting treatment)

Participants are not passive

They try to understand the study to help them to know

what they “should do” (termed subject effects subject effects)

Respond to subtle cues about what is expected (termed

demand characteristics demand characteristics)

• Placebo effect

Placebo effect: treatment effect due to expectations that the treatment will work

Based on the expectations of the researcher Can affect the outcome of studies if not controlled May be due to the experimenter providing

demand characteristics to the participant

Not the same as scientific fraud

(which is deliberate)

Three closely-tied concepts

• Validity

Validity

The accuracy of the study or procedure Increased by using appropriate control procedures

The more controls we employ, the higher the

level of constraint

Controls may increase some types of validity while, by

their unnatural aspect, decreasing other types of validity.

Risk is balanced by reward

A poorly designed study will provide no useful

information; therefore, any risk would be unacceptable Informed Consent

Virtually guarantees that you will have confounding

due to selection because some people will refuse to participate

A small price to pay to maintain ethical standards

Start by building a research hypothesis Testing the research hypothesis is actually testing

three hypotheses

(1) null; (2) confounding-variable; (3) causal

Several types of validity Many potential confounding variables Subject and experimenter effects can also affect

the outcome of the study