REVIEW: RESEARCH QUESTION 1. WHY DO PATIENTS SEEK OSTEOPATHIC - - PDF document

1 AGENDA

• REVIEW OF MATERIAL
• HYPOTHESIS/RESEARCH QUESTION
• P-VALUE
• STUDY DESIGNS
• VARIABLES AND MEASUREMENTS
• SAMPLE SIZE DETERMINATION
• CHOOSING MEASUREMENT INSTRUMENT/TOOL
• QUASI-EXPERIMENTAL DESIGNS
• RELIABILITY STUDIES

REVIEW: RESEARCH QUESTION

• 1. WHY DO PATIENTS SEEK OSTEOPATHIC TREATMENT?
• 2. DOES OSTEOPATHIC INTERVENTION X EFFECTIVELY REDUCE PATIENTS’ PAIN AFTER 5 SESSIONS?
• 3. IS THERE AN ASSOCIATION BETWEEN THE AGE OF PARTICIPANTS AND THE NUMBER OF OSTEOPATHIC

SESSIONS ATTENDED?

• 4. IS THERE A DIFFERENCE BETWEEN OSTEOPATIC INTERVENTION X AND INTERVENTION Y IN INCREASING

THE PARTICIPANTS’ QUALITY OF LIFE?

• 5. HOW RELIABLE IS A PARTICULAR TECHNIQUE IN DIFFERENTIATING EMPTY VS FILLED BLADDER?
• 6. IS THERE A CONSENSUS IN PUBLISHED STUDIES REGARDING THE EFFECTIVENESS OF INTERVENTION X?

2

REVIEW: HYPOTHESIS

Hypothesis = Research Question + Measurement Tool + “p ≤ 0.05”

Examples of Hypothesis formulation:

1.

Osteopathic treatment will significantly reduce the redness associated with acne as measured by infra-red photography, p ≤ 0.05.

2.

Five sessions of osteopathic intervention X will result in significant reduction in patients’ pain as measured by Visual Analog Scale, p ≤ 0.05.

3.

Three trained osteopathy students at the end of their curriculum could achieve at least moderate agreement on osteopathic sacral palpatory diagnostic tests, evaluated using Fleiss Κ (Kappa) statistics, p ≤ 0.05.

4.

Osteopathic treatment X is more effective than osteopathic intervention Y in increasing the participants’ quality of life as measured by WHOQOL questionnaire, p ≤ 0.05.

REVIEW: HYPOTHESES

Null Hypothesis (Ho):

Osteopathic treatment will NOT significantly reduce the redness associated with acne as measured by infra-red photography, p > 0.05.

Alternative Hypothesis (Ha):

Osteopathic treatment will significantly reduce the redness associated with acne as measured by infra-red photography, p ≤ 0.05.

0.05 p-value p > 0.05 Failed to reject the null hypothesis. There is insufficient evidence to conclude that

• steopathic treatment is effective.

Reject null and accept an alternative hypothesis. There is statistically significant reduction of acne skin redness as a result of osteopathic treatment. p < 0.05

UNDERSTANDING RESEARCH ARTICLES

_____________________________________

Source: A.M. Cuccia et al. Osteopathic manual therapy versus conventional conservative therapy in the treatment of temporomandibular disorders: A randomized controlled trial. Journal of Bodywork & Movement Therapies (2010) 14, 179-184 https://pdfs.semanticscholar.org/849d/3c122af15a27b3dc59de93a76dde196e52a4.pdf

3

REVIEW: STUDY DESIGNS

1.

Experimental and quasi-experimental research

• 2. Reliability, validity, and palpation studies
• 3. Technique studies
• 4. Case series studies
• 5. Fundamental studies - investigative but

non-experimental

• 6. Qualitative studies

Acceptable Forms of Student Research:

REVIEW: EXPERIMENTAL (RCT)

RESEARCH QUESTION: IS THERE A DIFFERENCE BETWEEN OSTEOPATHIC INTERVENTION X AND INTERVENTION Y IN INCREASING THE PARTICIPANTS’ QUALITY OF LIFE? post

Pre-intervention quality of life Post-intervention X quality of life

R

pre

Post-intervention Y quality of life

post

Intervention X Intervention Y

R O X1 O R O X2 O

REVIEW: QUASI-EXPERIMENTAL (CROSSOVER)

post

Pre-intervention quality of life Post-intervention Y quality of life

R

pre

Post-intervention X quality of life

post post

Post-intervention X quality of life Post-intervention Y quality of life

post

Intervention X Intervention X Intervention Y Intervention Y

R O X1 O

washout

O X2 O R O X2 O

washout

O X1 O

4

REVIEW: QUASI-EXPERIMENTAL (WITHIN SUBJECT)

RESEARCH QUESTION: DOES OSTEOPATHIC INTERVENTION X EFFECTIVELY REDUCE PATIENTS’ PAIN AFTER 5 SESSIONS?

pre post

Pre-intervention VAS pain score Post-intervention VAS pain score

Intervention X

O X O

REVIEW: RELIABILITY STUDY

RESEARCH QUESTION: HOW RELIABLE IS A PARTICULAR TECHNIQUE IN DIFFERENTIATING EMPTY VS FILLED BLADDER? Osteopath Practitioners (raters)

Osteopathic Technique

%

% correctly identified bladders Inter-rater reliability Intra-rater reliability

REVIEW: VARIABLES

Variable is a thing that changes in experiment. A variable is any factor, trait, or condition that can exist in differing amounts or types. Independent Variable –the variable that is changed or controlled in a scientific experiment. Usually the Treatment: technique, global or regional osteopathic intervention vs control. Dependent Variable –the outcome of interest, what we are hoping to change or alter. Variable type: Numerical(Age) or Categorical(Gender, Group)

5

REVIEW: MEASUREMENTS

Measurement is a variable that is being assessed (quantified / measured) using a particular technique, tool or instrument.

SAMPLE SIZE DETERMINATION

Level of significance (Type I error) – chance of finding effect if it does not exist Effect size – expected amount of change in dependent variable (treatment effect) Statistical power – credibility of the test, chance of finding effect if it does exist Level of significance (α)

0.05 for most social/business studies

Effect size (d)

estimated from large studies

One- or two- tailed

depends on hypothesis wording

Type (family) of statistical test

depends on data being collected

Desired power (1-β)

typically 80% is considered acceptable

Sample size (n)

Rule-of-thumb

depends on analysis being conducted

http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/download-and-register

HOW DO I KNOW EFFECT SIZE?

n Previous (published) studies with similar research question

n similar Population, Intervention, Outcome n look for numbers to quantify effect size (mean, standard deviation, %)

n Pilot study conducted with small group of participants (n = 12) n Based on practical significance

n Clinically important change, Minimal Important Difference (MID)

n Assume to be medium effect (Cohen’s d = 0.5)

Approaches to determine effect size:

6

FINDING PUBLISHED STUDIES

• GOOGLE SEARCH (START WITH GOOGLE SCHOLAR)
• PREVIOUS YEARS CCO STUDENTS’ THESIS
• THE JOURNAL OF THE AMERICAN OSTEOPATHIC ASSOCIATION

HTTP://JAOA.ORG/

• INTERNATIONAL JOURNAL OF OSTEOPATHIC MEDICINE

HTTP://WWW.JOURNALOFOSTEOPATHICMEDICINE.COM/

• THE JOURNAL OF ALTERNATIVE AND COMPLEMENTARY MEDICINE

HTTPS://WWW.LIEBERTPUB.COM/LOI/ACM

• INTERNATIONAL JOURNAL OF OCCUPATIONAL MEDICINE AND ENVIRONMENTAL

HEALTH

HTTP://IJOMEH.EU/

• INTERNATIONAL JOURNAL OF PHYSIOTHERAPY

HTTPS://WWW.IJPHY.ORG/

SAMPLE SIZE – RULES-OF-THUMB

Final notes on sample size:

n

For multiple groups, aim for balanced design (equal number of participants in each group).

n

Account for non-response rate during recruitment.

n

Account for attrition/drop-out rate during the study. Experimental: Minimum 12 Quasi-Experimental: Minimum 16 Reliability Studies: Minimum 40 Technique Studies: Minimum 24 Case Series: Minimum 6

SAMPLE SIZE DETERMINATION EXAMPLE

n crossover design n “increase” → one-tail test n literature search → Buscemi et al.

(2015) study reported effect size

n G*Power calculation → 24 subjects n 10% dropout rate → 27 subj to recruit

Research Question:

A global osteopathic treatment will increase urinary pH levels, as measured using urine test strips.

Reference: Buscemi, A., Carbone, J., Tacchi, M., Buttafuoco, S., Rapisarda, A., Perciavalle, V., & Coco, M. (2015). Changes of urine pH after the compression of the fourth ventricle. Medicina, Ricerche, Scienza della vita, Retrieved from http://www.scienza-ricerche.it/

7

STUDENTS’ RESEARCH

n Proposal (PICO statement)

¨ P = patient/problem (research question) ¨ I = intervention (experiment design) ¨ C = comparison (control) ¨ O = outcome (validated instrument to measure)

MEASUREMENT INSTRUMENT/TOOL

Ensure sufficient level of accuracy/precision and range

Examples:

Strain → Strain gauge Angle → Goniometer (manual or digital) Acceleration (3-axis) → Accelerometer (Fitbit or less expensive alternatives) Ground reaction force → Force platform/plate Object thickness → Caliper Time interval → Stopwatch (iPhone has one built-in) Weight → Scale Clinical measurements (pulse, blood pressure, temperature, respiratory rate)

MEASUREMENT INSTRUMENT/TOOL

Good instrument is both Reliable and Valid (validated).

Examples:

Tinnitus symptoms → Tinnitus Handicap Inventory (THI) Quality of life → Quality of Life Scale (QOLS) questionnaire Pain → Visual Analog Scale (VAS) Feet functioning → Foot and Ankle Survey (FAOS) or Foot Functioning Index (FFI)

8

INSTRUMENT RELIABILITY AND VALIDITY

Not valid and not reliable Reliable, but not valid Reliable and valid Valid, but not reliable

INSTRUMENT RELIABILITY AND VALIDITY

Reliability:

Internal consistency reliability (Cronbach’s α > 0.8) Test-retest reliability correlation (r > 0.7) Inter-rater (inter-observer) reliability (Kappa > 0.4 or interclass correlation coefficient > 0.7)

Poor Slight Moderate Substantial Fair Almost perfect < 0 0.00-0.20 0.41-0.60 0.61-0.80 0.21-0.40 0.81-1.00

Validity:

Correlation with “gold standard” instrument (r > 0.7) Overall accuracy with respect to actual state (diagnostic accuracy, sensitivity, specificity, PPV, NPV)

QUASI-EXPERIMENTAL (CROSSOVER)

R O O

washout

O X O R O X O

washout

O O

post

Pre-intervention quality of life Post-intervention Y quality of life

R

pre

Post-intervention X quality of life

post post

Post-intervention X quality of life Post-intervention Y quality of life

post

Sham Sham Intervention Intervention

9

QUASI-EXPERIMENTAL (WITHIN SUBJECT)

pre post

Pre-intervention VAS pain score Post-intervention VAS pain score

Intervention X

O X O O1 O2 O3 X O4 O5 O6 O1 X1 O2 O3 X2 O4 O5 X3 O6

RELIABILITY/VALIDITY/PALPATION STUDIES

n Practical aspects

¨ Live patients or objects (models) ¨ Repeated trials to make a diagnosis

n Benefits

¨ Relative simplicity in design ¨ Contribution to osteopathic profession ¨ Improving manual skills ¨ Osteopathic students as study participants

RELIABILITY STUDY EXAMPLE

Osteopath Practitioners (raters)

Assessment or Diagnostic Tool Κ

Inter-rater reliability Intra-rater reliability

...

Poor Slight Moderate Substantial Fair Almost perfect < 0 0.00-0.20 0.41-0.60 0.61-0.80 0.21-0.40 0.81-1.00

Categorical outcomes: Cohen’s Kappa (2 raters), Fleiss Kappa (3+ raters) Numerical outcomes: Cronbach’s α, Interclass Correlation Coefficient

Example:

Consorti et al. (2017) study explored inter-rater reliability of Osteopathic Sacral Palpatory Diagnostic Test using 52 patients and 3 trained osteopathy students (raters). Fleiss Kappa ranges between 0.06 to 0.34 (Table 3).

10

VALIDITY STUDY EXAMPLE

Categorical outcomes: Overall accuracy, sensitivity, specificity, NPV, PPV Numerical outcomes: Correlation coefficient, mean absolute error

Examples:

• Assessing accuracy of palpation technique to differentiate between empty and filled bladders
• Using wax blocks to assess participants’ skills in differentiating two heights (Christopher Reiach study)
• Evaluating palpation technique to determine knee problems (validate through radiographs)
• Palpation sensitivity study using a hydrodynamic model (Monica Noy project)

Osteopath Practitioners (raters)

Osteopathic Palpation Technique %

Overall accuracy Sensitivity, NPV, Specificity, PPV

...

True State (X-Ray, Ultrasound, patient report)

PALPATION STUDY EXAMPLE

Intervention examples:

• Feedback when using wax blocks
• Take home models to self-practice palpation skills
• Workshops with group practice sessions

%

Pre-intervention accuracy

Intervention to improve palpation skills Validity study

%

Post-intervention accuracy

Validity study

TRAINING STATION FOR SURGEONS

_____________________________________

Presented with the permission of Dr. Ilay Habaz and Dr. Eran Shlomovitz (University Health Network)

11

STUDENTS’ RESEARCH – PARTICIPANTS

Recruitment of study participants

n Specialized clinics n Osteopathic practices n Social media (Facebook, LinkedIn, Twitter)

¨ Post message on your own page ¨ Ask friends to re-post your message on their pages ¨ Join relevant Facebook group ¨ Paid advertisement

n Kijiji and other online posting sites

QUESTIONS? COMMENTS? THOUGHTS?

• ANTON SVENDROVSKI
• 647-833-3359
• WWW.STATSHELP.CA
• INFO@STATSHELP.CA

Research Proposals | Sample Size Calculation | Methodology/Design | Statistical Data Analysis | Interpretation