Basic Experimental Design Basic Concepts in Experimental Design - - PowerPoint PPT Presentation

▶
basic experimental design
SMART_READER_LITE
LIVE PREVIEW

Basic Experimental Design Basic Concepts in Experimental Design - - PowerPoint PPT Presentation

Aug 31, 2022 172 likes •481 views

Basic Experimental Design Basic Concepts in Experimental Design Prof. Dr. Luc Duchateau Ghent University 2018 UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 1 / 26 Basic concepts in experimental design

slide-1
SLIDE 1 UGent

STATS

VM

Basic Experimental Design

Basic Concepts in Experimental Design

  • Prof. Dr. Luc Duchateau

Ghent University

2018

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 1 / 26

slide-2
SLIDE 2 UGent

STATS

VM

Basic concepts in experimental design Overview

Overview

What do we want to test?

Specifying statistical hypotheses

What tools do we use?

Random assignment Blocking Blinding

What are the building blocks of an experiment?

Experimental unit versus observational unit Replication versus repeated measures

How much resources do we need?

Power analysis

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 2 / 26

slide-3
SLIDE 3 UGent

STATS

VM

Basic concepts in experimental design Specifying statistical hypotheses

Specifying statistical hypotheses Scientific hypothesis Statistical hypothesis

TRANSLATION

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 3 / 26

slide-4
SLIDE 4 UGent

STATS

VM

Basic concepts in experimental design Specifying statistical hypotheses

Example: Mastitis trial in dairy cows

104 CFU Low 106 CFU High E.Coli infusion MASTITIS Observe somatic cell count (SCC) every 3 hours in 24 hours timespan Observe milk reduction 24 hours after infusion

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 4 / 26

slide-5
SLIDE 5 UGent

STATS

VM

Basic concepts in experimental design Specifying statistical hypotheses

Mastitis: Scientific question Evaluate the effect of the infusion dose on SCC!

→ FAR TOO GENERAL → NOT TESTABLE → TRANSLATION REQUIRED

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 5 / 26

slide-6
SLIDE 6 UGent

STATS

VM

Basic concepts in experimental design Specifying statistical hypotheses

Mastitis: (Rather specific) statistical hypotheses

→ SCC is on average higher in high dose group H0 : µH,Av = µL,Av versus Ha : µH,Av > µL,Av → Maximum SCC is higher in high dose group H0 : µH,M = µL,M versus Ha : µH,M > µL,M → SCC increases faster in high dose group H0 : βH = βL versus Ha : βH > βL → time to attain a certain SCC treshold value is less in high dose group H0 : τH = τL versus Ha : τH < τL

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 6 / 26

slide-7
SLIDE 7 UGent

STATS

VM

Basic concepts in experimental design Specifying statistical hypotheses

Mastitis: More general hypotheses

→ Compare SCC at each time point BEWARE of multiple testing! → Global hypotheses with dose, time and their interaction in the model

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 7 / 26

slide-8
SLIDE 8 UGent

STATS

VM

Basic concepts in experimental design Specifying statistical hypotheses

Mastitis: one- versus two-sided statistical hypotheses

Compare two species, Escherichia coli and Staphylococcus aureus for their effect on milk production reduction ρ → Is there a difference between the two species? H0 : ρE = ρS versus Ha : ρE = ρS → Is the reduction larger with Escherichia coli? H0 : ρE = ρS versus Ha : ρE > ρS → Are the two species equivalent? H0 :| ρE − ρS |= ∆ versus Ha :| ρE − ρS |< ∆

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 8 / 26

slide-9
SLIDE 9 UGent

STATS

VM

Basic concepts in experimental design The tools

Randomisation

All variables with possible effect on response are distributed randomly

  • ver treatment groups

→ guard against unknown confounders In experiments, treatments are randomly assigned as compared to

  • bservational studies

To show causal relationship, randomisation is required! Observational studies can only show associations

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 9 / 26

slide-10
SLIDE 10 UGent

STATS

VM

Basic concepts in experimental design The tools

Blocking

Experimental units can be heterogeneous Group experimental units in blocks of more homogeneous units Optimal is when all treatments appear in a block ⇒ Compare treatments within block

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 10 / 26

slide-11
SLIDE 11 UGent

STATS

VM

Basic concepts in experimental design The tools

Example blocking

Difference in milk reduction with high and low CFU dose of E. coli Herds are the blocks; cows are more similar within herd Herd 1 Herd 2 Herd 3 L H L H L H 0.9 7.4 2.3 6.6 3.4 8.8 2.0 6.8 2.7 7.1 2.7 8.3 2.0 7.1 1.3 6.2 2.9 7.9 2.2 6.7 1.6 7.8 3.0 8.2 2.0 7.9 2.1 7.2 3.4 8.1

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 11 / 26

slide-12
SLIDE 12 UGent

STATS

VM

Basic concepts in experimental design The tools

Blinding

(1) Blind investigator for assessing response (2) Blind subject for treatment received ⇓ Takes care of placebo effect (1) and (2): DOUBLE BLIND

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 12 / 26

slide-13
SLIDE 13 UGent

STATS

VM

Basic concepts in experimental design Building blocks of an experiment

Experimental versus observational units

Experimental unit: entity to which a treatment is randomly assigned Observational unit: entity to which a response variable is measured Sometimes experimental unit = observational unit Sometimes more than 1 type of experimental unit

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 13 / 26

slide-14
SLIDE 14 UGent

STATS

VM

Basic concepts in experimental design Building blocks of an experiment

Mastitis: experimental unit=observational unit

Infusion Dose Low High

L H L H

One herd Experimental unit = cow = observational unit

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 14 / 26

slide-15
SLIDE 15 UGent

STATS

VM

Basic concepts in experimental design Building blocks of an experiment

Mastitis: experimental unit = observational unit

L L L L H H H H

herd 1 herd 2 Dose assigned to herd → Herd = experimental unit Cow = observational unit

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 15 / 26

slide-16
SLIDE 16 UGent

STATS

VM

Basic concepts in experimental design Building blocks of an experiment

Mastitis: experimental unit = observational unit

L L L L H H H H

herd 1 herd 2 Dose assigned to herd → Herd = experimental unit Cow = observational unit

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 15 / 26

slide-17
SLIDE 17 UGent

STATS

VM

Basic concepts in experimental design Building blocks of an experiment

Mastitis: experimental unit = observational unit

L L L L H H H H

herd 1 herd 2 Dose assigned to herd → Herd = experimental unit Cow = observational unit

L L L L H H H H H H H H L L L L

herd 1 herd 2 herd 3 herd 4

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 15 / 26

slide-18
SLIDE 18 UGent

STATS

VM

Basic concepts in experimental design Building blocks of an experiment

Mastitis: two types of experimental units

Dose + Parity Low High Heifer Multiparous

M H M H H H M M M M H H H M H M

H herd 1 L herd 2 L herd 3 H herd 4 Experimental unit for DOSE: HERD Experimental unit for PARITY: ANIMAL Observational unit: ANIMAL

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 16 / 26

slide-19
SLIDE 19 UGent

STATS

VM

Basic concepts in experimental design Building blocks of an experiment

Replication versus Repeated measure

Random assignment of treatment → REPLICATION Different observations on 1 experimental unit → REPEATED MEASURES in time/space Replication → information about variation between experimental units Repeated measure → better assessment of experimental unit Var(mean) < Var(observation)

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 17 / 26

slide-20
SLIDE 20 UGent

STATS

VM

Basic concepts in experimental design Building blocks of an experiment

Repeated measures in time

Cow with HIGH infusion dose Cow with LOW infusion dose

SCC at 9h SCC at 6h SCC at 3h SCC at 9h SCC at 6h SCC at 3h

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 18 / 26

slide-21
SLIDE 21 UGent

STATS

VM

Basic concepts in experimental design Building blocks of an experiment

Repeated measures in time

Cow with HIGH infusion dose Cow with LOW infusion dose

SCC at 9h SCC at 6h SCC at 3h SCC at 9h SCC at 6h SCC at 3h

DO NOT ANALYSE SUCH DATA

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 18 / 26

slide-22
SLIDE 22 UGent

STATS

VM

Basic concepts in experimental design Building blocks of an experiment

Repeated measures in space

Vaccination for mastitis herd 1 herd 2 all LOW dose all HIGH dose → No replications, only repeated measures Difference between HIGH and LOW infusion dose = difference between two herds

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 19 / 26

slide-23
SLIDE 23 UGent

STATS

VM

Basic concepts in experimental design Power analysis

Power analysis

Objective of experiment: Reject H0 Failure to reject H0 can be due to

(1) H0 is correct (2) Bad luck, unrepresentative sample (3) Large difference, but not enough evidence

(3) can be avoided by power analysis

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 20 / 26

slide-24
SLIDE 24 UGent

STATS

VM

Basic concepts in experimental design Power analysis

Sample size determination

The required sample is determined by 4 parameters Probability of type I error α Probability of type II error β, or power=1-β The variance between experimental units σ2 True underlying difference, e.g., for comparing two means: ∆ = µ1 − µ2

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 21 / 26

slide-25
SLIDE 25 UGent

STATS

VM

Basic concepts in experimental design Power analysis

Power as function of sample size

We discuss only comparison of two means µ1 and µ2 Required sample size for one-sided test H0 : µ1 = µ2 versus Ha : µ1 > µ2 is given by n = 2 (zβ + zα)2 σ2 ∆2 with n the sample size in each of the two groups For the two-sided test H0 : µ1 = µ2 versus Ha : µ1 = µ2 we have n = 2

  • zβ + zα/2

2 σ2 ∆2

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 22 / 26

slide-26
SLIDE 26 UGent

STATS

VM

Basic concepts in experimental design Power analysis

Determining zβ and zα

Z ∼ N(0, 1) P (Z < zα) = α in R: qnorm(α)

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 23 / 26

slide-27
SLIDE 27 UGent

STATS

VM

Basic concepts in experimental design Power analysis

Z ∼ N(0, 1) P (Z < z0.05) = 0.05 in R: qnorm(0.05)=-1.645

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 24 / 26

slide-28
SLIDE 28 UGent

STATS

VM

Basic concepts in experimental design Power analysis

Power analysis in practice

Compare low and high infusion dose for effect on milk reduction Set

power = 1 − β = 0.90, zβ = z0.1 = −1.28 α = 5%, two sided → z0.025 = −1.96 ∆ = 2%, relevant difference σ2 = 2

n = 2(−1.28−1.96)22

22

= 10.5 → Use 22 animals for experiment Try this now yourself for a one-sided hypothesis test, all other parameters remaining the same.

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 25 / 26

slide-29
SLIDE 29 UGent

STATS

VM

Basic concepts in experimental design Power analysis

Power analysis in practice

Compare low and high infusion dose for effect on milk reduction Set

power = 1 − β = 0.90, zβ = z0.1 = −1.28 α = 5%, two sided → z0.05 = −1.645 ∆ = 2%, relevant difference σ2 = 2

n = 2(−1.28−1.645)22

22

= 8.55 → Use 18 animals for experiment

  • Prof. Dr. Luc Duchateau (UGent)

Basic Experimental Design 2018 26 / 26