Current Issues in the Design of Cluster Randomization Trials Allan - - PowerPoint PPT Presentation

Current Issues in the Design of Cluster Randomization Trials

Allan Donner, PhD, FRSC

Department of Epidemiology and Biostatistics The University of Western Ontario London, Canada Robarts Clinical Trials, Robarts Research Institute London, Canada

What Are Cluster Randomization Trials

Cluster randomization trials are experiments in which intact social units or clusters of individuals rather than independent individuals are randomly allocated to intervention groups.

Examples:

Medical practices selected as the randomization unit in trials evaluating the efficacy of disease screening programs Communities selected as the randomization unit in trials evaluating the effectiveness of new vaccines in developing countries Hospitals selected as the randomization unit in trials evaluating educational guidelines directed at physicians and/or administrators

Reasons for Adopting Cluster Randomization

Administrative convenience To obtain cooperation of investigators Ethical considerations To enhance subject compliance To avoid treatment group contamination Intervention naturally applied at the cluster level

Role in vaccine trials

Randomization of geographic areas can be used to capture indirect (herd ) effects of vaccination. Incidence of disease among non-vaccinated individuals in the intervention group is compared to incidence of disease in the control group.

Unit of Randomization vs. Unit of Analysis

A key property of cluster randomization trials is that inferences are frequently intended to apply at the individual level while randomization is at the cluster or group level. Thus the unit of randomization may be different from the unit of analysis. In this case, the lack of independence among individuals in the same cluster, i.e., intracluster correlation, creates special methodological challenges in both design and analysis.

Quantifying the Effect of Clustering

Consider a trial in which k clusters of size m are randomly assigned to each of an experimental and control group denoted by i =1 and 2 respectively. Let denote the sample mean of the response variable in the group Then assuming is normally distributed with common variance where is the coefficient of intracluster correlation.

Equivalently, If clusters of size are randomized to each of two treatment groups, then the effective sample size per group is given by

Application of standard sample size approaches leads to an underpowered study (Type II error) Application of standard statistical methods generally tends to bias p-values downwards, i.e., could lead to spurious statistical significance (Type I error)

Examples:

COMPLETELY RANDOMIZED DESIGN

Study Purpose: To evaluate the effectiveness of vitamin A supplements on childhood mortality. 450 villages in Indonesia were randomly assigned to ether participate in a vitamin A supplementation scheme, or serve as a control. One year mortality rates were compared in the two groups. Sommer et al. (1986)

MATCHED PAIR DESIGN

Study Purpose: The COMMIT community intervention trial(1995) was designed to promote smoking cessation using a variety of community resources. The primary

• utcome measure was the 5-year smoking cessation

rate.

Unit of Randomization:

Community

Number of Matched Pairs:

11

Matching factors: size, pop. density

STRATIFIED DESIGN

Study Purpose: The purpose of the WHO antenatal trial was to compare the impact of two programmes of antenatal care on the health of mothers and newborns.

Unit of Randomization: Antenatal care clinic.

Stratification Variables: Primary stratification was

by country: Thailand, Cuba, Argentina, Saudi Arabia.

Number of Clusters per Stratum:

Ranged from 12 to 17. Villar et al. (2001)

CLUSTER-CROSSOVER DESIGN

All participating clusters receive both intervention and control in a sequence determined at random Reduces total number of clusters required with a considerable increase in study duration Turner(2007)

STEPPED WEDGE DESIGN

All clusters eventually cross over but only from the control to intervention at a time point determined at random Allows clusters to be enrolled gradually over time Hussey and Hughes(2007)

Key question: Are the trial inferences aimed at (i) the individual subject? or (ii) at a naturally defined cluster of individuals? This decision guides both the choice of design and the approach to the analysis

Selecting the Unit of Inference

Example 1:

The intervention consists of a blood pressure screening program intended to lower the risk of cardiovascular mortality Bass et al (1986)

Example 2:

The intervention consists of educational guidelines for the management of hyperlipidaemia intended to increase the proportion of eligible patients prescribed lipid-lowering drugs Diwan et al (1995)

Consider a trial randomizing hospitals designed to assess the effect of obtaining a second clinical

• pinion on the decision to proceed with a

caesarian section operation (Altabe et al 2008). The target of the intervention was the hospital rate

• f caesarian section.

In this case, the hospital was the natural unit of inference and standard methods of sample size estimation and analysis applied at the cluster level.

Key point: If the unit of inference is at the cluster level then an analysis at the cluster level is appropriate, and no consideration need be given to the intracluster correlation coefficient. From the perspective of sample size estimation and analysis the challenges are no different from those that arise in individually randomized trials.

Simplified data collection and lower cost Can be applied to any outcome variable Exact statistical inferences can always be constructed Can be adapted to adjust for baseline imbalances Informed consent issues may be eased

Cluster level Analyses:Advantages

A Common Misconception

Investigators have occasionally claimed that cluster level analyses will only provide valid statistical inferences when the intracluster correlation coefficient However this is too stringent a claim. For balanced designs, cluster level analyses provide inferences which are equivalent to those

• btained using a mixed linear regression model

for any value of

Threats to trial validity?

Selection bias in the recruitment of patients over time Experimental contamination

i)Risk of selection bias

Consider a trial using a simple randomization scheme practices in which physicians are asked to identify as well as to treat selected patients (e.g. Kinmouth et al [1998]). Can cluster randomization introduce bias through the way patients are differentially recruited across treatment groups?

Source of bias:

If the physician’s practice has already been randomized, recruitment for patient participation cannot be done blindly with respect to intervention group. If physicians in the experimental group are more diligent in seeking out patients than in the control group, or tend to identify patients who are less ill, bias may result.

Unbiased estimates of the effect of intervention can be assured only if analyses are (i) based on data from all cluster members

• r

(ii) based on a random sub-sample of cluster members or

Possible Solutions:

Identify eligible patients in each practice prior to randomization. If eligible patients are identified after randomization, recruitment should be done by an individual independent of the trial. Torgerson (2001) Farrin et al (2005)

ii) Is fear of contamination a legitimate reason adopting cluster randomization?

Suppose under individual randomization a proportion of R of the control group patients experience the same success rate P1 as seen among experimental patients.

Then the difference that can be detected is reduced to and the required sample size must be inflated by the factor IF = 1/ (1-R)2 e.g. If R =0.30, then IF =2.04 But under unrestricted cluster randomization, the inflation factor 1+(m-1) might be more Farrin et al (2005)

However…..

Uncertainty concerning true level of contamination Presence of contamination under individual randomization will lead to an underestimate of the effect of treatment

Logistical difficulties a factor-giving different treatments at random to different patients in the same

• ffice.

Can create pairs of clusters between which travel is difficult.

Issues Involving Informed Consent

Two distinct levels of informed consent must be distinguished in cluster randomization trials:

(i)

informed consent for randomization (usually provided by a ‘decision-maker’)

(ii)

informed consent for participants given that randomization has occurred.

By analogy to current ethical requirements for clinical trials, it would be unethical not to obtain informed consent from every cluster member prior to random assignment. Is such a strict analogy required for community randomized trials?

“Ethical advice indicated that, since we were only providing information to clinicians, there was no reason to seek patient consent” Wyatt et al (1998)

“Eliciting consent from the many thousands of patients involved would create severe budgetary and logistic

• bstacles.

The central human rights committee at the Hines Coordinating Center and those at the participating sites have deemed this type of quality improvement research exempt from direct patients consent”. US Dept. of Veterans Affairs

The Effect of Level of Intervention

Edwards et al. (1999) distinguished cluster randomization trials by the level of intervention: for “individual-cluster” trials intervention is provided directly to individual study subjects (e.g., STD treatment for prevention of HIV). for “cluster-cluster” trials intervention is provided at the cluster level (e.g., effect of local medical opinion leaders on patient treatment). The need for consent by individual study subjects is deemed of particular concern for “individual cluster” trials.

The need for consent by individual study subjects is deemed of particular concern for “individual cluster” trials. However these distinctions may only be relevant when randomizing larger clusters (e.g., work sites, classrooms, communities).

Is Informed Consent Required for Cluster- Cluster Trials?

“Informed consent to randomization may not be

required if it is not possible to approach subjects at the time of randomization. However potential subjects approached after randomization must be provided with a detailed description of the interventions in the trial arm to which their clusters have been randomized” McRae et al(2009)

Cluster Randomization Trials and the Zelen Randomized Consent Design

The requirements of informed consent are often perceived to be a barrier to patient accrual. Randomized consent designs were introduced by Zelen (1990) to lower this barrier, thus hastening trial completion. In the single consent design patients are randomly assigned to one of two groups , but with only patients in the intervention group asked to provide informed

• consent. If not, they may be offered the control

therapy.

Analyses are conducted based on the intervention to which patients were initially assigned, i.e., analysis by intention to treat. Randomized consent designs have proved quite controversial because of concern for the ethical implications of randomizing subjects prior to obtaining their consent. But this strategy is typical of many cluster randomized trials!

Factors Influencing Loss of Precision Cluster Randomization Trials

1. Interventions often applied on a group basis 2. Some studies permit the immigration of new subjects after baseline. 3. Entire clusters, rather than just individuals, may be lost to follow-up. 4. Difficulties associated with prevention trials

Why are so many CRT’s unpowered?

“Generally the size of effects has been meager in

relation to the effort expended” “We often do not have the resources to detect medium effects”

Susser (1995)

Strategies for Increasing Precision

Restrict inclusion criteria

e.g. recruit practices of same size, with physicians having similar years of experience in a controlled geographic area.

Conduct a baseline survey of the prevalence of the trial outcome

• serves as powerful risk factor for outcome
• provides estimate of
• allows trial personnel to gain experience
• can help to identify potential stratification factors

Increasing cluster size provides diminishing returns in statistical power if

e.g. If increasing beyond 100 provides little statistical gain.

Donner and Klar (2004)

Consider sample size re-estimation as part of an interim analysis.

Lake et al (2001)

Recognize likely size of in primary care research (0.01 to 0.05 for outcome variables; 0.05 to 0.15 for process variables). Campbell et al (2001)

Gulliford et al (2004)

IMPACT ON POWER OF INCREASING THE NUMBER OF CLUSTERS VS. INCREASING CLUSTER SIZE: Let As As

Statistical Challenges in the Meta-Analysis of Cluster Randomization Trials

Meta-analyses combining cluster randomization trials raise methodologic issues beyond those raised by meta-analyses which include only individually randomized trials:

• Increased Trial Heterogeneity
• Difficulties in Estimating Design Effects from

Individual Trials

• Special Issues Involving Publication Bias
• Choice of Statistical Method
• Assessment of Trial Quality

Study Heterogeneity

Sources of heterogeneity in meta-analyses involving cluster randomization trials:

(i)

Heterogeneity of study design

e.g., completely randomized, matched-pair, stratified

(ii)

Heterogeneity in unit of randomization

e.g., worksites schools households individuals

(iii) Variation in eligibility criteria at both the

cluster and individual level

(iv) Wide variation in methodological quality

Example of Design Heterogeneity in a Meta-Analysis:

 Brunner et al. (1997) reported on a meta-analysis of dietary

interventions aimed at reducing cardiovascular risk factors.

 17 randomized trials with a total of 6893 participants were

included in the meta-analysis.

 16 of the trials were individually randomized, one was cluster

randomized.

 The one cluster randomized trial (allocating worksites) included

approximately 42% of the 6893 participants.

 In this case, the effective sample size for the meta-analysis is

clearly much less than 6893.

Difficulties in Estimating Design Effects from Individual Trials

 For trials randomizing clusters of size m to two

intervention groups, ‘design effect’ is given by

 Estimation of the design effect is crucial for optimal

weighting of the trials to be combined.

However many trial reports fail to provide estimates

• f the design effect (Laopaiboon [2003], Thomas et
• al. [2003]). Therefore meta-analysts have resorted to

various ad-hoc strategies to obtain the required information: (i) increasing the variances of estimated intervention effects by an arbitrary amount (e.g., Fawzi et al. [1993]). (ii) using external data to obtain imputed values of the design effect (e.g., Rooney and Murray [1996]). (iii) using regression methods to estimate design effects from information on design effects reported in other trials (e.g., Beaton et al. [1993]). (iv) excluding cluster randomization trials from a meta-analysis (e.g., Xin et al. [2003]).

Assessing Quality of Individual Trials

Issues here particularly relevant to cluster randomization trials include:

• 1. Justification for randomizing clusters rather than

individuals.

• 2. Unambiguous definition of the randomization unit.
• 3. Eligibility criteria defined at both the individual and cluster

level.

• 4. Accounting for clustering in both the estimation of trial size

and in the analysis.

• 5. Reporting of observed intracluster correlation and/or

design effect.

Special Issues Involving Publication Bias

 Complication in cluster randomization trials is that many

such studies ignore the clustering in the analysis. These studies may then be reported as statistically significant when in fact a correctly performed analysis would not show significance.

 Conversely, if the analysis is correctly performed but power

considerations were ignored in the design, a finding of non- significance may be misleading.

Combining Trials in Which One Cluster Only Has Been Assigned to Each Group

 Community intervention trials have been reported in which

exactly one cluster has been assigned to the intervention group and one to the control group.

 These invariably result in interpretational difficulties due to

the confounding of two sources of variation:

(i) variation in response due to the effect of intervention. (ii) natural variation that exists between two clusters even in the absence of an intervention effect.

 However such trials could potentially be included in a meta-

analysis provided the randomization unit and subject population for such studies were comparable to those of the

• ther trials included. Under these conditions, the

intracluster correlation could also be assumed to be comparable to, and hence estimable from, the other trials.

SOME FUTURE CHALLENGES:

 What are the best methods of quantifying trial

heterogeneity?

 Development of efficient meta-regression techniques  Methods for combining individually randomized trials

with cluster randomized trials

OTHER ISSUES:

• 1. Is it fair to combine cluster randomization trials that have used

very different units of randomization? Or very different designs?

• 2. Since estimates of intracluster correlation and design effects are
• ften unreported, what are the best strategies for dealing with

this problem?

• 3. What are the best methods to adjust for clustering effects in

meta-analyses for different choices of endpoints and study designs?

• 3. Are fixed effects or random effects models for cluster trials most

appropriate?

• 4. Should trials which include only one cluster per intervention

group be included in a meta-analysis?

• 5. Can checklists be developed that assign quality scores to

individual trials? If so, how many points should be deducted when a study ignores clustering in the design or analysis?

Why adopt a non-randomized design ?

Intervention has already been implemented on a major scale, raising ethical issues. Often less expensive, easier logistics Easier to explain to public officials ,gain public acceptance.

The availability of a small number of clusters is not a reason to avoid a randomized design! Can match or stratify in the design Possibility of imbalance remains a problem

Estimation of the ICC: unresolved issues

i)Confidence interval construction for community intervention trials having a large number of small clusters and a binary outcome. ii)Definition of the ICC for time-to-event outcomes with censored data.

• Includes a checklist of items that should be included in

a trial report

• Extends CONSORT statement for reporting of

individually randomized trials (Begg et al, JAMA, 1996)

• Key Points
• Rationale for adopting cluster design
• Incorporation of clustering into sample size estimation and

analysis

• Chart showing flow of clusters through the trial, from

assignment to analysis

EXTENSION OF CONSORT STATEMENT TO CLUSTER RANDOMIZATION TRIALS (CAMPBELL ET AL, BMJ, 2004)