Best Practices for Independent Peer Reviews Jacqueline Patterson - - PowerPoint PPT Presentation

Toxicology Excellence for Risk Assessment

Independent ● Non-Profit ● Science

for Public Health Protection

Best Practices for Independent Peer Reviews

Jacqueline Patterson Toxicology Excellence for Risk Assessment (TERA)

Society for Risk Analysis (SRA) Annual Meeting December 11, 2013

Robust Processes

To insure quality and independence of results

• Scientific

– Selection of experts – Quality and completeness of review materials – Comprehensive charge questions – Clear reporting of results

• Administrative

– Peer review plan – Clear instructions to experts – Defined roles for participants – Documentation of results – Independence

Independent ● Non-Profit ● Science

for Public Health Protection

2

Principles or Best Practices for a Successful Peer Review

• Appropriate Experts
• Scientific Robustness
• Transparency
• Independence

Independent ● Non-Profit ● Science

for Public Health Protection

3

Appropriate Independent Experts

• Appropriate experience to

address key issues

• Balance of views on key

topics

• Diversity in institutional

backgrounds and perspectives

• Risk assessment expertise

essential

Peer– a person who is the equal

• f another in

abilities and qualifications

Independent ● Non-Profit ● Science

for Public Health Protection

4

Panel Selection Procedures

• Evaluate subject document closely to identify

key questions, guiding determination of types

• f experts and experience needed
• Broadly search for experts
• Evaluate credentials and potential for conflicts
• f interest and biases
• Select a panel that is balanced with regard to

necessary disciplines and has a diversity of perspectives

Independent ● Non-Profit ● Science

for Public Health Protection

5

Peer Review for Risk Assessment

– Data – Alternate interpretations – Mode of action – Assumptions – Calculations – Counterfactual evidence – Conclusions

Independent ● Non-Profit ● Science

for Public Health Protection

6

Complete Review Package

• Quality work product – Ready for Review
• Balance between essential information

and reviewers’ time

• References and data readily available
• Clear charge and instructions
• Assist peer reviewers

Independent ● Non-Profit ● Science

for Public Health Protection

7

Peer Review Charge

Purpose - identify the key scientific issues and guide the experts to consider each important issue in a multi- disciplined way

• Comprehensive - yet focused on most important

questions to address purpose of peer review

• Experts must understand the purpose and context for

the assessment – problem formulation

• Objective - developed independently- with input from

authors – to insure all key issues included and

• bjectivity of question wording
• Provide experts clearly worded questions, with well-

defined criteria for evaluation and clear instructions

Independent ● Non-Profit ● Science

for Public Health Protection

8

Panel Meetings

• Skilled chair with risk assessment expertise

to guide panel through issues

• Focused agenda and pacing
• Authors present to answer and ask

questions

• Interested parties allowed to contribute

scientific data and analyses and ask questions

Independent ● Non-Profit ● Science

for Public Health Protection

9

Transparency

• An operational philosophy

that allows interested persons to evaluate and judge the adequacy and credibility of the process and results

• Open meetings/process that

welcomes input

• Documentation of process and results
• Independent evaluation of responsiveness of

authors

Independent ● Non-Profit ● Science

for Public Health Protection

10

Independence –

more than just conflict of interest

• Process and participants need to be independent
• f the outcome
• A third party – not the authors or sponsoring

agency or organization, should -

– Select experts – Prepare charge questions – Keep authors, sponsors, and stakeholders at arm’s length from experts

• Evaluate authors’ responsiveness

Independent ● Non-Profit ● Science

for Public Health Protection

11

The most critical element

Independent ● Non-Profit ● Science

for Public Health Protection

12

Peer Review is Important

• Recent survey by George Mason University of SRA, SOT, and

ISRTP risk assessors found that*:

• 73% agreed that external independent peer review is very

important

• 78% indicated external peer review should be conducted

independently of the office or program that develops a risk assessment

• 65% would create an independent entity that insures

authors would respond to peer review comments *Robert Lichter presentation, Webinar and Study Report available at www.istrp.org

Independent ● Non-Profit ● Science

for Public Health Protection

13

What do peer reviewers think about current peer review practices?

• What is essential for a good peer

review?

• What hinders a good peer review?
• What improvements or changes

would you suggest?

Independent ● Non-Profit ● Science

for Public Health Protection

14

What do peer reviewers think about current peer review practices?

• What is essential for a good peer

review?

• What hinders a good peer review?
• What improvements or changes

would you suggest?

Independent ● Non-Profit ● Science

for Public Health Protection

15

References

• Meek, ME, Patterson, J; Strawson, J; Liteplo, R.

(2007) Engaging Expert Peers in the Development

• f Risk Assessments. Risk Anal. 27(6):1609-

1621.

• Turner, RS. 2009. Best Practices in Peer Review

Assure Quality, Value, Objectivity. Journal of the National Grants Management Association 17:1

• George Mason University. 2013. Expert Opinion
• n Regulatory Risk Assessment. Results of a Survey of

SRA, SOT/RASS and ISRTP members. Available at

www.Isrtp.org

Independent ● Non-Profit ● Science

for Public Health Protection

16