Istanbul University Assessing the quality of methods used in DISCLOSURES standardized terminology studies There are no conflicts of interest or relevant financial interests that have been disclosed by this presenter or the rest of the Second International Conference on planners and presenters of this activity that apply to this Research Methods for Standard Terminologies learning session. Selda Secginli, PhD, Associate Professor Istanbul University, Florence Nightingale Nursing Faculty, Public Health Nursing Department Istanbul, Turkey 1 Standardized Terminologies Objectives • A structured language consisting of terms, At the end of this session, the learner will be better able to: definitions, and codes that clinicians use to guide • define quality assessment and document practice. i.e. Omaha System • discuss the importance of use of the quality assessment • They are important for higher quality care, enhanced tools care coordination, improved documentation and • get an overview of how these tools are used in a patient outcomes, reduced costs. systematic review study based on Omaha System, one of the standardized terminology used in Turkey 3 4 Standardized Terminologies-cont. What Is Quality Assessment? • Selection of standard terminology; implications for • The confidence that the trial design, conduct, and design of forms, orders, decision support, storage; analysis has minimized or avoided biases in its treatment workflow analysis for context and organizing the data comparisons. for use in reports and clinical decision making is critical. • Quality assessment of research involves appraisal of a • The use of structured clinical records with standardized study's internal validity, i.e. the degree to which its data increases the likelihood of good data quality. design, conduct and analysis have minimized biases or • Good data quality enhanced study quality. errors. For practical reasons, study quality assessment in reviews often covers both internal and external validity. • Lack of standardization of terminology is an important • A broader notion of research quality should help barrier to performing high- quality research . researchers and research users to feel confident about the use of evidence in policy and practice. 5 6 1
Standards for assessing the quality of research Key Elements of a Systematic Review • Pose a significant, important question • Apply methods that best address the research • Complete coverage of the relevant literature • Provide the necessary information to reproduce or replicate the study • Ensure the study design, methods, and procedures • Provide sufficient description of the sample, the intervention. • Use appropriate and reliable conceptualization and measurement of variables • Evaluate alternative explanations for any findings • Assess the possible impact of systematic bias • Adhere to quality standards for reporting 7 Quality Assessment Tools Quality Assessment Tools-cont. • There are many tools for assessing the • Studies may be biased due to inadequate methodological quality of the studies. randomization, unsuitable comparison interventions, • Some of these tools were developed for specific lack of blind outcome assessment, inadequate follow-up, inability to define and assess outcomes, study designs (eg. RCTs, cohort studies, descriptive unreliable measurement techniques, and studies), while others were developed for a range of inappropriate statistical analyses. study designs. • With these tools included a number of quality items, • Mostly, these tools incorporate characteristics that which are scored numerically provide a quantitative are associated with bias. However, there are also estimate of overall study quality. many tools that contain items related to reporting and these are not related with bias. 9 10 Quality criteria for assessing Quality Assessment Tools-cont. experimental studies • Some examples should be: The Cochrane ROB tool was • Was the assignment to the treatment groups really designed for RCTs and is the instrument recommended random? by The Cochrane Collaboration for use in systematic • Was the treatment allocation concealed? reviews of RCTs. The Newcastle-Ottawa Scale is • Were the groups similar at baseline in terms of prognostic commonly used for nonrandomized studies, specifically factors? cohort and case-control studies. JBI critical appraisal • Were the eligibility criteria specified? tools can be used for RCTs, quasi-experimental, case- • Were outcome assessors blinded to the treatment control/cohort studies and descriptive/case series allocation? studies. • Was the care provider blinded? Was the patient blinded? • Did the analyses include an intention to treat analysis? 11 12 2
The Joanna Briggs Institute (JBI) JBI Critical Appraisal Tools Quality Assessment Tools • These checklists use a series of criteria that can be scored as being met, not met or unclear or if deemed appropriate not applicable (N/A) to that particular study. • The decision as to whether or not to include a study can be made based on meeting a predetermined proportion of all criteria, or on certain criteria being met. • Each items were answered dichotomously, where "yes" was allocated with one point and "no”, "unclear” and “not applicable” with zero. • The cut-off score for inclusion of studies after methodological appraisal was set at 4/10 (evaluating criteria) 13 14 • The electronic databases Ovid MEDLINE, PUB MED, Cochrane CENTRAL • To describe and analyze recent literature about Register of Controlled Trials, CINAHL, PsycINFO, Web of Knowledge, Scopus, Google Scholar, Omaha System in Turkey ULAKBIM Turkish Medical Database and Council of Higher Education Thesis Center were • To identify the Omaha System study design, searched. Additionally, the reference lists of included studies from the Turkish databases were methodologies and areas for future research, in hand searched. Turkey (1) studies published between 2000 and 2012, • In the study, papers that met the inclusion criteria (2) studies published both in English and Turkish, (3) were considered to be applicable to the review full text articles that were published in a peer- topic and retrieved for further assessment of reviewed journal, thesis and dissertations, (4) quality. conducted in Turkey, 15 16 5. Assessment of Turkish JBI Deneysel v e Yarı Deneysel Ara ştı rmalar İçin methodological quality MAStARI Kritik De ğ erlendirme Formu Quality assessment Tools De ğ erlendiren : Tarih : • The main object is to • Cohen’s kappa: .74 Yazar : Yı l : Kay ı t no : Evet Hayır Belirtil- Uygun (experimental studies) .67 assess the memi ş de ğ il (descriptive studies) 1. Kat ı l ı m cı lar giri ş im/tedavi grupla rı na ge rçe kten rastgele mi methodological quality atanm ış t ı r? • Test-retest reliability: .87 2. Kat ı l ı m cı lar giri ş im/tedavi grupla rı na kö r teknikle mi atanm ış t ı r? of a study (the (kat ı l ı m cı la rı n yap ı lan giri ş imi/tedaviyi bilmemesi) (experimental studies); .91 3. Kat ı l ı m cı la rı ça l ış ma grupla rı na da ğı tan ki ş i, kat ı l ı m cı la rı n hangi tedaviyi ald ığı ndan habersiz miydi? possibility of bias in its (descriptive studies) 4. Ç al ış madan ay rı lan kat ı l ı m cı la rı n sonu çl a rı verilmi ş ve analize dahil edilmi ş midir? design, conduct and • Kuder richardson: .68 5. Sonu çl a rı de ğ erlendirenler kat ı l ı m cı la rı n hangi grupta oldukla rı ndan habersiz miydi? analysis) and exclude (experimental studies); .64 6. Ç al ış man ı n ba ş lang ıcı nda deney ve kontrol grupla rı temel ö zellikler y ö n ü nden benzer miydi? studies that are of low (descriptive studies) 7. Gruplar uygulanan giri ş im(ler) d ışı nda ayn ı ş ekilde takip edilmi ş miydi? • Content validity index: .90 quality and identify the 8. Sonu çl ar t ü m gruplarda ayn ı ş ekilde mi ö l çü lm üş t ü ? (experimental studies); .87 9. Sonu çl ar g ü venilir ş ekilde ö l çü lm üş m ü d ü r? strengths and 10. Uygun istatistiksel analizler kullan ı lm ış m ı d ı r? (descriptive studies) limitations of the Genel de ğ erlendirme : ( ) Kabul et ( ) Kabul etme ( ) Daha fazla bilgi ara ş t ı r included studies. A çı klama (Kabul etmeme nedenleri): 17 18 3
Recommend
More recommend
