THE POLYGLOT SEARCH TRANSLATOR Justin Clark Centre for Research in - - PowerPoint PPT Presentation

SYSTEMATIC REVIEW ACCELERATOR: THE POLYGLOT SEARCH TRANSLATOR

Justin Clark Centre for Research in Evidence Based Practice (CREBP) Bond University, Gold Coast, Australia

SR tasks – full list of tasks and tools

Task No. Classification Task Description Automation tool 1 Preparation Formulate review question Decide on the research question of the review. SRA – TRIP data analysis 2 Preparation Find previous or upcoming SR Search for a SR that answers the same question. Cochrane library, Epistemonikos, PROSPERO 3 Write up Write the protocol Provide an objective, reproducible, sound methodology for peer review. Reporting guidelines, existing protocol templates 4 Preparation Devise search strategy Decide on databases and keywords to find all relevant trials. SRA - Registered Trials Finder (in development), SRA - Word Frequency Analyser 5 Retrieval Search Aim to find all relevant citations even if many irrelevant ones included. SRA - Polyglot Search Translator 6 Retrieval De-duplicate Remove identical citations. SRA - De-duplicator. Rayyan 7 Appraisal Screen abstracts Based on titles and abstracts, remove definitely irrelevant citations. SRA - Endnote Helper, Covidence, Rayyan 8 Retrieval Obtain full text Download, request copies from authors, inter-library loan, etc. Endnote - find full text feature, SRA – Endnote Helper 9 Appraisal Screen full text Exclude irrelevant citations. SRA - Endnote Helper, Covidence 10 Retrieval Citation analysis Follow citations, cited and citing, from included trials to find additional trials. SRA - SpiderCite (in development) 11 Synthesis Extract data Extract outcome numbers and associate with trial arm Robot Reviewer 12 Synthesis Synthesize data Convert extracted data to common representation (usually average and SD). Revman 13 Retrieval Recheck literature Repeat the search to find new literature published since the initial search. SRA - Polyglot Search Translator (future development) 14 Synthesis Meta-analyze Statistically combine the results from all included trials. RevMan, MetaXL 15 Write up Write up review Produce and publish the final report SRA – Revman Replicant

Polyglot Search Translator (PST)

• The PST translates either a PubMed or Ovid Medline

search strategy to the correct syntax for different databases

Polyglot Search Translator

• Copy your PubMed or Ovid Medline search and paste it in to the ‘Your Query’

box

• To see how the search translates across databases, click the arrow next to

the name of the database.

Polyglot Search Translator

• For some databases you will need to convert the subject terms (e.g. change

MeSH terms to Emtree terms)

Polyglot Search Translator

• The PST is designed to make searching databases easier, BUT it does not

remove the need to have knowledge of the process Medline Ovid to Cochrane Central

Evaluation of the Polyglot Search Translator a tool for improving searches in systematic reviews: a multi-centre randomized crossover trial

• Clark J1, Carter M1, Honeyman D2, Cleo G1, Auld Y3, Booth D4,

Condron P5, Dalais C6, Dern S2, Linthwaite B3, May N7, Munn J8, Ramsay L9, Rickett K6, Rutter C10, Sanders S1, Smith A11, Sondergeld P10, Wallin M11, Beller E1

• 1 Centre for Research in Evidence Based Practice, Bond University, Australia
• 2 Bond University Library, Bond University, Australia
• 3 Gold Coast Health Library Service, Gold Coast University Hospital, Australia
• 4 University Library, University of Newcastle, Australia
• 5 University Library, University of Melbourne, Australia
• 6 University Library, University of Queensland, Australia
• 7 Sturt Library, Flinders University, Australia
• 8 Centre for Teaching and Learning, Southern Cross University, Australia
• 9 University Library, Australian Catholic University, Australia
• 10 University Library, Queensland University of Technology, Australia
• 11 Hunter New England Health Libraries, NSW Health
• 12 University Library, Southern Cross University, Australia

Background

• Searching the literature to identify relevant studies is an

important part of a SR

• Translating the search to run in multiple databases is

needed for a good SR

• To improve the speed of this the Polyglot Search

Translator (PST) was developed.

Objectives

• To determine whether a PST-assisted search translation

improves the speed without an increase in errors when translating search strings from one database to another.

Methods

• Participants (n=20) were recruited, 4 dropped out leaving

16 (10 of whom completed all 10 translations)

• Participants were randomly allocated ten search

strategies from a pool of 20

• They were then randomly assigned to either use the PST

to help translate it, or to not use it

• We compared how long it took, how many errors were

made and how many search results they found

Results

• A PST assisted search strategy translation was faster

with, a mean time saving of 14 minutes and 18 seconds per translation.

• If you approximate a translation time of 1 hour per search

this meant a 24% reduction in time.

Figure 1 Mean time taken (minutes) to translate searches

45.1 62.6 79.1 66.8 72.8 70.6 44.7 41.5 27.6 33.6 26.8 28.3 23.7 42.0 11.4 30.8 43.8 49.7 23.5 49.8 52.5 56.7 30.9 17.7 22.5 15.5 24.5 18.1 20.0 5.8 10 20 30 40 50 60 70 80 90 All databases Ovid MEDLINE to All databases Ovid MEDLINE to CINAHL Ovid MEDLINE to Cochrane Ovid MEDLINE to Embase Ovid MEDLINE to Pubmed Ovid MEDLINE to Scopus Ovid MEDLINE to Web of Science PubMed to All databases PubMed to CINAHL PubMed to Cochrane PubMed to Embase PubMed to MEDLINE PubMed to Scopus PubMed to Web of Science PST-A method Manual method

Search hits: median percentage difference from the expected number to be found

PST-A method Manual method Translation Median % Difference Range Median % Difference Range PubMed-Web of Science

• 1.47

(-87.3 to 600.0) 22.8 (-77.2 to 625.7) PubMed-Scopus 768.3 (-95.2 to 18700.0) 46.1 (0.8 to 28000) PubMed-MEDLINE

• 12.9

(-66.7 to 82.4)

• 17.3

(-57.8 to 160.2) PubMed-Embase

• 2.5

(-60.7 to 86.6)

• 17.9

(-37.9 to 19.8) PubMed-Cochrane

• 1.1

(-100.0 to 486.9) 34.6 (-48.5 to 1467.2) PubMed-CINAHL 13.8 (-38.6 to 1262.5) 43.0 (-4.6 to 300.0) PubMed - All databases

• 1.5

(-100.0 to 18700) 2.1 (-77.2 to 28000.0) MEDLINE-Web of Science 11.7 (-75.9 to 596.7) 7.0 (-78.3 to 731.9) MEDLINE-Scopus 22.5 (-41.2 to 2012.5) 21.9 (-91.4 to 7424.8) MEDLINE-PubMed 2.4 (-98.8 to 753.1) 20.4 (-97.5 to 4658.5) MEDLINE-Embase

• 23.1

(-99.6 to 7.2)

• 12.4

(-89.7 to 13.7) MEDLINE-Cochrane 2.4 (-100.0 to 39716.7) 1.4 (-98.9 to 1562.9) MEDLINE-CINAHL 40.6 (-84.9 to 563.0) 104.0 (-36.6 to 2984.7) MEDLINE - All databases 3.0 (-100.0 to 39716.7) 7.0 (-98.9 to 4658.5) All databases 1.7 (-100.0 to 39716.7) 6.3 (-98.9 to 4658.5)

Errors: mean number of total errors

14 15 7 27 13 15 17 2 9 6 8 18 13 9 12 14 1 16 17 23 3 5 7 7 15 6 2 5 10 15 20 25 30 Total Ovid MEDLINE-total Ovid MEDLINE-CINAHL Ovid MEDLINE-Cochrane Ovid MEDLINE-Embase Ovid MEDLINE-PubMed Ovid MEDLINE-Scopus Ovid MEDLINE-Web of Science PubMed-total PubMed-CINAHL PubMed-Cochrane PubMed-Embase PubMed-MEDLINE PubMed-Scopus PubMed-Web of Science Mean total errors Polyglot (n=95) Mean total errors Manual (n=103)

Errors: Number of type of errors

Type of error All databases Polyglot (n=95) All databases Manual (n=103) Wrong wildcard or truncation syntax - recall 28 21 Missing or added truncation or wildcard - recall 6 14 Missing or added truncation or wildcard - Precision 1 Truncated term at wrong point - recall 2 5 Truncated term at wrong point - Precision 3 Missing subject term - recall 3 3 Missing keyword term - recall 4 12 Restricting to major subject terms - recall 3 3 Not exploding subject terms - recall 7 8 Exploding subject terms - precision 3 17 Incorrect MeSH conversion - recall 3 5 Incorrect MeSH conversion - Precision 3 1 Incorrect adjacency syntax - recall 5 1 Used incorrect adjacency - recall 2 Used incorrect adjacency- Precision 3 6 Incorrect field syntax - recall 9 2 Incorrect field syntax - precision 1 Chose incorrect fields - recall 4 13 Chose incorrect fields - precision 7 13 Incorrect phrase translation - recall 6 3 Incorrect phrase translation - Precision 1 6 Extra out of place characters inserted - recall 8 4 Spelling mistake in search - recall 1 7 Incorrect boolean operator - recall 2 1 Incorrect search structure - recall 6 7 Extra components added - recall Extra components added - Precision 3 Total unique errors 117 159

Conclusion

• For people skilled in conducting searches of the literature

the PST can increase the speed of translating search strategies h which this can be done, without a decrease in quality