romip one step forward one step aside
play

ROMIP: one step forward, one step aside http://romip.ru/en/ Pavel - PowerPoint PPT Presentation

Mar 15, 2024 •219 likes •378 views

ROMIP: one step forward, one step aside http://romip.ru/en/ Pavel Braslavski, Ilia Chetviorkin, Maxim Gubin, Natalia Lukashevich, Igor Nekrestyanov, Marina Nekrestyanova, Natalia Vassileva CLEF 2012 ROMIP at a glance TREC-like Russian

  1. ROMIP: one step forward, one step aside http://romip.ru/en/ Pavel Braslavski, Ilia Chetviorkin, Maxim Gubin, Natalia Lukashevich, Igor Nekrestyanov, Marina Nekrestyanova, Natalia Vassileva CLEF 2012

  2. ROMIP at a glance • TREC-like Russian initiative • Started 2002 • Several freely available text and image collections • 10-15 participating teams each year • Remote participation + live meeting • Popular testbed for IR research in Russia • Related activities: RuSSIR 20.09.2012 ROMIP 2

  3. ROMIP 2004 20.09.2012 ROMIP 3

  4. Largest text collections Evaluated within Size Collection Documents Topics ad-hoc search (compressed) track ~300,000 Legal 2 Gb 14,794 220 By.Web 1,524,676 8 Gb ~ 60,000 1 500+ KM.RU 3,010,455 13 Gb ~ 60,000 ~250 20.09.2012 ROMIP 4

  5. (Retired) text document tracks � Ad-hoc text retrieval � Text categorization � Snippet generation � QA and fact extraction � News clustering � Search by sample document 20.09.2012 ROMIP 5

  6. Image collections � Photo collection: 20,000 images from Flickr � Dups collection: 15 hrs video � 37 800 frames � Panoramic series: 55,000 images (data recycled from Internet Math 2011) 20.09.2012 ROMIP 6

  7. Image tracks � Content based image retrieval � Near-duplicate detection � Image annotation � Finding panoramic series 20.09.2012 ROMIP 7

  8. ROMIP by 2011 • Low participation from academia • Fatigue of classical IR tasks available relevance tables – no need to participate • • overfitting on available datasets; • hard to model realistic settings and data; • well-studied tasks – new results are hard to expect. • Limited resources • ML challenges (e.g. www.kaggle.com) 20.09.2012 ROMIP 8

  9. ROMIP light 2011 • Sentiment analysis • Search by query image (low participation � ) • Schedule shifted to fall 20.09.2012 ROMIP 9

  10. ROMIP timeline 30 systems applied 25 systems participated # of tracks 20 15 10 5 0 2003 2004 2005 2006 2007 2008 2009 2010 2011 20.09.2012 ROMIP 10

  11. Sentiment analysis (SA) � Three domains: movies , books , and digital cameras � ‘Transfer learning’ (data from different sources) � Classification into 2, 3, and 5 classes � 23 teams registered � 12 submitted results � 6 reports published � 2-class: 105 runs, 3-class: 81 runs, 5-class: 30 runs 20.09.2012 ROMIP 11

  12. SA: data Training set 15,000+ movie reviews (10-point scale) 24,000+ book reviews (10-point scale) 10,000+ camera reviews (5-point scale) Test set blog posts collected via blog search w. subsequent filtering 275 posts on movies 329 posts on books 270 posts on digital cameras 20.09.2012 ROMIP 12

  13. Plans • New edition of SA track • Finer granularity (sentence) • Opinions for a given entity • Re-launch of image tracks (in cooperation with Graphicon conference) • Machine translation track 20.09.2012 ROMIP 13

  14. MT evaluation track (2012) • Strong industrial players • 1M parallel sentences (Ru-En) to release • Collaboration with TAUS Labs • Metrics BLEU • • human assessment 20.09.2012 ROMIP 14

  15. Thank you! Questions? Pavel Braslavski pb@kontur.ru 20.09.2012 ROMIP 15

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

min { a i , a j } max { a i , a j } P j P j P i P i P i P j Step 1 Step 2 Step 3 Figure 9.1 A

min { a i , a j } max { a i , a j } P j P j P i P i P i P j Step 1 Step 2 Step 3 Figure 9.1 A

a i a j a i , a j a j , a i min { a i , a j } max { a i , a j } P j P j P i P i P i P j Step 1 Step 2 Step 3 Figure 9.1 A parallel compare-exchange operation. Processes P i and P j send their elements to each other. Process P i keeps min { a i ,

372 views • 22 slides
Step 1 Step 2 Step 3 Step 4 Step 5 Preparation of a sketch Submission of birth map of all

Step 1 Step 2 Step 3 Step 4 Step 5 Preparation of a sketch Submission of birth map of all

Step 1 Step 2 Step 3 Step 4 Step 5 Preparation of a sketch Submission of birth map of all customary information forms to Preparation and adoption of Convening and submission Submission of Application Ste land owned by ILG obtain

1.23k views • 4 slides
Russian Information Retrieval Evaluation Seminar (ROMIP) (ROMIP) http://romip.ru/en/ Igor

Russian Information Retrieval Evaluation Seminar (ROMIP) (ROMIP) http://romip.ru/en/ Igor

Russian Information Retrieval Evaluation Seminar (ROMIP) (ROMIP) http://romip.ru/en/ Igor Nekrestyanov, Pavel Braslavski CLEF 2010 ROMIP at a glance ROMIP at a glance TREC like Russian initiative Started 2002 Several text and

561 views • 12 slides
Using RtI for SLD eligibility: Using RtI for SLD eligibility: Step by step procedures Step by

Using RtI for SLD eligibility: Using RtI for SLD eligibility: Step by step procedures Step by

Using RtI for SLD eligibility: Using RtI for SLD eligibility: Step by step procedures Step by step procedures Webster Groves School District 2009 Response to Intervention (RtI) is a comprehensive early detection and prevention strategy

688 views • 23 slides
o o o o Step 1:

o o o o Step 1:

o o o o Step 1: Bachelor of Laws (Western Sydney School of Law) Step 2: Professional Legal Training (College of Law et ors) Step 3: Admission to the Supreme

366 views • 14 slides
Outbreak Investigation Outbreak Investigation Step by Step Step by Step Darin Areechokchai MD.,

Outbreak Investigation Outbreak Investigation Step by Step Step by Step Darin Areechokchai MD.,

Outbreak Investigation Outbreak Investigation Step by Step Step by Step Darin Areechokchai MD., DTM&H., MCTM. Surveillance and Investigation Section Bureau of Epidemiology, Department of Disease Control Ministry of Public Health, Thailand

1.27k views • 83 slides
Step by step guide Step 1: Purchasing an RSBlog! membership Step 2: Downloading RSBlog! Step 3:

Step by step guide Step 1: Purchasing an RSBlog! membership Step 2: Downloading RSBlog! Step 3:

Step by step guide Step 1: Purchasing an RSBlog! membership Step 2: Downloading RSBlog! Step 3: Installing RSBlog! 3.1 Installing the component 3.2 Installing a new language file Step 4: Import plugins (optional) 4.1 Import Joomla! articles

889 views • 67 slides
Quick guide Step 1: Purchasing an RSEvents! membership Step 2: Downloading RSEvents! Step 3:

Quick guide Step 1: Purchasing an RSEvents! membership Step 2: Downloading RSEvents! Step 3:

Quick guide Step 1: Purchasing an RSEvents! membership Step 2: Downloading RSEvents! Step 3: Installing RSEvents! Step 4: Configure RSEvents! Step 5: Add user permissions Step 6: Create event categories Step 7: Create event locations Step 8:

627 views • 35 slides
FROM XML TO RDF STEP BY STEP: APPROACHES FOR LEVERAGING

FROM XML TO RDF STEP BY STEP: APPROACHES FOR LEVERAGING

Co-funded by the Horizon 2020 Framework Programme of the European Union Grant Agreement Number 644771 FROM XML TO RDF STEP BY STEP: APPROACHES

613 views • 26 slides
FIRST DAY OF SCHOOL STEP BY STEP DIRECTION TO GETTING LOGGED ONTO YOUR FIRST DAY OF 7TH OR 8TH

FIRST DAY OF SCHOOL STEP BY STEP DIRECTION TO GETTING LOGGED ONTO YOUR FIRST DAY OF 7TH OR 8TH

FIRST DAY OF SCHOOL STEP BY STEP DIRECTION TO GETTING LOGGED ONTO YOUR FIRST DAY OF 7TH OR 8TH GRADE AUGUST 10,2020 IS A MINIMUM DAY STUDENTS WILL LOG 7:45AM - ONTO THEIR 12:27PM CLASSES AND ENGAGE WITH THEIR TEACHERS AND PEERS STEP 1:

276 views • 6 slides
Small step and Auto Zhuoran Liu May 30th Difference between small step and big step The big

Small step and Auto Zhuoran Liu May 30th Difference between small step and big step The big

Type Theory and Coq Small step and Auto Zhuoran Liu May 30th Difference between small step and big step The big step specify how a given expression can be evaluated to its final value. The style is simple and natural for many purposes and is

196 views • 19 slides
Step by step guide Step 1: Purchasing a RSTickets!Pro membership Step 2: Downloading

Step by step guide Step 1: Purchasing a RSTickets!Pro membership Step 2: Downloading

Step by step guide Step 1: Purchasing a RSTickets!Pro membership Step 2: Downloading RSTickets!Pro 2.1 Downloading the component 2.2 Downloading the plugins/modules 2.3 Downloading additional language packs Step 3: Installing RSTickets!Pro

548 views • 44 slides
Quick guide Step 1: Purchasing RSMail! Step 2: Download RSMail! Step 3: Installing RSMail! Step

Quick guide Step 1: Purchasing RSMail! Step 2: Download RSMail! Step 3: Installing RSMail! Step

Quick guide Step 1: Purchasing RSMail! Step 2: Download RSMail! Step 3: Installing RSMail! Step 4: RSMail! settings Step 5: Add Subscribers 5.1. Create subscriber lists 5.2. Add subscribers 5.2.1 Manual add 5.2.2 Import from CSV Step 6

625 views • 14 slides
Quick guide Step 1: Purchasing a RSComments! membership Step 2: Download RSComments! Step 3:

Quick guide Step 1: Purchasing a RSComments! membership Step 2: Download RSComments! Step 3:

Quick guide Step 1: Purchasing a RSComments! membership Step 2: Download RSComments! Step 3: Installing RSComments! Step 4: Configure RSComments 4.1 General Configuration 4.2 Configure comments Step 5: Add user permissions Step 6: Moderate

337 views • 8 slides
Step by step guide Step 1: Purchasing an RSEvents! membership Step 2: Downloading RSEvents! Step

Step by step guide Step 1: Purchasing an RSEvents! membership Step 2: Downloading RSEvents! Step

Step by step guide Step 1: Purchasing an RSEvents! membership Step 2: Downloading RSEvents! Step 3: Installing RSEvents! Step 4: Configure RSEvents! 4.1 General settings 4.2 Dashboard settings 4.3 Events 4.3.1 Event general settings 4.3.2

1.29k views • 80 slides
Background CLINICAL TRIAL AGREEMENT ACTIVITY TIME TRACKING PI / Department Reviews Other

Background CLINICAL TRIAL AGREEMENT ACTIVITY TIME TRACKING PI / Department Reviews Other

Background CLINICAL TRIAL AGREEMENT ACTIVITY TIME TRACKING PI / Department Reviews Other Intermountain Reviews Approvals & Signtures Manage and Close Step 1: Step 1a: Step 1b Step 1c Dept Step 2: Step 3: Step 4 : Step 5: Step 6 :

754 views • 31 slides
and superb craftsmanship she finds true expression in all aspects of creativity that interest and

and superb craftsmanship she finds true expression in all aspects of creativity that interest and

The art of simplicity rather than ostentatious splendor is our conception of luxury. It was around this ideal that Le Roch Hotel & Spa, a 37 room boutique hotel with the spirit of a home, offering a sincere welcome, was conceived. The interior

389 views • 22 slides
Multilingual Web Retrieval Experiments with Field Specific Indexing Strategies for CLEF 2006

Multilingual Web Retrieval Experiments with Field Specific Indexing Strategies for CLEF 2006

Ben Heuwing, Thomas Mandl, Robert Strtgen Information Science Universitt Hildesheim mandl@uni-hildesheim.de Multilingual Web Retrieval Experiments with Field Specific Indexing Strategies for CLEF 2006 Cross-Language Evaluation Forum

178 views • 17 slides
IMI: A Public Private Partnership Funder 25-26 July 2011, European Medicines Agency

IMI: A Public Private Partnership Funder 25-26 July 2011, European Medicines Agency

IMI: A Public Private Partnership Funder 25-26 July 2011, European Medicines Agency Transatlantic workshop: Drug-related Progressive Multifocal Leukoencephalopathy The Innovative Medicines Initiative (IMI): the largest PPP in life sciences

655 views • 23 slides
& Real Estate 1 www.mobypark.com About MobyPark Introduction We open the gates of private

& Real Estate 1 www.mobypark.com About MobyPark Introduction We open the gates of private

& Real Estate 1 www.mobypark.com About MobyPark Introduction We open the gates of private parking on behalf of individuals and companies. Today drivers can park their cars either on the streets or in public garages (Q-park; Apcoa,

438 views • 13 slides
Ascott Residence Trust A Leading Global Serviced Residence REIT 2Q 2018 Financial Results 24

Ascott Residence Trust A Leading Global Serviced Residence REIT 2Q 2018 Financial Results 24

Ascott Residence Trust A Leading Global Serviced Residence REIT 2Q 2018 Financial Results 24 July 2018 1 Financial Results for 2Q 2018 *24 July 2018* Important Notice The value of units in Ascott Residence Trust (Ascott REIT) (the

801 views • 46 slides
Data and Evaluation: Critical Resources for Research in Knowledge Processing Edouard Geoffrois

Data and Evaluation: Critical Resources for Research in Knowledge Processing Edouard Geoffrois

Data and Evaluation: Critical Resources for Research in Knowledge Processing Edouard Geoffrois French National Research Agency (ANR/STIC) & French National Defence Procurement Agency (DGA/DS/MRIS) CHIST-ERA Conference 2011 Cork, Ireland,

385 views • 19 slides
Powerpoint Presentation Templates Education Academic and education templates to assist students

Powerpoint Presentation Templates Education Academic and education templates to assist students

Powerpoint Presentation Templates Education Academic and education templates to assist students and teachers. Music score presentation (treble clef design), PowerPoint, FREE. Quiz show game. Welcom to Download Free powerpoint templates design -

314 views • 3 slides
An Egyptian Exporter Full Year 2009 Results Presentation March 2010 Corporate Summary

An Egyptian Exporter Full Year 2009 Results Presentation March 2010 Corporate Summary

An Egyptian Exporter Full Year 2009 Results Presentation March 2010 Corporate Summary Introduction: An Egyptian exporter Lecico Egypt S.A.E. is one of the worlds largest sanitary ware producers and a large tiles producer Kfarchima

373 views • 20 slides

More recommend