VANCOUVER WELCOMES YOU! MINIMALIST METONYMY RESOLUTION THE ROAD MAP - PowerPoint PPT Presentation

MILAN GRITTA, MOHAMMAD TAHER PILEHVAR, NUT LIMSOPATHAM, NIGEL COLLIER VANCOUVER WELCOMES YOU! MINIMALIST METONYMY RESOLUTION

THE ROAD MAP THE PROBLEM + BRIEF HISTORY PREWIN MODEL RELOCAR FEATURE SELECTION MINIMALIST NN NEW DATASET RESULTS + SUMMARY

THE ROAD MAP THE PROBLEM + BRIEF HISTORY PREWIN MODEL RELOCAR FEATURE SELECTION MINIMALIST NN NEW DATASET RESULTS + SUMMARY

GEOGRAPHICAL PARSING PHD RESEARCH WHAT IS A (METONYMIC) LOCATION? MOSCOW TO DISCUSS EBOLA RISKS IN WEST AFRICA.. ALUMINIUM AND GAS TRADING PICKED UP IN MOSCOW. LONDON VOTED TO STAY IN THE EUROPEAN UNION.. LONDON CONSTITUENCY VOTED TO REMAIN IN THE EU. METONYMY ~20% OF ALL LOCATIONS (BNC, WIKIPEDIA)

NER & METONYMY - GREATEST PROBLEM IN NLP? OR NOBODY CARES!?

VERY BRIEF HISTORY OF METONYMY RESOLUTION ▸ Nissim et al. (2002, 03, 05) Determiner of PMW Gram. role (obj, subj) SEMEVAL (2007) Shared Task Head modifier (dependency) 1 PMW (singular, plural) No of words in PMW ▸ Brun et al. (2007) 85.1% No of gram. roles of PMW ▸ Farkas et al. (2007) 85.2% Manual annotations: syntactical/grammatical Collocations list MASCARA additional data ▸ Nastase et al. (2009, 2012) 86.1% VerbNet Co-occurrence lists FrameNet ▸ Nastase et al. (2013) 86.2% Parsing BNC Multiple Parsers 2 Global Context POS Tags Parsing Wikipedia ▸ Zhang et al. (2015) 86.5** Proprietary Parser Levin’s Verb Classes Trigger Word Lists Selectional Preferences Handmade Features Custom Databases 3 Naïve Bayes Decision List ME Classifier Unsupervised Context Similarity SVM ** not peer reviewed, only in preprint

THE ROAD MAP THE PROBLEM + BRIEF HISTORY MODEL PREWIN RELOCAR MINIMALIST NN FEATURE SELECTION NEW DATASET RESULTS + SUMMARY

WHAT IS PREWIN? BASELINE 5, 10, 50 WINDOW COLLOBERT ET AL. (2011), MIKOLOV ET AL. (2014)

THE ROAD MAP THE PROBLEM + BRIEF HISTORY PREWIN MODEL RELOCAR FEATURE SELECTION MINIMALIST NN NEW DATASET RESULTS + SUMMARY

KERAS MINIMALIST MODEL

THE ROAD MAP THE PROBLEM + BRIEF HISTORY PREWIN MODEL RELOCAR FEATURE SELECTION MINIMALIST NN NEW DATASET RESULTS + SUMMARY

RELOCAR (REAL LOCATION RETRIEVAL) VERSUS SEMEVAL Germany, US and France MIXED 2% TRAIN 1,026 talk climate science, LITERAL 49% METONYMIC 49% TEST 1,000 ANNOTATED AT CAMBRIDGE UNI FACULTY FOR MODERN AND MEDIEVAL LANGUAGES

THE ROAD MAP THE PROBLEM + BRIEF HISTORY PREWIN MODEL RELOCAR FEATURE SELECTION MINIMALIST NN NEW DATASET RESULTS + SUMMARY

THE RESULTS SEMEVAL (SOTA 86.2%) RELOCAR (SOTA 84.8%) 90 90 ENSEMBLE 84.6% ENSEMBLE 84.8% 82 82 74 74 83.6 83.1 81.9 81.3 81.3 81.4 81.3 80 66 66 58 58 50 50 Base 5 Base 10 Paragraph PreWin Base 5 Base 10 Paragraph PreWin

THE RESULTS SEMEVAL TEST DATA RELOCAR TEST DATA 90 90 82 82 74 74 66 66 58 58 50 50 Trained on Trained on Trained on Trained on Trained on Trained on Semeval Relocar CONLL Semeval Relocar CONLL

THE RESULTS Prewin Window Size Comparison 90 88 SEMEVAL TEST DATA RELOCAR TEST DATA 86 90 90 84 82 82 82 74 74 80 66 66 78 76 CONLL Semeval Relocar 58 58 74 50 50 1 2 3 4 5 6 7 8 9 10 Trained on Trained on Trained on Trained on Trained on Trained on Semeval Relocar CONLL Semeval Relocar CONLL

SCIENCE REPLICATION GITHUB FOR CODE & DATA & INSTRUCTIONS. DATA.CAM.AC.UK REPOSITORY.CAM.AC.UK

SUMMARY & FUTURE WORK RELOCAR PREWIN MODEL MORE TEST DATA FEATURE SELECTION MINIMALIST NN ‣ NER DOES NOT DO METONYMY ‣ METONYMY AS ORGS, LOCS, PRODUCTS … ‣ SMALL DATA & MODEL CAN WORK

THANK YOU ALL ESPECIALLY WWW.DREAM-CDT.AC.UK