neural embeddings for
play

Neural Embeddings for Populated GeoNames Locations Mayank Kejriwal, - PowerPoint PPT Presentation

Jun 24, 2023 •17 likes •187 views

Neural Embeddings for Populated GeoNames Locations Mayank Kejriwal, Pedro Szekely USC Information Sciences Institute Motivation: feature extraction from locations Essential for machine learning problems involving locations Machine learning

  1. Neural Embeddings for Populated GeoNames Locations Mayank Kejriwal, Pedro Szekely USC Information Sciences Institute

  2. Motivation: feature extraction from locations • Essential for machine learning problems involving locations

  3. Machine learning applications • Toponym resolution e.g. "Boston" in England, UK vs. "Boston" in Massachusetts, USA • Much more likely to be Boston, MA if ‘New York’ and ‘Martha’s Vineyard’ also got extracted in a similar context • Features are hybrid i.e. must encode both location and ‘context’ (e.g. text)

  4. Machine learning applications • Toponym resolution e.g. "Boston" in England, UK vs. "Boston" in Massachusetts, USA • Much more likely to be Boston, MA if ‘New York’ and ‘Martha’s Vineyard’ also got extracted in a similar context • Features are hybrid i.e. must encode both location and ‘context’ (e.g. text) • Named entity disambiguation e.g. Was ‘Charlotte’ extracted as a name or a location?

  5. Motivation: feature extraction from locations • Essential for machine learning problems Why not use latitude-longitude directly?

  6. What makes for a ‘good’ feature space? • Captures proximity semantics • Real-valued, not very high-dimensional • Not too sensitive (1.0 vs. 1.001) • Extensible • Does not necessarily require manual tuning • Generic i.e. can be visualized in some way

  7. Do lat-long points capture proximity semantics? • Only in a very dense, non-linear space

  8. More formally... • dist(lat 1 , long 1 , lat 2 , long 2 ) is well-approximated using the Haversine formula

  9. Do lat-long points capture proximity semantics? • Discontinuous (in linear space)!

  10. Do lat-long points capture proximity semantics? • Sensitive (more than other features typically used in machine learning pipelines)

  11. What makes for a good feature space? • Captures proximity semantics • Real-valued, not very high-dimensional • Not too sensitive (1.0 vs. 1.001) • Extensible • Does not necessarily require manual tuning • Generic i.e. can be visualized in some way

  12. Id Idea: ‘Embed’ Geonames as a weighted, directed network... • ...in a vector space! • Vector similarities (using dot product similarity) depend inversely on geodesic distances 2-dimensional un-normalized 100-dimensional embeddings (latitude- normalized embeddings in longitude) in complex, dot product space sensitive space

  13. Step 1: Determine set of nodes in network • Nodes in Geonames identifies by following feature codes: [`PPL', `PPLA', `PPLA2', `PPLA3', `PPLA4', `PPLC', `PPLCH', `PPLF', `PPLG', `PPLH', `PPLL', `PPLQ', `PPLR', `PPLS', `PPLW', `PPLX', ~4.4 million nodes `STLMT']

  14. Step 2: Determine edges and weights • Pairwise in the worst case • Slide a window over nodes sorted by latitude or longitude, only form edges between nodes in the same window. ~357,000 nodes • Postprocess by removing nodes with ~9 million edges 0 population

  15. Step 3: Run DeepWalk on network • DeepWalk (Perozzi et al., 2014) is a powerful neural network algorithm for embedding nodes in graphs; has achieved powerful results • Very fast!

  16. Example in paper: North Dakota

  17. Vectors, code and raw data all on GitHub (also, figshare) https://github.com/mayankkejriwal/Geonames-embeddings

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings Agenda Traditional NLP Word Embeddings-1 Word Embeddings-2 Text preprocessing Topic Modeling ELMo Bag-of-words model Neural

900 views • 64 slides
IN5550 Neural Methods in Natural Language Processing Convolutional Neural Networks Erik

IN5550 Neural Methods in Natural Language Processing Convolutional Neural Networks Erik

IN5550 Neural Methods in Natural Language Processing Convolutional Neural Networks Erik Velldal University of Oslo 25 February 2020 So far: MLPs + embeddings as inputs Embeddings have benefits over discrete feature vectors; makes

902 views • 35 slides
IN5550 Neural Methods in Natural Language Processing Convolutional Neural Networks Erik

IN5550 Neural Methods in Natural Language Processing Convolutional Neural Networks Erik

IN5550 Neural Methods in Natural Language Processing Convolutional Neural Networks Erik Velldal University of Oslo 25 February 2020 So far: MLPs + embeddings as inputs Embeddings have benefits over discrete feature vectors; makes

806 views • 58 slides
IN5550: Neural Methods in Natural Language Processing Lecture 11/1 Contextualized embeddings

IN5550: Neural Methods in Natural Language Processing Lecture 11/1 Contextualized embeddings

IN5550: Neural Methods in Natural Language Processing Lecture 11/1 Contextualized embeddings Andrey Kutuzov University of Oslo 14 April 2020 1 Contents Brief Recap 1 Problems of static word embeddings 2 Solution: contextualized

372 views • 24 slides
Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to generate vector embeddings? One approach: feedforward neural language models Training a neural language model just to get word embeddings is expensive!

569 views • 38 slides
Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to generate vector embeddings? One approach: feedforward neural language models Training a neural language model just to get word embeddings is expensive!

537 views • 35 slides
IN5550: Neural Methods in Natural Language Processing Lecture 6 Evaluating Word Embeddings and

IN5550: Neural Methods in Natural Language Processing Lecture 6 Evaluating Word Embeddings and

IN5550: Neural Methods in Natural Language Processing Lecture 6 Evaluating Word Embeddings and Using them in Deep Neural Networks Andrey Kutuzov, Vinit Ravishankar, Lilja vrelid, Stephan Oepen, & Erik Velldal University of Oslo 21

581 views • 42 slides
Image Space Embeddings and Generalized Convolutional Neural Networks Nate Strawn September

Image Space Embeddings and Generalized Convolutional Neural Networks Nate Strawn September

Image Space Embeddings and Generalized Convolutional Neural Networks Nate Strawn September 20th, 2019 Georgetown University Table of Contents 1. Introduction 2. Smooth Image Space Embeddings 3. Example: Dictionary Learning 4. Convolutional

530 views • 49 slides
SI425 : NLP Set 14 Neural NLP Fall 2020 : Chambers Why are these so different? Last time :

SI425 : NLP Set 14 Neural NLP Fall 2020 : Chambers Why are these so different? Last time :

SI425 : NLP Set 14 Neural NLP Fall 2020 : Chambers Why are these so different? Last time : Word2Vec learned word embeddings This time : use word embeddings as input to classifiers Recall: Logistic Regression x = it was the best of times it

740 views • 44 slides
Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation

Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation

1 Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation Nikolaos Pappas 1 , Lesly Miculicich 1 , 2 , James Henderson 1 1 Idiap Research Institute, Switzerland 2 Ecole polytechnique f ed erale de

456 views • 18 slides
Jointly Learning Word and Phrase Embeddings Using Neural Networks and Implicit Tensor

Jointly Learning Word and Phrase Embeddings Using Neural Networks and Implicit Tensor

Jointly Learning Word and Phrase Embeddings Using Neural Networks and Implicit Tensor Factorization Kazuma Hashimoto Tsuruoka Laboratory, University of Tokyo 19/06/2015 Talk@UCL Machine Reading Lab. Self Introduction Name Kazuma

726 views • 39 slides
Normalizing tweets with edit scripts and recurrent neural embeddings Grzegorz Chrupaa |

Normalizing tweets with edit scripts and recurrent neural embeddings Grzegorz Chrupaa |

Normalizing tweets with edit scripts and recurrent neural embeddings Grzegorz Chrupaa | Tilburg University Normalizing tweets Convert tweets to canonical form easy to understand for downstream applications Examples I will c wat i can do

373 views • 19 slides
Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models Jamie Ryan Kiros,

Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models Jamie Ryan Kiros,

Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models Jamie Ryan Kiros, Ruslan Salakhutdinov, Richard Zemel Presentation by David Madras University of Toronto January 25, 2017 Image Captioning ??????? Image Retrieval

430 views • 32 slides
Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling Gbor Berend

Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling Gbor Berend

Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling Gbor Berend 31/07/2017 Vancouver, ACL Continuous word representations apple [1 0 0 0 0 0 0 0 0 0] [3.2 -1.5] ... banana [0 0 0 0 1 0 0

702 views • 30 slides
Lecture 26 Word Embeddings and Recurrent Nets Julia Hockenmaier juliahmr@illinois.edu 3324

Lecture 26 Word Embeddings and Recurrent Nets Julia Hockenmaier juliahmr@illinois.edu 3324

CS447: Natural Language Processing http://courses.engr.illinois.edu/cs447 Lecture 26 Word Embeddings and Recurrent Nets Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center Where were at Lecture 25: Word Embeddings and neural LMs

767 views • 49 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

1.25k views • 75 slides
Preparing a Benefit-Cost Office of the Chief Analysis Economist United States July 23, 2019

Preparing a Benefit-Cost Office of the Chief Analysis Economist United States July 23, 2019

Maritime Administration Port Infrastructure Development Grant Program Presented by: Preparing a Benefit-Cost Office of the Chief Analysis Economist United States July 23, 2019 Department of Maritime Administration 1200 New Jersey Ave.,

425 views • 25 slides
Neural Networks Neural Net Basics Dan Klein, John DeNero UC Berkeley Slides adapted from Greg

Neural Networks Neural Net Basics Dan Klein, John DeNero UC Berkeley Slides adapted from Greg

Neural Networks Neural Net Basics Dan Klein, John DeNero UC Berkeley Slides adapted from Greg Durrett Neural Networks Neural Networks Linear classification: argmax y w > f ( x, y ) possible because Linear Neural we transformed

316 views • 4 slides
Overview Last time we studied the evolution of a discrete linear dynamical system, and today we

Overview Last time we studied the evolution of a discrete linear dynamical system, and today we

Overview Last time we studied the evolution of a discrete linear dynamical system, and today we begin the final topic of the course (loosely speaking). Today well recall the definition and properties of the dot product. In the next two weeks

537 views • 37 slides
Silvano DAL ZILIO LAAS-CNRS, Vertics team presentation for our paper: MCC: a Tool for Unfolding

Silvano DAL ZILIO LAAS-CNRS, Vertics team presentation for our paper: MCC: a Tool for Unfolding

A tool to remove colors from your High-Level Petri nets ! Silvano DAL ZILIO LAAS-CNRS, Vertics team presentation for our paper: MCC: a Tool for Unfolding Colored Petri Nets in PNML Format for the Petri Nets Conference June 2020 MCC is not an

319 views • 27 slides
SOFTWARE ENGINEERING IN STARTUPS Dr. Vadim Zaytsev Universiteit van Amsterdam 20 January 2014

SOFTWARE ENGINEERING IN STARTUPS Dr. Vadim Zaytsev Universiteit van Amsterdam 20 January 2014

SOFTWARE ENGINEERING IN STARTUPS Dr. Vadim Zaytsev Universiteit van Amsterdam 20 January 2014 CC-BY-SA Introduction Vadim Zaytsev aka @grammarware currently at UvA worked at CWI (Amsterdam) worked at Uni Koblenz (Germany)

581 views • 33 slides
Classbased Detailed Routing in VLSI Design C. Schulte, T. Nieberg Research Institute for Discrete

Classbased Detailed Routing in VLSI Design C. Schulte, T. Nieberg Research Institute for Discrete

Classbased Detailed Routing in VLSI Design C. Schulte, T. Nieberg Research Institute for Discrete Mathematics University of Bonn CTW 2008 Overview of the Talk Introduction to VLSI Routing Challenges of the new 65nm Technology

356 views • 24 slides
GEOMETRIC STEINER TREE PACKING WITH DENSITY CONSTRAINTS Nicolai Hhnle, Pietro Saccardi Aussois

GEOMETRIC STEINER TREE PACKING WITH DENSITY CONSTRAINTS Nicolai Hhnle, Pietro Saccardi Aussois

GEOMETRIC STEINER TREE PACKING WITH DENSITY CONSTRAINTS Nicolai Hhnle, Pietro Saccardi Aussois Combinatorial Optimization Workshop January 12, 2017 Research Institute for Discrete Mathematics, Bonn 1/30 OVERVIEW Motivation: global routing

694 views • 67 slides
ACM.org Highlights For Scientists, Programmers, Designers, and Managers: Learning Center -

ACM.org Highlights For Scientists, Programmers, Designers, and Managers: Learning Center -

ACM.org Highlights For Scientists, Programmers, Designers, and Managers: Learning Center - https://learning.acm.org View past TechTalks & Podcasts with top inventors, innovators, entrepreneurs, & award winners Access to

1.12k views • 88 slides

More recommend