text mining and historical research
play

Text Mining and Historical Research Beatrice Alex - PowerPoint PPT Presentation

Oct 11, 2023 •316 likes •698 views

Text Mining and Historical Research Beatrice Alex balex@inf.ed.ac.uk MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014 OVERVIEW What is text mining? Types of text analyses. Trading Consequences: text

  1. Text Mining and Historical Research Beatrice Alex balex@inf.ed.ac.uk MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  2. OVERVIEW What is text mining? Types of text analyses. Trading Consequences: text mining applied. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  3. TEXT MINING D escribes a set of linguistic, statistical and/or machine learning techniques that model and structure the information content of textual resources. Turns unstructured text into structured data (e.g. relational database or linked data) . Is very useful for analysing large text collections automatically (overcoming data paralysis). Goal in DHSS research: By analysing large amounts of textual data, help HSS scholars to discover novel patterns and explore hypotheses. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  4. MINING WHAT TEXT? Electronic text or things that can be turned into it. Born electronic text (research papers, literature, tweets, blogs, comments on blogs etc.). Digitised text documents. Meta data (collection and document level). Image subtitles (Flickr image titles and subtitles). Video/audio transcripts (YouTube transcripts, TED talks, MOOC transcripts etc.) MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  5. TYPES OF ANALYSES Named entity recognition. Grounding, e.g. geo-referencing. Relation extraction. Clustering, e.g. topic modelling. Sentiment analysis. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  6. NAMED ENTITY RECOGNITION Identification and classification of entity mentions in text, things like: Names of persons, locations, organisations,... Dates, amounts ... Often used for improving access to collections. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  7. CONNECTED HISTORIES http://www.connectedhistories.org MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  8. OLD BAILEY ONLINE http://www.oldbaileyonline.org MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  9. GROUNDING Linking entity mentions in text to a unique identifier, e.g.: Person names to their Wikipedia pages Location names to lat/longs or Geonames IDs Gene names to gene ontologies Goal is to disambiguate between mentions with the same surface form (e.g. “Paris”, “Victoria”). MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  10. EDINBURGH GEOPARSER MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  11. RELATION EXTRACTION Identifying relations between entities in text or in meta data in order to Triples: person - author_of -> book title, commodity - traded_at -> location person - born_in -> location person - born_at -> date MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  12. RELATION EXTRACTION ChartEX, Discovering spatial descriptions and relationships in medieval charters, October 2013. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  13. RELATION EXTRACTION ChartEX, Discovering spatial descriptions and relationships in medieval charters, October 2013. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  14. RELATION EXTRACTION ChartEX, Discovering spatial descriptions and relationships in medieval charters, October 2013. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  15. SUMMARY SO FAR Different types of text analyses applied to historical and literary research but new opportunities are endless. Text mining can assist scholars in their research but it is not replacing them! Traditional scholarship is well suited to close reading. HSS scholars can focus on questions which can be answered by computational methods. Human interpretation is vital. Visualisation of TM output is important. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  16. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  17. PROJECT TEAM Ewan Klein, Bea Alex, Claire Grover, Richard Tobin: text mining Colin Coates, Andrew Watson: historical analysis Jim Clifford: historical analysis James Reid, Nicola Osborne : data management, social media Aaron Quigley, Uta Hinrichs: information visualisation MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  18. TRADITIONAL HISTORICAL RESEARCH Global Fats Supply 1894-98 Gillow and the Use of Mahogany in the Eighteenth Century, Adam Bowett, Regional Furniture, v.XII, 1998. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  19. PROJECT GOALS Text mining, data extraction and information visualisation to explore big historical datasets. Focus on how commodities were traded across the globe in the 19th century. Help historians to discover novel patterns and explore new research questions. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  20. DOCUMENT COLLECTIONS Collection # of Documents # of Images House of Commons Parliamentary Papers 118,526 6,448,739 (ProQuest) Early Canadiana Online 83,016 3,938,758 Directors’ Letters of 14,340 n/a Correspondence (Kew) Confidential Prints (Adam 1,315 140,010 Matthews) Foreign and Commonwealth Office 1,000 41,611 Collection Asia and the West (Gale) 4,725 948,773 (OCRed: 450,841) MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  21. DOCUMENT COLLECTIONS Collection # of Documents # of Images House of Commons Parliamentary Papers 118,526 6,448,739 (ProQuest) Early Canadiana Online 83,016 3,938,758 Over 10 million document pages, Directors’ Letters of 14,340 n/a Correspondence (Kew) Over 7 billion word tokens. Confidential Prints (Adam 1,315 140,010 Matthews) Foreign and Commonwealth Office 1,000 41,611 Collection Asia and the West (Gale) 4,725 948,773 (OCRed: 450,841) MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  22. MINED INFORMATION Example sentence: Normalised and grounded entities: commodity: cassia bark [concept: Cinnamomum cassia] date: 1871 (year=1871) location: Padang (lat=-0.94924;long=100.35427;country=ID) location: America (lat=39.76;long=-98.50;country=n/a) quantity + unit: 6,127 piculs MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  23. MINED INFORMATION Example sentence: Extracted entity attributes and relations: origin location: Padang destination location: America commodity–date relation: cassia bark – 1871 commodity–location relation: cassia bark – Padang commodity–location relation: cassia bark – America MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  24. COMMODITY LEXICON Seed set from customs import records. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  25. LEXICON CREATION Seed lexicon ~600 Extended lexicon ~17,000 With pluralisation of ~20,500 single word entries MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  26. LEXICON PRECISION ... From the top 1,757 entries only 84 (4.8%) had to be filtered. The top 1,757 entities amount to 99.8% of mentions. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  27. LEXICON PRECISION ... From the top 1,757 entries only 84 (4.8%) had to be filtered. The top 1,757 entities amount to 99.8% of mentions. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  28. OCR ERRORS Extract of Early Canadiana Online document 9_00952_3, p. vi. Extract of Early Canadiana Online document 9_00952_3, p. vi. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  29. OCR ERRORS Extract of Early Canadiana Online document 9_00952_3, p. vi. Extract of Early Canadiana Online document 9_00952_3, p. vi. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  30. OCR ERRORS Extract of Early Canadiana Online document 9_00952_3, p. vi. Extract of Early Canadiana Online document 9_00952_3, p. vi. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  31. OCR ERRORS Extract of Early Canadiana Online document 9_00952_3, p. vi. Extract of Early Canadiana Online document 9_00952_3, p. vi. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  32. OCR ERRORS MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  33. BRINGING ARCHIVES ALIVE MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

  34. SUMMARY You have access to enormous amounts of data. Text mining can be applied to process large text collections, enrich existing text with information or pull out trends which can be visualised. Text mining is a way to enable distant reading, even if such technology is not 100% accurate. OCR errors in digitised collections can skew your results. MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Digital history and big data: Text mining historical documents on trade in the British Empire

Digital history and big data: Text mining historical documents on trade in the British Empire

Digital history and big data: Text mining historical documents on trade in the British Empire Beatrice Alex balex@inf.ed.ac.uk Digital scholarship: day of ideas 2, Edinburgh, 02/05/2013 OVERVIEW What is text mining? Text Mining in digital

484 views • 27 slides
Enabling Digital history: Text mining big historical document collections on trade in the British

Enabling Digital history: Text mining big historical document collections on trade in the British

Enabling Digital history: Text mining big historical document collections on trade in the British Empire Beatrice Alex balex@inf.ed.ac.uk PQIS All Team Meeting, ProQuest, April 23rd 2014 TEXT MINING D escribes a set of linguistic, statistical

408 views • 39 slides
Text Mining for Historical Documents Motivation and Case Studies Caroline Sporleder

Text Mining for Historical Documents Motivation and Case Studies Caroline Sporleder

Text Mining for Historical Documents Motivation and Case Studies Caroline Sporleder Computational Linguistics/MMCI Universit at des Saarlandes Wintersemester 2011/12 22.02.2012 Caroline Sporleder Text Mining for Historical Documents IT

288 views • 18 slides
Text Mining and Geo-referencing Historical Text Beatrice Alex Edinburgh Language Technology

Text Mining and Geo-referencing Historical Text Beatrice Alex Edinburgh Language Technology

Text Mining and Geo-referencing Historical Text Beatrice Alex Edinburgh Language Technology Group School of Informatics balex@inf.ed.ac.uk @bea_alex DCHRN Workshop Cultural Heritage Sparks, Edinburgh, Jan 29th 2016 EDINBURGH LTG Language

288 views • 12 slides
Hotspot Mapper for World War II Unlocking the Secrets of the Past: Text Mining for Historical

Hotspot Mapper for World War II Unlocking the Secrets of the Past: Text Mining for Historical

Hotspot Mapper for World War II Unlocking the Secrets of the Past: Text Mining for Historical Documents Mariona Coll Ardanuy Seyed Mehdi Khodadad Hosseini Ehsan Khoddam Mohammadi Nikolina Koleva Peter Stahl Demo 2 Historical Motivation

419 views • 23 slides
Assisted Curation: Does Text Mining Really Help? (Alex et al. 2008) by Benedict Fehringer

Assisted Curation: Does Text Mining Really Help? (Alex et al. 2008) by Benedict Fehringer

Assisted Curation: Does Text Mining Really Help? (Alex et al. 2008) by Benedict Fehringer Seminar: Unlocking the Secrets of the Past: Text Mining for Historical Documents Supervisor: Dr. Caroline Sporleder (and Martin Schreiber)

900 views • 64 slides
Text Mining for Historical Documents Introduction to Computational Linguistics Caroline Sporleder

Text Mining for Historical Documents Introduction to Computational Linguistics Caroline Sporleder

Text Mining for Historical Documents Introduction to Computational Linguistics Caroline Sporleder Computational Linguistics Universit at des Saarlandes Wintersemester 2011/12 22.02.2011 Caroline Sporleder Text Mining for Historical

547 views • 34 slides
Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 21 to March 4, 2011

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 21 to March 4, 2011

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 21 to March 4, 2011 Machine Learning & Computational Biology Research Group MPIs Tbingen .: Data Mining in Bioinformatics, Page 1 What is text mining?

813 views • 31 slides
Text Mining Text Mining Web pages Emails Technical documents Corporate documents

Text Mining Text Mining Web pages Emails Technical documents Corporate documents

Motivation for Text Mining Motivation for Text Mining Approximately 90% of the Worlds data is held in unstructured formats Text Mining Text Mining Web pages Emails Technical documents Corporate documents Books

456 views • 9 slides
Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Text Mining with R Yanchang Zhao http://www.RDataMining.com R and Data Mining Course Beijing University of Posts and Telecommunications, Beijing, China July 2019 Chapter 10: Text Mining, in R and Data Mining: Examples and Case Studies

904 views • 64 slides
Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Text Mining with R Yanchang Zhao http://www.RDataMining.com Tutorial on Machine Learning with R The Melbourne Data Science Week 2017 1 June 2017 Chapter 10: Text Mining, in R and Data Mining: Examples and Case Studies .

733 views • 57 slides
Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow)

Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow)

Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow) & David Robinson (DataCamp) https://www.tidytextmining.com/ R for Data Science: Garrett Grolemund & Hadley Wickham http://r4ds.had.co.nz/

866 views • 15 slides
2. Text Mining D-BSSE Karsten Borgwardt Data Mining II Course, Basel Spring Semester 2016 118

2. Text Mining D-BSSE Karsten Borgwardt Data Mining II Course, Basel Spring Semester 2016 118

2. Text Mining D-BSSE Karsten Borgwardt Data Mining II Course, Basel Spring Semester 2016 118 / 179 Text Mining Goals To learn key problems and techniques in the mining one of the most common types of data To learn how to represent text

738 views • 59 slides
Text Mining in Search Engines By: DJ Ambler With special thanks to the Internet Overview

Text Mining in Search Engines By: DJ Ambler With special thanks to the Internet Overview

Text Mining in Search Engines By: DJ Ambler With special thanks to the Internet Overview What is text mining? How is it used in search engines? Text Mining Definition A way to extract meaning from text Structuring, deriving

314 views • 10 slides
Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 25 to March 10

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 25 to March 10

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 25 to March 10 Bioinformatics Group MPIs Tbingen Karsten Borgwardt: Data Mining in Bioinformatics, Page 1 What is text mining? Definition Text mining is the use

603 views • 31 slides
Combining data mining and text mining for detec1on of early

Combining data mining and text mining for detec1on of early

Combining data mining and text mining for detec1on of early stage demen1a: the SAMS framework Christopher Bull, Dommy Asfiandy, Ann Gledson, Joseph

284 views • 14 slides
How to Set Up an Effective Electronic Communications Plan on Short Notice Hosted by: Bonnie

How to Set Up an Effective Electronic Communications Plan on Short Notice Hosted by: Bonnie

How to Set Up an Effective Electronic Communications Plan on Short Notice Hosted by: Bonnie Olinger and Alaina Longo, Office for Communications Meghan Cokeley, Office for the New Evangelization Why Mass Email / Text? During the pandemic and as

471 views • 9 slides
MT SH HTTP://CS.AU.DK/~MKORN/ @MATSCH_ O 0 1 AARHUS UNIVERSITY FROM HYBRID SPACES TO

MT SH HTTP://CS.AU.DK/~MKORN/ @MATSCH_ O 0 1 AARHUS UNIVERSITY FROM HYBRID SPACES TO

AARHUS UNIVERSITY 27. JANUARY 2012 MT @ SH FROM HYBRID SPACES TO EXPERIENCING AUGMENTED PLACES MATTHIAS KORN PHD STUDENT MKORN@CS.AU.DK MT SH HTTP://CS.AU.DK/~MKORN/ @MATSCH_ O 0 1 AARHUS UNIVERSITY FROM HYBRID SPACES TO EXPERIENCING

840 views • 25 slides
Cloud Computing Karlsruhe Institute of Technology (KIT), Germany Institute for Applied

Cloud Computing Karlsruhe Institute of Technology (KIT), Germany Institute for Applied

About the Speaker: Stefan Tai Professor, Cloud Computing Karlsruhe Institute of Technology (KIT), Germany Institute for Applied Informatics (AIFB) Prof. Dr. Stefan Tai Karlsruhe Service Research Institute (KSRI) stefan.tai@kit.edu Director

521 views • 30 slides
Joint Meeting of the Town Board and Planning Board September 16, 2014 Technical assistance

Joint Meeting of the Town Board and Planning Board September 16, 2014 Technical assistance

Town of Selma Joint Meeting of the Town Board and Planning Board September 16, 2014 Technical assistance provided by Division of Community Assistance N.C. Department of Commerce Introduction Betsy Kane, J.D., Senior Planner Central

577 views • 31 slides
Lecture 1: Introduction to Digital Logic Design CSE 140: Components and Design Techniques for

Lecture 1: Introduction to Digital Logic Design CSE 140: Components and Design Techniques for

Lecture 1: Introduction to Digital Logic Design CSE 140: Components and Design Techniques for Digital Systems Winter 2016 CK Cheng Dept. of Computer Science and Engineering University of California, San Diego 1 Information about the

367 views • 35 slides
ELECTRONIC TEXT REUSE ACQUISITION PROJECT INTRODUCTION & MOTIVATION M arco Bchler, Greta

ELECTRONIC TEXT REUSE ACQUISITION PROJECT INTRODUCTION & MOTIVATION M arco Bchler, Greta

ELECTRONIC TEXT REUSE ACQUISITION PROJECT INTRODUCTION & MOTIVATION M arco Bchler, Greta Franzini, Emily Franzini TABLE OF CONTENTS 1. Who are we? 2. Motivation 2/16 WHO ARE WE? WHO AM I? 2001-2002: Head of Quality Assurance

293 views • 16 slides
When Digital Humanities Meet Artificial Intelligence: Introduction La Saint-Raymond, ENS

When Digital Humanities Meet Artificial Intelligence: Introduction La Saint-Raymond, ENS

DHAI seminar, October 7, 2019 When Digital Humanities Meet Artificial Intelligence: Introduction La Saint-Raymond, ENS (DMA) 1. A brief history of the encounter DH-AI 2. The meeting points 3. and the issues / tensions behind 1.A

611 views • 36 slides
Using Stylometry to Model Transmission of Arabic in Medieval Europe: the Case of the Bocados de

Using Stylometry to Model Transmission of Arabic in Medieval Europe: the Case of the Bocados de

Using Stylometry to Model Transmission of Arabic in Medieval Europe: the Case of the Bocados de oro David Joseph Wrisley American University of Beirut @DJWrisley SLIDE 1 What is stylometry? Also known as computational stylistics. A

283 views • 3 slides

More recommend