Good morning! Please: 1. Download all files for this lesson 2. Open - PowerPoint PPT Presentation

Oct 07, 2023 •462 likes •596 views

Good morning! Please: 1. Download all files for this lesson 2. Open all 3 notebooks in Jupyter 3. Make a Twitter account (optional) 4. Log in to your Twitter account Outline 1. APIs: a. Overview b. Example with Twitter 2. Scraping a.

Good morning! Please: 1. Download all files for this lesson 2. Open all 3 notebooks in Jupyter 3. Make a Twitter account (optional) 4. Log in to your Twitter account
Outline 1. APIs: a. Overview b. Example with Twitter 2. Scraping a. Overview b. Examples
What’s an API? ● “Application Programming Interface” ○ “Application:” program that does things for humans ○ API: does things for other programs ● Uses ○ Get data ○ Get services
Some things with APIs ● Twitter ○ Get tweets, post them, etc. ● Google ○ Search, translate, NLP… ● Patents <link> ● New York Times ● Library of Congress ● _____?
Cautions ● Every API is different ● Read the documentation ○ Especially: rate limits, query options ● Google for example code
Neat example with Twitter www.proporti.onl
Ethics sidebar Randall Collins. 1998. Sociology of Philosophies .
Ethics sidebar Gunter Grau. 1995. Hidden Holocaust .
Twitter example
Scraping Overview ● Sometimes, there is no API. ● “Scraping:” converting web pages to usable data
Things one might scrape ● Event information ● Policy statements ● Data tables ● Faculty lists ● Public comments or posts ○ (e.g. on legislation, news) ● _____?
Cautions 1. Use the API (if it exists) 2. Every website is different 3. Read robots.txt 4. Think seriously about ethics a. (OKC debacle, TOS, CAPCHA) 5. BE NICE (or get us all banned...) 6. Recursion is dangerous (exponential growth)
Scraping examples

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Trevor Edwards, President, NIKE Brand: Good morning. Good morning, everybody, and welcome. That

Trevor Edwards, President, NIKE Brand: Good morning. Good morning, everybody, and welcome. That was a perfect introduction, because everything we do starts with the consumer. And today, you will see just how laser-focused we are on serving the

265 views • 13 slides

Dictionaries A Good morning dictionary English: Good morning Spanish: Buenas das

Dictionaries A Good morning dictionary English: Good morning Spanish: Buenas das Swedish: God morgon German: Guten morgen Venda: Ndi matscheloni Afrikaans: Goeie mre Whats a dictionary? A dictionary is a table of items. Each

560 views • 35 slides

1 Good morning and welcome ladies and gentlemen. This morning Russ and I will provide you with a

1 Good morning and welcome ladies and gentlemen. This morning Russ and I will provide you with a brief overview of our progress over the last six months. It has been another good period for the business with us making progress against our strategy

854 views • 43 slides

from home. With such social reality, I will try to relate our efforts to address today's issues-

HOW THE UNIONS COPE WITH THE ALL ODDS IN NEPAL? Good morning Comrades; Good morning every body! This morning some of the comrades asked me where exactly Nepal locates? Before answering you, let me begin with a question, how do you recognise

554 views • 8 slides

Good morning, everyone. Thank you all for joining us today. I know that some of you had to

Good morning, everyone. Thank you all for joining us today. I know that some of you had to get up early this morning and travel to get here, and we appreciate it very much. We have some exciting news to tell you about this morning

657 views • 28 slides

Morning Presentation: Struggles of Our Life F . A: Good morning, everyone. We are from 3A. I

Morning Presentation: Struggles of Our Life F . A: Good morning, everyone. We are from 3A. I am Tinkie. B: I am Fiona. A: Today, we are going to talk about a story, Struggles of Our Life. B: Once upon a time, a daughter complained to

395 views • 11 slides

Nick Holland CEO Thank you very much. Good afternoon everybody, or good morning depending

Nick Holland CEO Thank you very much. Good afternoon everybody, or good morning depending where you are in the world today. Thank you for joining us to discuss Gold Fields results for the quarter and year ended 31 December 2015. With me

654 views • 7 slides

Mark Parker, Chairman, President & Chief Executive Officer, NIKE, Inc. Good morning. Good

Mark Parker, Chairman, President & Chief Executive Officer, NIKE, Inc. Good morning. Good morning, everybody. It's really great to have you here. I'm really glad you could join us this morning. From the beginning, our mission at NIKE has been

497 views • 10 slides

Good morning Madam Chair Elizabeth Copland

Good morning Madam Chair Elizabeth Copland ; Good morning Board Members and Staff of NIRB, President and staff of QIA, Government Agencies and the

646 views • 7 slides

MITOCW | watch?v=lbqlj1g8gu0 TOM KOCHAN: Well, good afternoon, everyone, or good morning or good

MITOCW | watch?v=lbqlj1g8gu0 TOM KOCHAN: Well, good afternoon, everyone, or good morning or good evening. No matter where you are in the world, it's some time of the day or night. And I'm delighted that we can use this time now to provide a

1.02k views • 33 slides

Good Morning to everyone.

Good Morning to everyone. Dear friends and colleagues,

292 views • 10 slides

Good morning! Good morning! Source: This Is Pawsome S ESSION I E XCELLENCE A CADEMY Excellence

E XCELLENCE A CADEMY S ESSION I B O A R E D N G A G E M E N T Excellence Academy 1 Aspen Impact W ELCOME ! C LASS OF 2022 Good morning! Good morning! Source: This Is Pawsome S ESSION I E XCELLENCE A CADEMY Excellence Academy

450 views • 29 slides

Good Morning. It has been a rich few days with you. Thank you for sharing insights, laughter,

Slide 1 Good Morning. It has been a rich few days with you. Thank you for sharing insights, laughter, conversation and depth of liturgical worship. This morning we have the immense task of talking about mental health recovery. And if that

438 views • 7 slides

Good morning / Good day! My name is Erica Vagliengo and I am an Italian journalist and web-writer,

Good morning / Good day! My name is Erica Vagliengo and I am an Italian journalist and web-writer, authoress of I want to write for Vanity Fair , an amusing novel about Emma Travet, a twenty-six years old journalist from Turin who dreams to

417 views • 4 slides

Good morning, Chairman Neufeld and committee members. Thank you for the invitation to present to

Note from ENEV Clerk: Due to cancellation of meeting, this organization is not rescheduled to appear but has asked that this presentation, prepared for their appearance, be considered their written submission to the committee. Good morning,

160 views • 13 slides

August 13, 2018 Welcome and Introductions Overview of the Day Good Morning! Wake up with This

Developing PLAAFPs and Goals using Evaluation Information. How do related services fit in the puzzle? August 13, 2018 Welcome and Introductions Overview of the Day Good Morning! Wake up with This or That! Morning, Day or Evening Cats,

709 views • 42 slides

Arrays (contd) Arrays (contd) The previous example creates

Arrays An array is a set of variables of the same type accessed by their index Arrays (Savitch, Chapter 7.1-7.2) int[] day = new int[4]; TOPICS 31 28 31 30 Array

419 views • 7 slides

MAT 137 LEC 0601 Instructor: Alessandro Malus TA: Julia Kim October 15th, 2020 Warm-up

MAT 137 LEC 0601 Instructor: Alessandro Malus TA: Julia Kim October 15th, 2020 Warm-up question : Is this correct? Consider the function f ( x ) = 1 x . We have that f ( 1) = 1 < 0 and f (1) = 1 > 0. By IVT, there exists c

85 views • 6 slides

David W. Blackwell, Ph.D. Provost We Are Whats Wildly Possible . We Are Whats Wildly

David W. Blackwell, Ph.D. Provost We Are Whats Wildly Possible . We Are Whats Wildly Possible . Transformed Spaces 1. Academic Spaces 2. Social and Living Spaces - School of Art & - Campus housing Visual Studies (2015)

446 views • 19 slides

Basics of Most C++ Programs http://cs.mst.edu 01. // Programmer: Clayton Price date:

Basics of Most C++ Programs http://cs.mst.edu 01. // Programmer: Clayton Price date: 9/4/10 02. // File: fahr2celc.cpp 03. // Purpose: this file contains the main function of the program which will input 04. // Fahrenheit

527 views • 15 slides

CS5412: LECTURE 8 Ken Birman USING TIME IN I O T APPLICATIONS Spring, 2019

CS5412: LECTURE 8 Ken Birman USING TIME IN I O T APPLICATIONS Spring, 2019 HTTP://WWW.CS.CORNELL.EDU/COURSES/CS5412/2019SP 1 CONTEXT: QUICK REMINDER Real-time clocks have limitations on precision, accuracy and skew. Lamport defines the

476 views • 43 slides

Release management in Debian - Can we do better? Frans Pop FOSDEM 2009, Brussels Frans Pop

Release Management Stable release Management The etch-and-a-half release The Lenny release Release management in Debian - Can we do better? Frans Pop FOSDEM 2009, Brussels Frans Pop Release management in Debian - Can we do better? Release

194 views • 18 slides

Overall framework of decommissioning procedures Nov. 2013 201311 (Start of spent fuel

Efforts for Procedures for Decommissioning and Contaminated Water Management Decommissioning at Tokyo Electric Power Company (TEPCO)'s Fukushima Daiichi NPS and Progress Overall framework of decommissioning procedures Nov. 2013 201311

436 views • 6 slides

Can Partnerships Help Us Address the New Normal? Union of BC Municipalities September

Can Partnerships Help Us Address the New Normal? Union of BC Municipalities September 23, 2019 George Abbott, PhD. Government has been getting plenty of advice on fire, flood and climate change in recent years. The Challenge

487 views • 47 slides