Creating Data A Boat Filled With Sauerkraut @lukasvermeer Lukas - - PowerPoint PPT Presentation

Creating Data

A Boat Filled With Sauerkraut

Lukas Vermeer

Experiments at Booking.com

SAUERKRAUT

Introduction

HMS Endeavour

Our voyage starts when a ship sails from Plymouth on 26 August 1768.

• Wikipedia

“Provisions loaded at the outset of the voyage included 6,000 pieces of pork and 4,000 of beef, nine tons of bread, five tons of flour, three tons

• f sauerkraut, one ton of raisins

and sundry quantities of cheese, salt, peas, oil, sugar and oatmeal.”

REVOLUTIONS

Chapter 1

• Thomas Kuhn

“Progress in science is not a simple line leading to the truth.”

Aristarchus of Samos

• c. 310 BC – c. 230 BC

• Archimedes

“[Aristarchus’] hypotheses are that the fixed stars and the Sun remain unmoved, that the Earth revolves about the Sun on the circumference

• f a circle, […]”

• George Pólya

“His fame rests on his heliocentric

• theory. […] Perhaps ‘theory’ is too

strong a word, for his proofs were weak; yet it was a great idea.”

Claudius Ptolemy

• c. 100 AD – c. 170 AD

• Claudius Ptolemy

“But it has escaped [heliocentric proponents’] notice in the light of what happens around us in the air that such a notion would seem altogether absurd.”

• Claudius Ptolemy

“For the earth would always

• utstrip them in its eastward

motion, so that all other bodies would seem to be left behind and to move towards the west.”

• Claudius Ptolemy

No westward motion. No stellar parallax. Geocentric math works. QED

• Claudius Ptolemy

No observed westward motion. No observed stellar parallax. Geocentric math works to explain what we have observed. GED

Nicolaus Copernicus

1473 – 1543

Dē RevolutionibusOrbiumCoelestium

Copernicus's vision of the universe, published in 1543, the year of his death, though he had formulated the theory several decades earlier.

• Andreas Osiander

“These hypotheses need not be true nor even probable. On the contrary, if they provide a calculus consistent with the observations, that alone is enough.”

Galileo Galilei

1564 – 1642

• Michael Fowler

“The real breakthrough that ultimately led to the acceptance of Copernicus’ theory was due to Galileo, but was actually a technological rather than a conceptual breakthrough.”

Galileo did not invent the idea. He built a better telescope.

Galileo first observed the moons of Jupiter

This observation upset the notion that all celestial bodies revolve around the Earth. Galileo published a full description in March 1610.

Multiple models could probably explain the data you already have. Determining which one is closer to the truth requires a directed effort to collect new data (to the contrary).

Data You

Have

Data You

Need

Data You

Have

Data You

Need

Sauerkraut Science

TRANSIT

Chapter 2

“On the sizes and distances”

Aristarchus's 3rd-century BC calculations on the relative sizes of (from left) the Sun, Earth and Moon, from a 10th-century AD Greek copy.

Aristarchus (3rd century BC)

Distance to the sun

380 - 1520

Earth Radii

Diagram from Edmund Halley's 1716 paper

Addressed to the Royal Society showing how the Venus transit could be used to calculate the distance between the Earth and the Sun.

Route of the first voyage of James Cook

An expedition to the south Pacific Ocean aboard HMS Endeavour, from 1768 to 1771. It was the first of three Pacific voyages of which Cook was the commander.

Three years of travel. For two timestamps.

• James Cook

“Not a Clowd was to be seen the Whole day and the Air was perfectly clear, so that we had every advantage we could desire in Observing the whole of the passage of the Planet Venus over the Suns disk.”

The "black drop effect"

As recorded during the 1769 transit by James Cook.

Right place, right time, right idea. Insufficiently accurate telescope.

Jérôme Lalande (1771)

Distance to the sun

24 000

Earth Radii

Science is limited by data. Data is limited by engineering.

• Elon Musk

“In the absence of the engineering, you do not have the data. You just hit a limit. You can be real smart within the context of the limit of the data you have, but unless you have a way to get more data, you can’t make progress.”

Data You

Have

Data You

Need

Have

Need

Data You

COULD CREATE

TRANSMUTATION

Chapter 3

• Michael Palmer

“Data is just like crude. It’s valuable, but if unrefined it cannot really be

• used. It has to be changed into gas,

plastic, chemicals, etc to create a valuable entity that drives profitable activity; so must data be broken down, analyzed for it to have value.”

• Wikipedia

“The philosopher's stone is a legendary alchemical substance capable of turning base metals such as mercury into gold or silver.”

Data Science. versus Data Alchemy.

Kaggle.com

“Your home for data science”.

Something good, something bad

Hotel reviews on Booking.com.

Sentiment Analysis

Excerpt from “Entity Based Sentiment Analysis on Twitter” by Siddharth Batra and Deepak Rao (Stanford University).

Kaggle is to real-life machine learning as chess is to war Intellectually challenging and great mental exercise, but YOU DON'T KNOW, MAN! YOU WEREN'T THERE!

Sentiment Analysis

At Booking.com, we solve the sentiment analysis challenge at data collection time.

Have

Need

Data You

COULD CREATE

Have

Need

Data You

COULD CREATE

Delete or archive?

Have

Need

Data You

COULD CREATE

Keep, or recreate?

Have

Need

Data You

COULD CREATE

Proxies?

Deciding which data to collect, and how, is a fundamental step in the scientific method. Limited both by available theories and engineering.

Some of us are philosophers. Some of us build telescopes.

• Voltaire

“Judge a man by his questions rather than by his answers.”

EXPERIMENTING WITH SCURVY

Appendix A

COOK’S SECRET SECOND OBJECTIVE

Appendix B

Terra Australis Nondum Cognita

1570 map by Abraham Ortelius depicting a large continent on the bottom of the map and also an Arctic continent.

Route of the first voyage of James Cook

An expedition to the south Pacific Ocean aboard HMS Endeavour, from 1768 to 1771. It was the first of three Pacific voyages of which Cook was the commander.

A new map of the world

With Captain Cook's tracks, his discoveries and those of the other

• circumnavigators. Published in 1800 by W. Palmer.

• Matthew Flinders (1814)

“There is no probability, that any

• ther detached body of land, of

nearly equal extent, will ever be found in a more southern latitude”