Geoapplications development http://rgeo.wikience.org Higher School - - PowerPoint PPT Presentation

geoapplications development http rgeo wikience org
SMART_READER_LITE
LIVE PREVIEW

Geoapplications development http://rgeo.wikience.org Higher School - - PowerPoint PPT Presentation

Geoapplications development http://rgeo.wikience.org Higher School of Economics, Moscow, www.cs.hse.ru 2 Agenda What is geospatial data? How is it different from other types of data? Examples of geospatial data What is


slide-1
SLIDE 1

Geoapplications development http://rgeo.wikience.org

Higher School of Economics, Moscow, www.cs.hse.ru

slide-2
SLIDE 2

Agenda

2

  • What is geospatial data?
  • How is it different from other types of data?
  • Examples of geospatial data
  • What is geoapplication?
  • Course goals & organization
  • Real-world examples: companies using geospatial data
  • Tentative projects to participate in
  • Main readings
  • Quiz
  • Practical lesson: 3D Globes
slide-3
SLIDE 3

Geospatial data: 2 types (simplified)

3

VECTOR

What is raster data? http://help.arcgis.com/EN/arcgisdesktop/10.0/help/index.html#//009t00000002000000

RASTER matrix

slide-4
SLIDE 4

The main difference from other data types (simplified)

4

RASTER matrix

A fundamental difference: the presence of spatial coordinates

each cell has (lat, lon, value)

(lat_i, lon_i) for each vertice

VECTOR

slide-5
SLIDE 5

Is that all?

5

slide-6
SLIDE 6

is there something strange here??

6

Hint: areas…

slide-7
SLIDE 7

is Greenland larger than Africa??

7

Let us consult an Encyclopedia or Wikipedia

2,175,600 km2 World size ranking: 14 30,370,000 km2 World size ranking: 2

slide-8
SLIDE 8

is Greenland larger than Africa??

8

https://takeitscienceblog.wordpress.com/2016/08/31/how-big-is-your-country/ http://geoffboeing.com/2015/08/map-projections-that-lie/

slide-9
SLIDE 9

Distances

9

Two airplanes are flying … Plane A: (60ᵒN, 40ᵒE) → (60ᵒN, 30ᵒE) Plane B: (40ᵒN, 40ᵒE) → (40ᵒN, 30ᵒE) 40ᵒ - 30ᵒ = 10ᵒ Both traveled the same distance?

slide-10
SLIDE 10

Everything is not so simple…

10

To name a few: * distances between two points with the same longitudes are different for different latitudes

slide-11
SLIDE 11

A small caveat: circular/periodic axes

11

longitude range is [ –180 … +180 ] but what if I want to move 10 degrees eastward from 179 degrees? 179 + 10 = 189 …. actually -171

slide-12
SLIDE 12

Lesson learned

12

Areas, shapes, angles, distances depend on…

slide-13
SLIDE 13

Examples of geospatial data: vector

13

slide-14
SLIDE 14

Geo prefix

14

→ → → → → →

slide-15
SLIDE 15

Examples of geospatial data: raster & vector

15

More details are at: http://www.wikience.org/tropical-cyclones/wind-swaths/

slide-16
SLIDE 16

Examples of geospatial data: satellite raster

16

slide-17
SLIDE 17

Visualization examples

17

Much more examples are at: http://goworldwind.org/demos/

Video on terrain Analytic Surface Airspaces Extruded Shapes Scankort Denmark Data We can do more together during course work

NASA World Wind engine

slide-18
SLIDE 18

18

Satellites, smartphones, GPS (1): basic theory: geographic coordinate systems, geocoding, vector and raster data types and operations (3): I/O with vector (SHP, KML, GeoJSON, WKT) and raster (GeoTIFF, NetCDF, HDF) formats (4): network protocols for geodata exchange: WCS, WFS, WMS, OPeNDAP (4): frameworks: Mina, Netty that is used by Twitter (3,4,5): spatial databases and servers: PostGIS, SciDB, GeoServer, TDS (5): software libraries for geodata processing: vector Java Topology Suite and raster ImageMagic, GDAL (6): visualizing geodata with NASA WorldWind, Mapbox, Leaflet, CartoDB

Course goals & organization

Topics cover technologies for a complete geospatial data management stack:

slide-19
SLIDE 19

19

Course assessment

Code Description Contribution, % LP Lecture presence 20 PP Work during practical lessons 5 CW Control work 10 HA1 Home assignment 1 30 HA2 Home assignment 2 15 EX Exam 20 TT Total contribution

slide-20
SLIDE 20

Geospatial technology stack (not exhaustive)

20

Storage Transmission Processing Visualization DBMS Files Almost all DBMS are geo-enabled

SHP, KML, GeoJSON, GeoTIFF, NetCDF, HDF network protocols WCS, WFS, WMS, OPeNDAP

Libraries in Java, Python, R, JavaScript, …

slide-21
SLIDE 21

Geospatial theory stack (not exhaustive)

21

Notations Topology Processing GPS, WiFi, geohash, … Projections

ISO/IEC standard

slide-22
SLIDE 22

The technologies and theory are building blocks for solving real-world tasks

22

slide-23
SLIDE 23

The technologies and theory are building blocks for solving real-world tasks

23

Example task: Because of dramatic traffic increase, the town planners want to widen the main road and add a second lane. The lane width is 20 meters. Find properties that must be demolished. main road

new lane (its central line)

house tree

slide-24
SLIDE 24

The solution (DBMS, WKT, Buffer, Topology, SQL, …)

24

LINESTRING (290 230, 330 250, 390 260, 440 260, 495 252, 535 238, 560 220, 580 190, 590 150)

main road house tree

new lane (its central line)

DBMS

SELECT a_geom, address, owner FROM city_database WHERE ST_Intersects ( ST_Buffer( ST_GeomFromText("LINESTRING (290 230,…) ",4326), 10), a_geom)

POLYGON ((325.5278640450004 258.94427190999915, 326.90755828109235, ….))

BUFFER (ST_Buffer)

WKT: ISO/IEC standard

slide-25
SLIDE 25

Consumers of geospatial data

25

Who needs geospatial data? How do they process them? What kinds of geospatial data exist?

slide-26
SLIDE 26

Consumers of geospatial data: motivating examples

26

http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf

slide-27
SLIDE 27

Consumers of geospatial data: motivating examples

27

http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf

slide-28
SLIDE 28

Consumers of geospatial data: motivating examples

28

http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf

slide-29
SLIDE 29

Consumers of geospatial data: motivating examples

29

http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf

slide-30
SLIDE 30

Consumers of geospatial data: motivating examples

30

http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf

slide-31
SLIDE 31

Geoapps: Marine Traffic

31

Real-time ship tracking http://www.marinetraffic.com http://shipfinder.co/ https://www.vesselfinder.com/

slide-32
SLIDE 32

Geoapps: FlightRadar

32

www.flightradar24.com

slide-33
SLIDE 33

EigenRisk

33

Hurricane swaths, flood footprints, assets Spatial operations Visualization

Hurricane Katrina, 2005

slide-34
SLIDE 34

Zillow

34

Selling real estates Filter, nearby, etc.

slide-35
SLIDE 35

Examples of geoapplications – 2GIS

35

Places Positioning Routes ….

slide-36
SLIDE 36

Yelp

36

Seek restaurants, hairdressing saloons, etc. + add other

slide-37
SLIDE 37

UrtheCast

37

Live video streaming from International Space Station A platform for imagery processing

slide-38
SLIDE 38

Sputnix, Dauria – Skolkovo residents

38

Native satellite manufacturers – ensure Russian national interests

Д.К. Мозговой и О.В. Кравец «Использование данных MODIS для экологического мониторинга и контроля чрезвычайных ситуаций»

Blooming in Каховском водохранилище

Precision agriculture

slide-39
SLIDE 39

UAV: Unmanned Aerial Vehicles

http://www.youtube.com/watch?v=4E rEBkj_3PY

slide-40
SLIDE 40

Consumer:

40

Optimize wind farms locations and operation Data: wind speed and direction reanalysis and forecasts – raster data

For calendar year 2014, the electricity produced from wind power in the United States amounted to 181.79 terawatt-hours, or 4.44% of all generated electrical

  • energy. https://en.wikipedia.org/wiki/Wind_power_in_the_United_States
slide-41
SLIDE 41

Wind speed data

41

Energy sector, hazard monitoring, etc.

https://earth.nullschool.net http://wikience.org

slide-42
SLIDE 42

Thermal power stations

42

Optimize load depending on climate/weather forecast

  • > reduce penalties

Air temperature is

  • ne of the crucial

factors upon which the station load depends

slide-43
SLIDE 43

Railroads

43

Qingyun (Jeffrey) Xie, High Performance Raster Database Manipulation and Data Processing with Oracle Spatial and Graph

slide-44
SLIDE 44

More examples

44

slide-45
SLIDE 45

Our Projects

45

ChronosDB – a new DBMS

Michael Stonebraker PostgreSQL, Vertica, Streambase, Illustra, VoltDB, IEEE John von Neumann Medal (2005) ACM Turing Award (2014)

ChronosDB

  • utperforms SciDB by

up to 75× on average

R.A. Rodriges Zalipynis. ChronosDB: Distributed, File Based, Geospatial Array DBMS

Rio de Janeiro, Brazil 27 – 31 August 2018

VLDB (Very Large Data Bases) – a top conference on Big Data, Distributed Systems, Databases

2018: the only talk from Russia 10 years ago: the previous talk from Russia

SciDB University of California, Berkeley, University of Michigan, Massachusetts Institute of Technology

https://db-engines.com/en/ranking

slide-46
SLIDE 46

Moscow & Rio

46

slide-47
SLIDE 47

Corcovado

47

The photo is not mine

slide-48
SLIDE 48

www.climate.wikience.org Tentative course work topics

JavaScript: Web GIS (Geographical Information Service) Data compression, processing, visualization, user interaction

slide-49
SLIDE 49

Performance evaluation Tentative course work topics

performance evaluation We will develop distributed & multithreaded application to simulate a large load

DASK GeoTrellis RasDaMan SciDB ….

slide-50
SLIDE 50

Tentative course work topics: machine learning

https://habrahabr.ru/company/avito/blog/325632/ https://habrahabr.ru/company/ods/blog/325096/

slide-51
SLIDE 51

Main readings

51

http://www.spatial.cs.umn.edu/Book/

slide-52
SLIDE 52

Web sites

52

http://flowingdata.com/

slide-53
SLIDE 53

Web sites

53

http://geoawesomeness.com/

slide-54
SLIDE 54