Ba Basi sic Data Visu sualization 01219335 Data Acquisition and - - PowerPoint PPT Presentation

▶
ba basi sic data visu sualization
SMART_READER_LITE
LIVE PREVIEW

Ba Basi sic Data Visu sualization 01219335 Data Acquisition and - - PowerPoint PPT Presentation

Sep 15, 2023 383 likes •678 views

Ba Basi sic Data Visu sualization 01219335 Data Acquisition and Integration Chaipo Chaiporn J n Jaik aikae aeo De Department of f Computer Engineering Kasetsart Unive versity Revised 2020-11-04 Ou Outline Visualization design

slide-1
SLIDE 1

Ba Basi sic Data Visu sualization

Chaipo Chaiporn J n Jaik aikae aeo De Department of f Computer Engineering Kasetsart Unive versity

01219335 Data Acquisition and Integration

Revised 2020-11-04

slide-2
SLIDE 2

2

Ou Outline

  • Visualization design and consideration
  • Server-side vs. client-side visualization
  • Interactive visualization examples
  • Visualizing geographical information
slide-3
SLIDE 3

Survey & Questionnaires Survey & Questionnaires

IoT Devices IoT Devices

Web APIs Web APIs Web Sites Web Sites

Data Acquisition and Integration Framework

Survey & Questionnaires

Web Sites Web APIs

Data Integration

Web API

raw files / web scraping

  • ther apps/

developers

Data Acquisition Devices

MQTT

Data Visualization

general public policy makers

slide-4
SLIDE 4

4

Impo Importanc nce o e of D f Data Vi Visua sualization

  • It is difficult for human brains to interpret large numbers
  • Visualization helps communicate quantitive contents more

effectively by means of abstraction and visual representation

"A picture is worth a thousand words"

slide-5
SLIDE 5

5

Visualiz Visualizatio ion C n Consider nsideratio ion

  • Good data visualization should be
  • Telling a story
  • Answering certain questions
  • Visually appealing
  • Never misleading

https://paragraft.wordpress.com/2008/06/03/the-chart-junk-of-steve-jobs/

slide-6
SLIDE 6

https://extremepresentation.typepad.com/blog/2006/09/choosing_a_good.html

slide-7
SLIDE 7

7

Wh Where to Generate Visualizati tion

  • Server-side generation
  • The server generates visualizations in forms of images (e.g., PNG, PDF, SVG)
  • The client, i.e., web browser, displays images on screen
  • Client-side generation
  • The server only provides data for generating visualizations, preferably via API
  • The client generates visualizations from the data using some client-side

visualization libraries, mostly written in JavaScript

HTTP web browser (client) web server

slide-8
SLIDE 8

8

Se Server-Sid Side G Generated V Vis isualiz alization ion

  • Lots of server-side libraries and languages
  • Computationally expensive for the server
  • Difficult to add interactivity
  • Suitable for statically generated and cached images
  • Images can be generated in advance

HTTP

web server web browser data image

png, pdf, svg, etc.

slide-9
SLIDE 9

9

Cl Client-Sid Side G Generated V Vis isualiz alization ion

  • Less load on the server
  • Also many (free) JavaScript libraries readily available
  • Flexible and responsive user interaction

HTTP

web server web browser data

drawing csv, json, xml, etc. JavaScript code

slide-10
SLIDE 10

10

Som Some V Vis isualiz alization ion J JS Lib S Librar arie ies

  • Google Charts
  • Chart.js
  • Plotly.js
  • Highcharts.js – free to use for non-profit work
  • D3.js – very powerful with steep learning curve
  • Many many more…
slide-11
SLIDE 11

11

Ex Example: ample: Ra Rainfall API Visualization

  • Clone/download example code from
  • https://gitlab.com/cjaikaeo/rain-vis-daq-2020f
  • Root path is /rain-api/v3
  • Provides geometry boundary for each basin
  • All REST endpoints and links have already been

implemented

slide-12
SLIDE 12

12

St Star artin ing R g REST A API PI Se Server

  • Inside the project folder, create and activate a virtual

environment

  • Generate code from the OAS
  • Install required libraries

java -jar openapi-generator-cli-4.3.1.jar generate \

  • i openapi/rain-api.yaml -o autogen -g python-flask

python3.9 -m venv env . env/bin/activate # macOS and Linux env\Scripts\activate.bat # Windows pip install -r requirements.txt May type them all in one line without the \ character For Windows’ command prompt, use the ^ character instead of \ to continue on the second line

slide-13
SLIDE 13

13

St Star artin ing g Gr GraphQL aphQL Se Server

  • Create config.py from config.py.example
  • Start the REST API server
  • Optionally test the API at http://localhost:8080/rain-api/v3/ui
  • In another terminal, run openapi-to-graphql with CORS

(Cross-Origin Resource Sharing) enabled

  • Open the page http://localhost:3000/graphql
  • The browser should display GraphQL window

python app.py

  • penapi-to-graphql --cors -u http://localhost:8080/rain-api/v3 openapi/rain-api.yaml
slide-14
SLIDE 14

14

Op Opening g Index Page

  • Open the file html/index.html with your web browser
slide-15
SLIDE 15

15

Ex Example: ample: Av Average Monthly Rainfalls

  • Source code: html/monthly-avg.html
slide-16
SLIDE 16

16

Av Average Monthly Rainfalls: Code (1)

  • HTML document template

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title>Average Monthly Rainfalls</title> <script src="https://cdn.plot.ly/plotly-latest.min.js"></script> <body> <div id="chart" style="width:100%;height:75vh;"></div> <script> // Plotting script goes here // : </script> </body> </html>

Load Plotly Javascript library Create a div for the chart's placeholder

slide-17
SLIDE 17

17

Av Average Monthly Rainfalls: Code (2)

  • Fetching data using GraphQL API

var resp = await fetch('http://localhost:3000/graphql', { method: 'POST', headers: { 'Content-Type': 'application/json', 'Accept': 'application/json', }, body: JSON.stringify({ query: ` { basin(basinId: 3) { name avgMonthlyRainfalls { amount month } } }` }) });

Fetch only Ping river's data (for now)

slide-18
SLIDE 18

18

Av Average Monthly Rainfalls: Code (3)

  • Transform the JSON result into another format expected by

Plotly

var json = await resp.json(); var table = json.data; var data = [{ x: table.basin.avgMonthlyRainfalls.map(row => row.month), y: table.basin.avgMonthlyRainfalls.map(row => row.amount), type: 'line', name: table.basin.name, }];

{ "data": { "basin": { "name": "Ping", "avgMonthlyRainfalls": [ { "amount": 7.31, "month": "Jan" }, { "amount": 8.46, "month": "Feb" }, { "amount": 25.01, "month": "Mar" }, { "amount": 55.14, "month": "Apr" }, { "amount": 177.9, "month": "May" }, { "amount": 130.2, "month": "Jun" }, { "amount": 127.25, "month": "Jul" }, { "amount": 167.59, "month": "Aug" }, { "amount": 209.1, "month": "Sep" }, { "amount": 153.51, "month": "Oct" }, { "amount": 36.7, "month": "Nov" }, { "amount": 8.76, "month": "Dec" } ] } } }

API's result Plotly's format

slide-19
SLIDE 19

19

Av Average Monthly Rainfalls: Code (4)

  • Configure plot's layout and create the plot
  • Available configurable layout parameters can be found from
  • https://plotly.com/javascript/reference/layout/

var layout = { title: 'Average Monthly Rainfalls for Ping River Basin', xaxis: { title: 'Month' }, yaxis: { title: 'Monthly Rainfall (mm)', range: [0,500] } }; var config = { responsive: true }; Plotly.newPlot("chart", data, layout, config); }

slide-20
SLIDE 20

20

Visualiz Visualizing ing Geo Geographic aphical D al Data

  • Geometry representation formats
  • Well-known text (WKT)
  • Well-known binary (WKB)
  • GeoJSON
  • The rain database contains the table basin_geom storing

basin geometries in WKB format

  • Plotly supports visualization of GeoJSON data over various

map layers

  • The wellknown.js library can convert data between GeoJSON

and WKT

slide-21
SLIDE 21

21

Ex Example: ample: Ba Basi sin/Station Ma Map

  • Source code: html/stations.html
slide-22
SLIDE 22

22

Ba Basi sin/Station Ma Map: Co Code (1)

  • HTML document template

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title>Station Locations</title> <script src="https://cdn.plot.ly/plotly-latest.min.js"></script> <script src="https://cdn.rawgit.com/mapbox/wellknown/master/wellknown.js"></script> <body> <div id="chart" style="width:100vw;height:100vh;"></div> <script> // Plotting script goes here // : </script> </body> </html>

Load wellknown.js for WKT à GeoJSON conversion

slide-23
SLIDE 23

23

Ba Basi sin/Station Ma Map: Co Code (2)

  • Fetching data using GraphQL API

var resp = await fetch('http://localhost:3000/graphql', { method: 'POST', headers: { 'Content-Type': 'application/json', 'Accept': 'application/json', }, body: JSON.stringify({ query: ` { basin(basinId:3) { geom stations { name lat lon } } }` }) });

Fetch only Ping river's data (for now)

slide-24
SLIDE 24

24

Ba Basi sin/Station Ma Map: Co Code (3)

  • Transform API's JSON result to Plotly format for the station

location data

var json = await resp.json(); var table = json.data; var data = [ { type: "scattermapbox", text: table.basin.stations.map(row => row.name), lon: table.basin.stations.map(row => row.lon), lat: table.basin.stations.map(row => row.lat), marker: { color: "black", size: 10 }, }, ];

Utilize integrated Mapbox GL JS library

slide-25
SLIDE 25

25

Ba Basi sin/Station Ma Map: Co Code (4)

  • Configure a layout with Open Street Map as the base map

and basin's geometry as an overlay layer

var layout = { dragmode: "zoom", showlegend: false, mapbox: { style: "open-street-map", center: { lat: 17.16, lon: 99.86 }, zoom: 6, layers: [ { source: { type: "Feature", geometry: wellknown.parse(table.basin.geom) }, type: "fill", color: "red", opacity: 0.2, }, ] }, margin: { r: 0, t: 0, b: 0, l: 0 } }; var config = { responsive: true }; Plotly.newPlot("chart", data, layout, config);

Convert WKT to GeoJSON

Use Mapbox with OpenStreetMap as the back-end tilemap (no access token required)

slide-26
SLIDE 26

26

Co Conclusi sion

  • Visualization helps communicate quantitative contents in

abstract and visual ways

  • Visualization can be generated at the server side as images,
  • r at the client side using JavaScript code
  • Various open-source libraries are available for client-side

visualization

slide-27
SLIDE 27

27

Furt Further R her Rea eadi ding ng

  • Data Visualization – Best Practices and Foundations by

Mayra Magalhaes Gomes

  • https://www.toptal.com/designers/data-visualization/data-

visualization-best-practices

  • Plotly JavaScript Open-Source Graphing Library
  • https://plotly.com/javascript/
slide-28
SLIDE 28

28

As Assignment 10. 10.1: : Mon

  • nthly

ly Rain ainfalls alls

  • Modify html/monthly-avg.html so that it displays

monthly average rainfalls for all the five basins

slide-29
SLIDE 29

29

As Assignment 10. 10.2: : Ba Basin sin/St Station tion Map

  • Modify html/map.html so that it displays basin

boundary and station map for all the five basins