Ba Basi sic Data Visu sualization 01219335 Data Acquisition and - PowerPoint PPT Presentation

Ba Basi sic Data Visu sualization 01219335 Data Acquisition and Integration Chaipo Chaiporn J n Jaik aikae aeo De Department of f Computer Engineering Kasetsart Unive versity Revised 2020-11-04

Ou Outline • Visualization design and consideration • Server-side vs. client-side visualization • Interactive visualization examples • Visualizing geographical information 2

Data Acquisition and Integration Framework Survey & Survey & Survey & Questionnaires raw files / Questionnaires Questionnaires Web Sites web scraping Web Sites Web Sites other apps/ developers Data Web API Web APIs Web APIs Web APIs Integration general public Data Data IoT Devices MQTT IoT Devices Visualization Acquisition Devices policy makers

Impo Importanc nce o e of D f Data Vi Visua sualization "A picture is worth a thousand words" • It is difficult for human brains to interpret large numbers • Visualization helps communicate quantitive contents more effectively by means of abstraction and visual representation 4

Visualiz Visualizatio ion C n Consider nsideratio ion • Good data visualization should be ◦ Telling a story ◦ Answering certain questions ◦ Visually appealing ◦ Never misleading https://paragraft.wordpress.com/2008/06/03/the-chart-junk-of-steve-jobs/ 5

https://extremepresentation.typepad.com/blog/2006/09/choosing_a_good.html

Wh Where to Generate Visualizati tion • Server-side generation ◦ The server generates visualizations in forms of images (e.g., PNG, PDF, SVG) ◦ The client, i.e., web browser, displays images on screen • Client-side generation ◦ The server only provides data for generating visualizations, preferably via API ◦ The client generates visualizations from the data using some client-side visualization libraries, mostly written in JavaScript web browser web server (client) HTTP 7

Se Server-Sid Side G Generated V Vis isualiz alization ion • Lots of server-side libraries and languages • Computationally expensive for the server • Difficult to add interactivity • Suitable for statically generated and cached images ◦ Images can be generated in advance image png, pdf, svg, etc . HTTP data web browser web server 8

Cl Client-Sid Side G Generated V Vis isualiz alization ion • Less load on the server • Also many (free) JavaScript libraries readily available • Flexible and responsive user interaction JavaScript code csv, json, xml, etc . drawing HTTP data web browser web server 9

Som Some V Vis isualiz alization ion J JS Lib S Librar arie ies • Google Charts • Chart.js • Plotly.js • Highcharts.js – free to use for non-profit work • D3.js – very powerful with steep learning curve • Many many more… 10

Ex Example: ample: Ra Rainfall API Visualization • Clone/download example code from ◦ https://gitlab.com/cjaikaeo/rain-vis-daq-2020f • Root path is /rain-api/v3 ◦ Provides geometry boundary for each basin • All REST endpoints and links have already been implemented 11

St Star artin ing R g REST A API PI Se Server • Inside the project folder, create and activate a virtual environment python3.9 -m venv env . env/bin/activate # macOS and Linux env\Scripts\activate.bat # Windows • Generate code from the OAS java -jar openapi-generator-cli-4.3.1.jar generate \ -i openapi/rain-api.yaml -o autogen -g python-flask May type them all in one line For Windows’ command prompt, use the ^ character without the \ character instead of \ to continue on the second line • Install required libraries pip install -r requirements.txt 12

Star St artin ing g Gr GraphQL aphQL Se Server • Create config.py from config.py.example • Start the REST API server python app.py ◦ Optionally test the API at http://localhost:8080/rain-api/v3/ui • In another terminal, run openapi-to-graphql with CORS (Cross-Origin Resource Sharing) enabled openapi-to-graphql --cors -u http://localhost:8080/rain-api/v3 openapi/rain-api.yaml • Open the page http://localhost:3000/graphql ◦ The browser should display GraphQL window 13

Opening Op g Index Page • Open the file html/index.html with your web browser 14

Ex Example: ample: Av Average Monthly Rainfalls • Source code: html/monthly-avg.html 15

Av Average Monthly Rainfalls: Code (1) • HTML document template <!DOCTYPE html> <html lang="en"> <head> Load Plotly Javascript library <meta charset="utf-8"> <title>Average Monthly Rainfalls</title> <script src="https://cdn.plot.ly/plotly-latest.min.js"></script> <body> <div id="chart" style="width:100%;height:75vh;"></div> <script> // Plotting script goes here Create a div for the chart's placeholder // : </script> </body> </html> 16

Av Average Monthly Rainfalls: Code (2) • Fetching data using GraphQL API var resp = await fetch('http://localhost:3000/graphql', { method: 'POST', headers: { 'Content-Type': 'application/json', 'Accept': 'application/json', }, Fetch only Ping river's data body: JSON.stringify({ (for now) query: ` { basin(basinId: 3) { name avgMonthlyRainfalls { amount month } } }` }) }); 17

Average Monthly Rainfalls: Code (3) Av • Transform the JSON result into another format expected by Plotly { var json = await resp.json(); "data": { var table = json.data; "basin": { "name": "Ping", "avgMonthlyRainfalls": [ { "amount": 7.31, "month": "Jan" }, var data = [{ { "amount": 8.46, "month": "Feb" }, { "amount": 25.01, "month": "Mar" }, x: table.basin.avgMonthlyRainfalls.map(row => row.month), { "amount": 55.14, "month": "Apr" }, y: table.basin.avgMonthlyRainfalls.map(row => row.amount), { "amount": 177.9, "month": "May" }, { "amount": 130.2, "month": "Jun" }, type: 'line', { "amount": 127.25, "month": "Jul" }, { "amount": 167.59, "month": "Aug" }, name: table.basin.name, { "amount": 209.1, "month": "Sep" }, { "amount": 153.51, "month": "Oct" }, }]; { "amount": 36.7, "month": "Nov" }, { "amount": 8.76, "month": "Dec" } ] } } } API's result Plotly's format 18

Av Average Monthly Rainfalls: Code (4) • Configure plot's layout and create the plot • Available configurable layout parameters can be found from ◦ https://plotly.com/javascript/reference/layout/ var layout = { title: 'Average Monthly Rainfalls for Ping River Basin', xaxis: { title: 'Month' }, yaxis: { title: 'Monthly Rainfall (mm)', range: [0,500] } }; var config = { responsive: true }; Plotly.newPlot("chart", data, layout, config); } 19

Visualiz Visualizing ing Geo Geographic aphical D al Data • Geometry representation formats ◦ Well-known text (WKT) ◦ Well-known binary (WKB) ◦ GeoJSON • The rain database contains the table basin_geom storing basin geometries in WKB format • Plotly supports visualization of GeoJSON data over various map layers ◦ The wellknown.js library can convert data between GeoJSON and WKT 20

Ex Example: ample: Ba Basi sin/Station Ma Map • Source code: html/stations.html 21

Basi Ba sin/Station Ma Map: Co Code (1) • HTML document template <!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title>Station Locations</title> <script src="https://cdn.plot.ly/plotly-latest.min.js"></script> <script src="https://cdn.rawgit.com/mapbox/wellknown/master/wellknown.js"></script> <body> <div id="chart" style="width:100vw;height:100vh;"></div> <script> Load wellknown.js for // Plotting script goes here // : WKT à GeoJSON conversion </script> </body> </html> 22

Ba Basi sin/Station Ma Map: Co Code (2) • Fetching data using GraphQL API var resp = await fetch('http://localhost:3000/graphql', { method: 'POST', headers: { 'Content-Type': 'application/json', 'Accept': 'application/json', }, Fetch only Ping river's data body: JSON.stringify({ (for now) query: ` { basin(basinId:3) { geom stations { name lat lon } } }` }) }); 23

Ba Basi sin/Station Ma Map: Co Code (3) • Transform API's JSON result to Plotly format for the station location data var json = await resp.json(); var table = json.data; Utilize integrated var data = [ Mapbox GL JS library { type: "scattermapbox", text: table.basin.stations.map(row => row.name), lon: table.basin.stations.map(row => row.lon), lat: table.basin.stations.map(row => row.lat), marker: { color: "black", size: 10 }, }, ]; 24