Big Data with ADAMS Big Data with ADAMS What the heck is ADAMS? - - PowerPoint PPT Presentation

▶

Jan 06, 2023 483 likes •674 views

Big Data with ADAMS Big Data with ADAMS What the heck is ADAMS? Peter Reutemann What is ADAMS? Java, GPLv3 Data mining: MOA, WEKA, MEKA, R Spreadsheets and databases Image and video processing Visualizations (plots, GIS)

SLIDE 1

Peter Reutemann

Big Data with ADAMS Big Data with ADAMS

What the heck is ADAMS?

SLIDE 2

10/08/2015 Peter Reutemann 2 of 18

What is ADAMS?

Java, GPLv3
Data mining: MOA, WEKA, MEKA, R
Spreadsheets and databases
Image and video processing
Visualizations (plots, GIS)
Scripting via Jython and Groovy
...

SLIDE 3

10/08/2015 Peter Reutemann 3 of 18

Flow

Operators are called “actors”
Actors arranged in tree, no connections
Actor “handlers” nest other actors
e.g., sequence of actors
Control actors control data flow
e.g., branch, tee, if-then-else, switch
Input/output defines
standalone , source , transformer , sink

SLIDE 4

10/08/2015 Peter Reutemann 4 of 18

Flow (2)

Tree only supports 1-to-n connections
Simulating n-to-m semantics
Containers
Variables
Internal storage
Callable actors

SLIDE 5

10/08/2015 Peter Reutemann 5 of 18

Examples

Output file to read Read file Set class attribute Apply filter Display data

Load dataset, apply filter and display dataset

Execute nested actors one after the other

SLIDE 6

10/08/2015 Peter Reutemann 6 of 18

Examples (2)

Generate data stream Feed data into branches 1st sequence of steps 2nd sequence of steps Apply stream filter Evaluate classifier Filter measurement of interest Generate data for plot Apply different stream filter Plot

Filter data stream in two separate branches with different filters, evaluate classifier and plot metric

SLIDE 7

10/08/2015 Peter Reutemann 7 of 18

Examples (3)

... ... groups actors accessible via their name (“callable actors”) combined plot 1st evaluation: create plotting data Pump data into referenced plot 2nd evaluation: create plotting data Pump data into referenced plot

Generate combined plot of two evaluations by using “callable actors” functionality

SLIDE 8

10/08/2015 Peter Reutemann 8 of 18

Research (demos)

Compare two MOA classifiers (drift)
Compare MOA classifier on different streams
MOA cluster visualization
Track mouse in video

SLIDE 9

10/08/2015 Peter Reutemann 9 of 18

MOA - Drift

SLIDE 10

10/08/2015 Peter Reutemann 10 of 18

MOA - Drift

SLIDE 11

10/08/2015 Peter Reutemann 11 of 18

MOA - different streams

SLIDE 12

10/08/2015 Peter Reutemann 12 of 18

MOA - different streams

SLIDE 13

10/08/2015 Peter Reutemann 13 of 18

MOA - Cluster visualization

SLIDE 14

10/08/2015 Peter Reutemann 14 of 18

MOA - Cluster visualization

Stream 1 Stream 2

SLIDE 15

10/08/2015 Peter Reutemann 15 of 18

Track mouse

SLIDE 16

10/08/2015 Peter Reutemann 16 of 18

Track mouse

SLIDE 17

10/08/2015 Peter Reutemann 17 of 18

Industry

BLGG - environmental lab in NL
Spectral analysis
XRF: 10,000, MIR: 2,000, NIR: 1,500
In operation since 2006
Predictive modelling: soil, plant (~250 models)
1,000 to 3,000 samples per day
Savings due to less wet chemistry
USD 18 million to USD 33 million per year

SLIDE 18

10/08/2015 Peter Reutemann 18 of 18

Interested?

https://adams.cms.waikato.ac.nz/

@TheAdamsFlow