Responsive Analytics of Highly-Connected Big Data Dr. Peter - - PowerPoint PPT Presentation

www.cit.tu-berlin.de

• Dr. Peter Janacik, peter.janacik@tu-berlin.de

Stream Reasoning Workshop 2016 – TU Berlin, December 8, 2016

Responsive Analytics of Highly-Connected Big Data

www.cit.tu-berlin.de

Linking concepts

Content streams Users Graphs Data from social networks (Twitter/Instagram), Wikipedia, Wiktionary, evocation/
 synonym databases, medical knowledge, etc. posted favored/
 commented

• ccurs in

concept
 affinity

2

Extracting sense from tokens

Afuer patiently waiting a black cat ,

Symbol/token layer

Lemma Lemma Lemma Sense Sense Lemma Sense Lemma Sense Sense Lemma Sense

www.cit.tu-berlin.de

• Approach
• Symbol/token: notion in

general, oriented gradient cluster, recognized object, word

• Semantic overlay absorbs flow
• f meaning
• Follow-up processing
• Polarization and gist extraction
• Wikipedia/wiktionary as

knowledge model for semantic

• verlay

3

www.cit.tu-berlin.de

Apache Flink

• Was initiated at TU Berlin (first under the name

Stratosphere)

• MapReduce does not provide sufficient means

to implement state-of-the-art analysis methods

• Flink allows to connect transformations

(vertices) to a graph using data streams (directed edges)

• Distributed execution and placement within a

cluster: Flink program -> subtasks -> slots

• Number of slots on one physical node is

configurable but usually it is equal to number

• f cores
• Maximum degree of parallelism can be defined

and is used by Flink during execution

4

A B D C D E

Degree of parallelism is adjustable, here 2

www.cit.tu-berlin.de

Future Work

• Partitioning of data flow graph
• ver several data centers based
• n available resources, data

stream bandwidth, data privacy criteria

• Optimization criteria
• Min processing time
• Min costs
• Best fit
• Matching different criteria
• Dynamic migration in order to

accommodate changing characteristics of physical topology (available bandwidth/ resources (nodes), price, follow the sun, etc.)

5

A B D C F E

E, F, B, Z in data center 1

Z

Cut a Cut b D, C, A in data center 2 Data streams with different bandwidth

Interdependence of areas

www.cit.tu-berlin.de 6

Concept/ relationship/story detection

Data/results

Human data/ feedback generation Visualization Distribution at web- scale for insightful analysis

Comprehensible presentation Results/ recommendation to trigger Enabled by Alters models/algo behavior Enabled by

www.cit.tu-berlin.de 7

Semantic graph as result of analysis

www.cit.tu-berlin.de 8

Semantic graph as result of analysis

www.cit.tu-berlin.de 9

Instagram interaction heat map

www.cit.tu-berlin.de 10

Instagram interaction heat map

www.cit.tu-berlin.de 11

But how to visualize, what these graphs are about, 
 when there are typically millions to trillions of edges?

Semantic graph as result of analysis

Approaches to visualization

www.cit.tu-berlin.de 12

• 2-dimensional
• Works well with most of

the currently available devices, no special hardware needed

• Supported by broad range
• f platforms
• Less complex, easier to

implement

• Fewer problems with

readability and overlap

Approaches to visualization

www.cit.tu-berlin.de 13

• 3-dimensional
• Chance to make use of

additional dimension to untangle the big graph

• Different perspectives

may cover different aspects/lead to different conclusions

• Can exploit the full

potential of touch interfaces

www.cit.tu-berlin.de

Responsive Analytics of Highly-Connected Big Data

• Dr. Peter Janacik, peter.janacik@tu-berlin.de

Stream Reasoning Workshop 2016 – TU Berlin, December 8, 2016