PNDA.io: when big data and OSS collide [Build Slide] Simplified - - PowerPoint PPT Presentation
PNDA.io: when big data and OSS collide [Build Slide] Simplified OSS / BSS Stack Bills and Order Customer Reports Order Billing and BSS Mgmt Reporting OSS analytics is responsible for Orchestration is responsible collecting data from
OSS BSS Network and Services Customer Order
Order Mgmt
Provisioning & Activation
Service
Data
Monitoring and analysis
Billing and Reporting
Bills and Reports
[Build Slide]
Orchestration is responsible for service provisioning and pushes state to the infrastructure The “C” in FCAPS OSS analytics is responsible for collecting data from the infrastructure, monitoring and analysis The “F_APS” in FCAPS
OSS Analytics Orchestration
Engineering effort (time) Performance Big data analytics-based Small data analysis
PMO FMO Orientation Single domain Cross domain Realisation Small data, tool driven Big data, data driven Data aggregation and analysis Coupled Decoupled Domain Data Schema Scheme-on-write Schema-on-read Analysis Prescriptive Prescriptive + Stochastic + ML Customisation Design time Run time
aggregation/store/ analysis
pipelines implemented from open source components
patterns ~75% of effort wasted / duplicated
lead to industry divergence
Telemetry Metrics Data sources HDFS Data store Spark Streaming MapR Data analysis Hbase Storm Kafka Streamsets Data aggregation Kafka
Impala Query
Outputs
Dashboard & Reporting
NiFi Logs
PNDA brings together a number of open source technologies to provide a simple, scalable open big data analytics Platform for Network Data Analytics Linux Foundation Collaborative Project based on the Apache ecosystem
platform
services for developing analytics applications
developing big data analytics applications whilst significantly reducing the TCO
convergence of network data analytics
PNDA Plugins ODL Logstash OpenBPM pmacct
XR TelemetryReal
Data Distribution
File Store
Platform Services: Installation, Mgmt, Security, Data Privacy App Packaging and Mgmt
Stream
Batch
Processing
SQL Query
OLAP Cube Search/ Lucene NoSQL Time Series
Data Exploration Metric Visualisation Event Visualisation
PNDA Mnged App PNDA Mnged App Unmnged App Unmnged App Query Visualisation and Exploration PNDA Applications PNDA Producer API PNDA Consumer API
analytics and data processing applications
processing and in-depth batch analysis
data analysis
platform apps developed for PNDA or client apps integrated with PNDA
structured query interfaces or consume streams directly.
data analytics
PNDA Plugins ODL Logstash OpenBPM pmacct
XR TelemetryReal
Data Distribution
File Store
Platform Services: Installation, Mgmt, Security, Data Privacy App Packaging and Mgmt
Stream
Batch
Processing
SQL Query
OLAP Cube Search/ Lucene NoSQL Time Series
Data Exploration Metric Visualisation Event Visualisation
PNDA Mnged App PNDA Mnged App Unmnged App Unmnged App Query Visualisation and Exploration PNDA Applications PNDA Producer API PNDA Consumer API
There are a bewildering number of big data technologies out there, so how do you decide what to use? We've evaluated and chosen the best tools, based on technical capability and community support. PNDA combines them to streamline the process of developing data processing applications.
Innovation in the big data space is extremely rapid, but combining multiple technologies into an end-to-end solution can be extremely complex and time-consuming PNDA removes this complexity and allows you to focus on developing the analytics applications, not on developing the pipeline – significantly reducing the effort required and time-to- value
pico standard
with BMP protocol support
established between BGP Speakers and OBMP
perform ‘avro’ encoding of BMP data
Batch job, running periodically
Internet
event information?