Design and Run-Time Quality of Service Management Techniques for - - PowerPoint PPT Presentation
Design and Run-Time Quality of Service Management Techniques for Publish/ Subscribe Distributed Real-Time and E mbedded Systems http://www.dre.vanderbilt.edu/~jhoffert/dissertation.pdf Joe Hoffert jhoffert@dre.vanderbilt.edu Institute for
jhoffert@dre.vanderbilt.edu Institute for Software Integrated Systems Vanderbilt University Nashville, Tennessee
Client-server technology may not suffice for all DRE systems => move towards publish/subscribe middleware
are complementary technologies Characteristics of Pub/Sub
pub/sub
time-independent data distribution
unbounded # of senders/receivers
3
Manifesto for QoS-enabled Pub/Sub The right data…to the right place…at the right time
Event & Notification Services Java Message Service WS Brokered Notification
Application
‘Global’ Data Store
Application Application Application Application
Data Distribution Service
4
systems”
computing environments, emergency response systems
vs. vs. vs.
(often conflicting) QoS demands
(dis)continuous changes in runtime environments
dependable upgrades, availability of critical tasks, dynamic resource management DRE systems increasingly realized by composing loosely-coupled services (e.g., pub/sub)
5
Variability in the solution space (both design- and run-time)
Data reliability Provisioning of data resources Data for late arriving readers Data with time deadlines Ordered data Data priority Inter-arrival data spacing Determining liveness Data redundancy
8
…
System Execution Timeline
QoS Mechanism 1 (adequate) QoS Mechanism 1 (inadequate) QoS Mechanism 2 (adequate)
(initial environment)
(environment modification)
2. Designed new composite metrics & a flexible middleware framework to evaluate & benchmark QoS mechanisms. 3. Designed machine learning-based adaptation logic to provide accurate configurations & predictable response times in flexible envs. 4. Designed monitoring mechanisms & improved machine learning- based logic to improve adaptation accuracy in dynamic envs. 1. Developed model-based techniques to reduce manual effort & ameliorate accidental complexities in deploying pub/sub DRE systems.
9
…
Development Support: QoS configurations can have numerous entities & QoS policies; how can we help DRE developers manage the complexity of developing configurations?
System Execution Timeline
QoS Mechanism 1 (adequate) QoS Mechanism 1 (inadequate) QoS Mechanism 2 (adequate)
(initial environment)
(environment modification)
10
…
MuxModule MulticastModule SequencerModule
…
ANT Framework
mechanisms: Several QoS mechanisms are available; how can we help developers evaluate QoS mechanisms for pub/sub middleware?
System Execution Timeline
QoS Mechanism 1 (adequate) QoS Mechanism 1 (inadequate) QoS Mechanism 2 (adequate)
(initial environment)
(environment modification)
11
…
TCP/IP UDP/IP Multicast Custom protocol
computing environments: Cloud computing resources which affect QoS aren’t known until runtime; how can we configure the middleware based on resources provided?
System Execution Timeline
QoS Mechanism 1 (adequate) QoS Mechanism 1 (inadequate) QoS Mechanism 2 (adequate)
(initial environment)
(environment modification)
12
…
environments: As environments or operating conditions change, QoS can diminish; how can we adapt the middleware to support predictable QoS? System Execution Timeline
QoS Mechanism 1 (adequate)
QoS Mechanism 1 (inadequate) QoS Mechanism 2 (adequate)
(initial environment) (environment modification)
System Execution Timeline
QoS Mechanism 1 (adequate) QoS Mechanism 1 (inadequate) QoS Mechanism 2 (adequate)
(initial environment)
(environment modification)
13
…
Presented solutions to these in qualifying exam; briefly review here
App
‘Global’ Data Store
App App App App Protocol 1 Protocol 2 Operating environment
QoS? QoS? QoS? QoS? QoS?
(design)
System Lifecycle Timeline
System Deployment
QoS Configuration Validation Manual QoS Configuration Techniques
Focus Area 2 Focus Area 1
System Execution Timeline
QoS Mechanism 1 (adequate) QoS Mechanism 1 (inadequate) QoS Mechanism 2 (adequate)
(initial environment)
(environment modification)
DQML addresses the QoS configuration management challenges of (1) Correctly specified QoS properties, DQML addresses the QoS configuration management challenges of (1) Correctly specified QoS properties, (2) Correctly managed related & interacting QoS, and DQML addresses the QoS configuration management challenges of (1) Correctly specified QoS properties, (2) Correctly managed related & interacting QoS, and (3) Implementation artifacts that accurately represent design
Application- specific interpreter
Focus Area 2 Focus Area 3 Focus Area 1 Focus Area 4
15ms 10ms
parameters 15
Book Chapter
Productivity Analysis for the Distributed QoS Modeling Language. Model-Driven Domain Analysis & Software Development: Architectures & Functions. Ed. Dr. Janis Osis & Dr. Erika Asnina, Riga Technical University, Latvia. Conference Publications
June). A QoS Policy Configuration Modeling Language for Publish/Subscribe Middleware
International Conference on Distributed Event- Based Systems (DEBS), Toronto, Canada.
November). DQML: A Modeling Language for Configuring Distributed Publish/Subscribe Quality of Service Policies. Proceedings of the 10th International Symposium on Distributed Objects, Middleware, & Applications (DOA), Monterrey, Mexico. Poster Publications
Schmidt, D. (2007, March). Enhancing Security in Ultra-Large Scale (ULS) Systems using Domain-specific Modeling. Spring 2007 Conference for Team for Research in Ubiquitous Secure Technology (TRUST), Berkeley, CA.
& Birman, K. (2008, April). Trustworthy Conferencing via Domain-specific Modeling & Low Latency Reliable
Team for Research in Ubiquitous Secure Technology (TRUST), Berkeley, CA.
(2007, September). QoS Management in Publish/Subscribe Systems using Domain- specific Modeling. ACM/IEEE 10th International Conference on Model Driven Engineering Languages & Systems (MoDELS), Nashville, TN.
First Author
16
FLEXMAT addresses the challenges of (1) Supporting multiple “antagonistic” QoS via new, custom protocols & (2) Understanding how environments affect multiple QoS concerns
18
Focus Area 1 Focus Area 3 Focus Area 2 Focus Area 4
Evaluate transport protocols with multiple operating environments using FLEXMAT testbed:
sending rate
MuxModule MulticastModule SequencerModule
…
ANT Framework
FLEXMAT Testbed
integrated with DDS implementations
Conference Publications 1. Hoffert, J., Schmidt, D., & Gokhale, A. (2009, November). Evaluating Transport Protocols for Real-time Event Stream Processing Middleware & Applications. Proceedings of the 11th International Symposium on Distributed Objects, Middleware, & Applications (DOA'09), Algarve, Portugal. Workshop Publications 2. Hoffert, J., & Schmidt, D. (2008, July). Supporting Scalability & Adaptability via Adaptive Middleware & Network Transports. Proceedings of the OMG’s Workshop on Distributed Object Computing for Real-time & Embedded Systems, Washington, D.C., USA. 3. Hoffert, J., Schmidt, D., Balakrishnan, M., & Birman, K. (2008, September). Supporting Large- scale Continuous Stream Datacenters via Pub/Sub Middleware & Adaptive Transport Protocols. Proceedings of the 2nd Workshop on Large-Scale Distributed Systems & Middleware, Yorktown, NY. 4. Balakrishnan, M., Hoffert, J., Birman, K., & Schmidt, D., (2008, September). Rethinking Reliable Transport for the Datacenter. Proceedings of the 2nd Workshop on Large-Scale Distributed Systems & Middleware, Yorktown, NY. 5. Hoffert, J., & Schmidt, D. (2009, July). FLEXible Middleware & Transports (FLEXMAT) for Real- time Event Stream Processing (RT-ESP) Applications. Proceedings of the OMG’s Workshop on Distributed Object Computing for Real-time & Embedded Systems, Washington, D.C., USA.
First Author Second Author
19
20
…
This was the focus area in my qualifying exam that I proposed to complete my dissertation.
System Execution Timeline
QoS Mechanism 1 (adequate) QoS Mechanism 1 (inadequate) QoS Mechanism 2 (adequate)
(initial environment)
(environment modification)
21
…
This is a new focus area that I included while completing my dissertation.
System Execution Timeline
QoS Mechanism 1 (adequate) QoS Mechanism 1 (inadequate) QoS Mechanism 2 (adequate)
(initial environment)
(environment modification)
Focus Area 1 Focus Area 2
23
Focus Area 3 Focus Area 4
–Resources on demand –“Pay-as-you-go” usage fee –Computing resources
–Networking resources
–Amazon Elastic Compute Cloud (EC2), Google App Engine, GoGrid, AppNexus, Emulab –OS, Database, RAM, CPU, Disk space, cores, load balancing, applications (e.g., Apache, Facebook servers), bandwidth
Not straightforward to use Cloud in DRE systems
25
DRE Cloud Scenario ‒Regional disasters (e.g., hurricane, flooding) ‒Survivors trapped ‒Search & rescue mission initiated ‒Search application fuses multiple sensor streams
unmanned aerial vehicles (UAVs)
infrastructure
datacenter for fusion & dissemination
26
Datacenter Requirements ‒Operate in flexible environments, e.g.,
applications
‒Support Multiple QoS
Ad-hoc datacenter Cloud computing infrastructure UAV providing infrared scan stream Infrastructure camera providing video stream Rescue helicopter Disaster victims
adapt to leverage available resources
bandwidth, CPU speed, RAM
27
Challenge 1: Reduction of Development Complexity Developing adaptive behavior is challenging:
from design to implementation
data sending rate
network loss
850 MHz CPUs, 2 GB RAM 100 Mb/s LAN
27
28
Challenge 2: Accurate Configuration in Cloud Environments Environment resources unknown a priori make static configuration inadequate
Ad-hoc datacenter
Cloud computing infrastructure TCP/IP
QoS mechanisms
UDP/IP Multicast
protocol
28
Challenge 3: Timely Configuration in Cloud Environments DRE systems require timely configuration
Ad-hoc datacenter
Cloud computing infrastructure TCP/IP
QoS mechanisms
UDP/IP Multicast Custom protocol
29
HW/SW Embedded Systems”, EURASIP Journal on Embedded Systems, pages 1 – 10. Hindawi Publishing Corp., 2008.
systems," Proceedings of the 2010 IEEE Conference on Emerging Technologies and Factory Automation (ETFA), September 2010, Bilboa, Spain. P.-C. David & T. Ledoux. “An Aspect-Oriented Approach for Developing Self-Adaptive Fractal Components”, Software Composition, pages 82–97. Springer LNCS, 2006.
ACM/IFIP/USENIX 2005 International Conference on Middleware, November 2005, Grenoble, France.
Network”, Autonomic Communication, pp. 202–213. Springer-Verlag, 2006.
Proceedings of the IEEE International Conference on Emerging Technologies and Factory Automation, September 2008, Hamburg, Germany. Xiangping Bu et al. "A Reinforcement Learning Approach to Online Web Systems Auto- configuration“, 29th IEEE International Conference on Distributed Computing Systems, June 2009, Montreal, Canada.
Good for configuring local components
30
HW/SW Embedded Systems”, EURASIP Journal on Embedded Systems, pages 1 – 10. Hindawi Publishing Corp., 2008.
systems," Proceedings of the 2010 IEEE Conference on Emerging Technologies and Factory Automation (ETFA), September 2010, Bilboa, Spain. P.-C. David & T. Ledoux. “An Aspect-Oriented Approach for Developing Self-Adaptive Fractal Components”, Software Composition, pages 82–97. Springer LNCS, 2006.
ACM/IFIP/USENIX 2005 International Conference on Middleware, November 2005, Grenoble, France.
Network”, Autonomic Communication, pp. 202–213. Springer-Verlag, 2006.
Proceedings of the IEEE International Conference on Emerging Technologies and Factory Automation, September 2008, Hamburg, Germany. Xiangping Bu et al. "A Reinforcement Learning Approach to Online Web Systems Auto- configuration“, 29th IEEE International Conference on Distributed Computing Systems, June 2009, Montreal, Canada.
31
Good for developing autoconfiguration applications
HW/SW Embedded Systems”, EURASIP Journal on Embedded Systems, pages 1 – 10. Hindawi Publishing Corp., 2008.
systems," Proceedings of the 2010 IEEE Conference on Emerging Technologies and Factory Automation (ETFA), September 2010, Bilboa, Spain. P.-C. David & T. Ledoux. “An Aspect-Oriented Approach for Developing Self-Adaptive Fractal Components”, Software Composition, pages 82–97. Springer LNCS, 2006.
ACM/IFIP/USENIX 2005 International Conference on Middleware, November 2005, Grenoble, France.
Network”, Autonomic Communication, pp. 202–213. Springer-Verlag, 2006.
Proceedings of the IEEE International Conference on Emerging Technologies and Factory Automation, September 2008, Hamburg, Germany. Xiangping Bu et al. "A Reinforcement Learning Approach to Online Web Systems Auto- configuration“, 29th IEEE International Conference on Distributed Computing Systems, June 2009, Montreal, Canada.
32
Good for autoconfiguration when timeliness is not a driving concern
Property Description Distributed Configuration Does the technique help autoconfiguration of QoS across machine boundaries? Online Configuration Does the technique perform configuration adjustments while system is running? Timely Configuration Does the technique provide bounded-time – ideally, constant-time – response? What properties help us assess research to configure QoS in cloud environments?
33
34
Does the technique autoconfigure across machines?
Bu Grace David Imtiaz Zoitl Eustache Valetto
35
Bu Grace David Imtiaz Zoitl Eustache Valetto
Does the technique provide
36
Bu Grace David Imtiaz Zoitl Eustache Valetto
Does the technique provide bounded time complexity?
Current gap makes it hard for DRE systems in cloud environments to configure QoS in a timely manner
37
Bu Grace David Imtiaz Zoitl Eustache Valetto
Gap: Configure across machines in a timely manner
38
Bu Grace David Imtiaz Zoitl Eustache Valetto
ADAMANT
Solution Approach: Bounded time, accurate configuration of transport protocols for QoS-enabled pub/sub middleware in ADAMANT
39 Network % loss # receivers Sending rate protocol parameters
Policy-based configuration Artificial neural network Decision tree Support vector machine (SVM)
Approach Boundedness Accuracy (known) Accuracy (unknown) Development complexity Policy based Yes Perfect (100%) Low (default) Medium -High Reinforcement Learning No High Medium Low Decision Tree Yes (data dependent) High (99%) High (87%) Low Neural Network Yes (constant) Perfect (100%) High (85%) Low SVM Yes (constant) Perfect (100%) High (79%) Low
40
Evaluated approaches based on:
Approach Boundedness Accuracy (known) Accuracy (unknown) Development complexity Policy based Yes Perfect (100%) Low (default) Medium -High Reinforcement Learning No High Medium Low Decision Tree Yes (data dependent) High (99%) High (87%) Low Neural Network Yes (constant) Perfect (100%) High (85%) Low SVM Yes (constant) Perfect (100%) High (79%) Low
41
Evaluated approaches based on:
Approach Boundedness Accuracy (known) Accuracy (unknown) Development complexity Policy based Yes Perfect (100%) Low (default) Medium -High Reinforcement Learning No High Medium Low Decision Tree Yes (data dependent) High (99%) High (87%) Low Neural Network Yes (constant) Perfect (100%) High (85%) Low SVM Yes (constant) Perfect (100%) High (79%) Low
42
Evaluated approaches based on:
Approach Boundedness Accuracy (known) Accuracy (unknown) Development complexity Policy based Yes Perfect (100%) Low (default) Medium -High Reinforcement Learning No High Medium Low Decision Tree Yes (data dependent) High (99%) High (87%) Low Neural Network Yes (constant) Perfect (100%) High (85%) Low SVM Yes (constant) Perfect (100%) High (79%) Low
43
Evaluated approaches based on:
Approach Boundedness Accuracy (known) Accuracy (unknown) Development complexity Policy based Yes Perfect (100%) Low (default) Medium -High Reinforcement Learning No High Medium Low Decision Tree Yes (data dependent) High (99%) High (87%) Low Neural Network Yes (constant) Perfect (100%) High (85%) Low SVM Yes (constant) Perfect (100%) High (79%) Low
44
Evaluated approaches based on:
Adaptive Network Transports (ANT) framework
XOR encoding FEC-sender ACK-based Reed-Solomon encoding FEC-receiver NAK-based
Custom protocol 1 Custom protocol 2
XOR encoding FEC-sender ACK-based Reed-Solomon encoding FEC-receiver NAK-based FEC-group Tornado encoding
Data Distribution Service (DDS)
Protocol Optimization
Artificial Neural Network (ANN)
environments
5 10 15 20 25 2 4 6
Interpolation between training data
45
ADAMANT
SAR Topic(s)
Adaptive Network Transport (ANT) Protocols
Data Writer Data Reader Data Reader
App Publisher App Subscriber Domain
…
Protocol Optimizer (ANN)
Key:
Control interaction between subsystems
DDS
environment for resources.
passed to ADAMANT.
in a timely manner & notifies ANT
Cloud Computing Environment
protocol for the middleware
46
ADAMANT addresses challenge 1 (development complexity) via ANNs to manage protocol selection & implementation transformation ANNs manage the development complexity of protocol management:
environment and protocols
developing implementation)
RAM Network speed Network % loss protocol parameters ANN
Adaptive Network Transport (ANT) Framework
Autonomic Configuration Controller
data sending rate
network loss 3 GHz CPUs, 4 GB RAM
47
1 Gb/s LAN
CPU speed Sending rate
… DDS
48
ADAMANT addresses challenge 2 (accurate configuration) by overfitting ANN to data Overfitting data increases ANN’s accuracy for selecting appropriate protocol
Ad-hoc datacenter
Cloud computing infrastructure TCP/IP
QoS mechanisms
UDP/IP Multicast Custom protocol
48
ADAMANT addresses challenge 3 (timely configuration) via ANN w/ bounded constant-time response ANNs are equation based:
CPU speed, RAM Network speed Network % loss protocol parameters ANN
training time)
49
Difference in hardware triggers a difference in appropriate transport protocol Experimental environment:
50
1000 2000 3000 4000 5000 6000 7000 8000 9000 1 2 3 4 5 ReLate2 Values Experiment
850 MHz CPU, 100Mb LAN, 3 rcvrs, 5% loss
NAKCast 0.001 - 25Hz Ricochet R4 C3 - 25Hz 3500 3700 3900 4100 4300 4500 1 2 3 4 5 ReLate2 Values Experiment
3 GHz CPU, 1Gb LAN, 3 rcvrs, 5% loss
NAKcast 0.001 25Hz Ricochet R4C3 - 25Hz
10 20 30 40 50 60 70 80 90 100 1 2 3 4 5 6 7 8 9 10
Accuracy for excluded data (%) Training Run
ANN Accuracy (2-fold cross-validation)
36 hidden nodes 24 hidden nodes 12 hidden nodes 6 hidden nodes
51
82 84 86 88 90 92 94 96 98 100 102 1 2 3 4 5 6 7 8 9 10
Accuracy (%) Training Run
ANN Accuracy (known environments)
36 hidden nodes 24 hidden nodes 12 hidden nodes 6 hidden nodes
ANNs w/ 24 nodes provide most instances of 100% accuracy, highest average accuracy with 2-fold cross-validation (78%) Experimental environment:
Evaluation Criteria Description (H1) Adjust for known environment Hypothesize that ADAMANT will provide adjustment improvement for known environments at least 85% of the time
52
Using ANNs, ADAMANT provides 100% accurate adjustment for known environments
ANNs provide the predictable decision-making timeliness needed for DRE systems ADAMANT addresses the challenges of (1) Development complexity via machine learning to determine protocols, (2) Configuration accuracy via overfitted supervised machine learning & (3) Configuration timeliness via equation-based machine learning
53
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 9 10
Time (µs) Classification Run
Average ANN Response Times
6 hidden nodes 12 hidden nodes 24 hidden nodes 36 hidden nodes 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 2 3 4 5 6 7 8 9 10
Time (µs) Classification Run
Std Deviation ANN Response Times
6 hidden nodes 12 hidden nodes 24 hidden nodes 36 hidden nodes
Experimental environment:
Conference Publications 1. Hoffert, J., Schmidt, D., & Gokhale, A. (2010, November). Adapting Distributed Real-time and Embedded Publish/Subscribe Middleware for Cloud-Computing Environments, Proceedings of the ACM/IFIP/USENIX 11th International Middleware Conference (Middleware 2010), Bangalore, India. 2. Hoffert, J., & Schmidt, D. (2010, October). Evaluating Supervised Machine Learning for Adapting Enterprise DRE Systems, Proceedings of the 2010 International Symposium on Intelligence Information Processing and Trusted Computing (IPTC 2010), Huanggang, China. Workshop Publications 3. Hoffert, J., Schmidt, D., & Gokhale, A. (2010, April). Adapting and Evaluating Distributed Real- time and Embedded Systems in Dynamic Environments,The 1st International Workshop on Data Dissemination for Large scale Complex Critical Infrastructures (DD4LCCI 2010), Valencia, Spain.
First Author
54
56
Focus Area 1 Focus Area 2 Focus Area 4 Focus Area 3
Motivating Example: Smart City Ambient Assisted Living (SCAAL)
59 High-resolution health monitoring GPS Videocamera Cell phone PDA Mobility Sensory Enhancing Equipment Smart City Law enforcement Doctors Surveillance infrastructure Firefighters EMS Healthcare facilities
Scenario
equipment & sensing/aware “smart cities”
Scenario
equipment & sensing/aware “smart cities”
60 High-resolution health monitoring GPS Videocamera Cell phone Mobility
Personal DataCenter
PDA Sensory Enhancing Equipment Smart City Law enforcement Doctors Surveillance infrastructure Firefighters EMS Healthcare facilities
Requirements
monitoring
61
Personal DataCenter
more updates for critical data)
62
Challenge 1: Environment monitoring & update dissemination
(initial environment)
Elderly person traveling through smart city
(environment modification)
… Low sending rate High sending rate, More detailed information Normal health information, Low update rate required
As environment changes, updates need to be propagated throughout the application.
Doctor detects health anomaly, update rate increased
63
Challenge 2: Optimal accuracy for unknown environments while maintaining timeliness
QoS Mechanism 1 (inadequate) Transport protocol 1 (inadequate) High sending rate, more detailed health information
addressed while selecting an adequate QoS mechanism
Transport protocol 1 (adequate)
unknown environments to rival accuracy for known environments
lead to reduced health or death
PrismTech’s Tuner application, http://www.opensplice.com Real-Time Innovations’ RTI Analyzer, RTI Scope, & RTI Protocol Analyzer, http://rti.com/products/developer_platform
Publish/Subscribe Applications”, IEEE Transactions on Software Engineering, vol. 29, no. 12, pages1059–1071.
ACM/IFIP/USENIX 2005 International Conference on Middleware, November 2005, Grenoble, France
Middleware, September 2005, Lisbon, Portugal
Conference on Service Oriented Computing, December 2008, Sydney, Australia
Good for manually checking the run-time QoS status
66
PrismTech’s Tuner application, http://www.opensplice.com Real-Time Innovations’ RTI Analyzer, RTI Scope, & RTI Protocol Analyzer, http://rti.com/products/developer_platform
Publish/Subscribe Applications”, IEEE Transactions on Software Engineering, vol. 29, no. 12, pages1059–1071.
ACM/IFIP/USENIX 2005 International Conference on Middleware, November 2005, Grenoble, France
Middleware, September 2005, Lisbon, Portugal
Conference on Service Oriented Computing, December 2008, Sydney, Australia
Good for developing adaptation applications
67
PrismTech’s Tuner application, http://www.opensplice.com Real-Time Innovations’ RTI Analyzer, RTI Scope, & RTI Protocol Analyzer, http://rti.com/products/developer_platform
Publish/Subscribe Applications”, IEEE Transactions on Software Engineering, vol. 29, no. 12, pages1059–1071.
ACM/IFIP/USENIX 2005 International Conference on Middleware, November 2005, Grenoble, France
Middleware, September 2005, Lisbon, Portugal
Conference on Service Oriented Computing, December 2008, Sydney, Australia
Good for adaptation when timeliness is not a driving concern
68
Property Description Monitor environment Does the technique know when the environment has changed? Dynamic adaptation Does the technique perform adaptation adjustments while system is running? Timely adaptation Can the technique change to a more appropriate protocol in a timely manner? What properties help us assess research to support QoS in dynamic environments?
69
70
PrismTech RTI
Does the technique monitor the system for changes?
Bu Grace Caporuscio Vienne Herssens
71
PrismTech RTI Bu Grace Caporuscio Vienne Herssens
Does the technique adapt while the system is running?
72
Does the technique provide timely transition to support QoS?
PrismTech RTI Bu Grace Caporuscio Vienne Herssens
73
PrismTech RTI Bu Grace Caporuscio Vienne Herssens
Current gap makes it hard for DRE systems in dynamic environments to adapt QoS in a timely manner Gap: Monitor, analyze, & adapt in a timely manner
73
74
PrismTech RTI Bu Grace Caporuscio Vienne Herssens
74
Solution Approach: Timely autonomic transport protocol adaptation for QoS-enabled pub/sub middleware in ADAMANT++
ADAMANT++
SAR Topic
Adaptive Network Transport (ANT) Protocols
Data Writer Data Writer Data Reader
Domain
…
Topic
Data Reader Data Writer Data Writer
…
Autonomic Adaptation Controller Protocol Optimizer
Key:
Order of interaction between subsystems
DDS
environment feedback
sends to optimizer
protocol & settings (leveraging multiple machine learning techniques), returns to controller
needed
and/or settings to maintain QoS
1 2 3 4 N
Video & Infrared Publisher Rescue Helicopter Subscriber
75
Boldface Update from previous ADAMANT architecture for configuration
76
ADAMANT++ addresses Challenge 1 (disseminating updates) via environment monitoring topic
SAR Topic
Adaptive Network Transport (ANT) Protocols
Data Writer Data Writer Data Reader
Domain
…
Topic
Data Reader Data Writer Data Writer
… DDS
Video & Infrared Publisher Rescue Helicopter Subscriber
77
ADAMANT++ addresses Challenge 2 (increasing accuracy) via Timely Integrated Machine Learning (TIML)
Environment update Perfect Hash Artificial Neural Network Support Vector Machines
Timely Integration of Machine Learning (TIML)
Evaluation Criteria Description (H2) Adjust for unknown environment Hypothesize that ADAMANT will provide adjustment improvement for unknown environments more than 50% of the time
78
Leveraging TIML, ADAMANT++ provides 86% accuracy for unknown environments.
79
TIML provides the predictable adaptation timeliness needed for DRE systems Experimental environment:
10 11 12 13 14 1 101 201 301 401 501 601 701 801 901
Time (µs) Classification Run
TIML Response Times
ANN path SVM path
80
ANT provides the predictable reconfiguration timeliness needed for DRE systems Experimental environment:
1 2 3 4 5 6 1 101 201 301 401 501 601 701 801 901
Time (µs) Reconfiguration Run
ANT Reconfiguration Times
No reconfiguration Ricochet to NAKcast NAKcast to Ricochet
ADAMANT++ addresses the challenges of (1) Disseminating environment updates, (2) Maximizing accuracy while maintaining timeliness
ADAMANT++ validates the three hypotheses from my qualifying exam: (1) > 85% accuracy for known environments (achieved 100%), (2) > 50% accuracy for unknown environments (achieved 86%), (3) Constant-time response Evaluation Criteria Description (H3) Provide bounded, constant time adaptation Hypothesize that ADAMANT will adjust to new
(i.e., O(1))
81
Leveraging equation- based machine learning and ANT, ADAMANT++ responds to new
in constant time.
These decisions could be made at an application level & obeyed by ADAMANT++
82
Should adaptation always occur to get better QoS? Are there times when the adjustment doesn’t warrant the adaptation? How can we analyze the value of adapting?
13% QoS Increase 24% QoS Increase 3% QoS Increase 21% QoS Increase
Using threshold of 10% increase, we would reject adaptation for only 3% increase.
Focus Area Challenge Approach Contribution Valid QoS Design
QoS Validation
configuration & generates implementation artifacts
Evaluation of QoS Mechanisms
Guidance & Flexibility
flexible protocol framework
analysis
metrics Autonomic Configuration for QoS
in Flexible Environments
flexible resource envs
supervised learning
Autonomic Adapation for QoS
in Dynamic Environments
protocols in dynamic envs
integration of supervised learning integration
Enhancing Productivity & Flexibility for QoS-enabled Pub/Sub DRE Systems www.dre.vanderbilt.edu/~jhoffert/research
83
Conference Publications 4. Hoffert, J., Jiang S., & Schmidt, D. (2007, April). A Taxonomy of Discovery Services & Gap Analysis for Ultra-Large Scale Systems. Proceedings of the 45th Annual Southeast Regional Conference, Winston-Salem, NC 5. Hoffert, J., Schmidt, D., & Gokhale, A. (2007, June). A QoS Policy Configuration Modeling Language for Publish/Subscribe Middleware Platforms. Proceedings of the Inaugural International Conference on Distributed Event-Based Systems, Toronto, Canada. 6. Hoffert, J., Schmidt, D., & Gokhale, A. (2008, November). DQML: A Modeling Language for Configuring Distributed Publish/Subscribe Quality of Service Policies. Proceedings of the 10th International Symposium on Distributed Objects, Middleware, & Applications, Monterrey, Mexico.
First Author
84
Journal Publications 1. Hoffert, J., Mack, D., & Schmidt, D. (2010) Integrating Machine Learning Techniques to Adapt Protocols for QoS-enabled Distributed Real-time and Embedded Publish/Subscribe Middleware, International Journal of Network Protocols and Algorithms, Vol. 2, No. 3. 2. Hoffert, J., Schmidt, D., & Gokhale, A. (2011) Evaluating Timeliness and Accuracy Trade-offs of Supervised Machine Learning for Adapting Enterprise DRE Systems in Dynamic Environments, (In submission to) International Journal of Computational Intelligence Systems. 3. Hoffert, J., Gokhale, A., & Schmidt, D. (2011) Autonomic Adaptation of Publish/Subscribe Middleware in Dynamic Environments, (In submission to) International Journal of Adaptive, Resilient and Autonomic Systems.
Conference Publications (cont.) 7. Hoffert, J., Schmidt, D., & Gokhale, A. (2009, November). Evaluating Transport Protocols for Real-time Event Stream Processing Middleware & Applications. The11th International Symposium on Distributed Objects, Middleware, & Applications, Algarve, Portugal. 8. Hoffert, J. & Schmidt, D. (2009, July). Maintaining QoS for Publish/Subscribe Middleware in Dynamic Environments. 3rd ACM International Conference on Distributed Event-Based Systems, Nashville, TN. 9. Hoffert, J., & Schmidt, D. (October, 2010). Evaluating Supervised Machine Learning for Adapting Enterprise DRE Systems, International Symposium on Intelligence Information Processing and Trusted Computing, Huanggang, China.
Embedded Publish/Subscribe Middleware for Cloud-Computing Environments, ACM/IFIP/USENIX 11th International Middleware Conference , Bangalore, India.
First Author
85
Book Chapters
First Author Second Author
Workshop Publications
Middleware & Network Transports. Proceedings of the OMG’s Workshop on Distributed Object Computing for Real-time & Embedded Systems, Washington, D.C., USA.
scale Continuous Stream Datacenters via Pub/Sub Middleware & Adaptive Transport Protocols. Proceedings of the 2nd Workshop on Large-Scale Distributed Systems & Middleware, Yorktown, NY.
Transport for the Datacenter. Proceedings of the 2nd Workshop on Large-Scale Distributed Systems & Middleware, Yorktown, NY.
time Event Stream Processing (RT-ESP) Applications. Proceedings of the OMG’s Workshop on Distributed Object Computing for Real-time & Embedded Systems, Washington, D.C., USA.
Pub/Sub System QoS in Dynamic Environments. Proceedings of the 8th Workshop on Adaptive & Reflective Middleware, Urbana Champaign, IL.
time and Embedded Systems in Dynamic Environments,1st International Workshop on Data Dissemination for Large scale Complex Critical Infrastructures, Valencia, Spain
86
First Author
Poster Publications
Ultra-Large Scale (ULS) Systems using Domain-specific Modeling. Spring 2007 Conference for Team for Research in Ubiquitous Secure Technology (TRUST), Berkeley, CA.
Publish/Subscribe Systems using Domain-specific Modeling. ACM/IEEE 10th International Conference on Model Driven Engineering Languages & Systems (MoDELS), Nashville, TN.
via Domain-specific Modeling & Low Latency Reliable Protocols. Spring 2008 Conference for Team for Research in Ubiquitous Secure Technology (TRUST), Berkeley, CA.
Systems in Dynamic Environments, EuroSys 2010 Conference, Paris, France.
87
Tutorials
Publish/Subscribe Middleware, The 2010 International Symposium on Intelligence Information Processing and Trusted Computing, Huanggang, China,
88