SATURN 2016 May 2-5, San Diego Strategic Prototyping for Developing Big Data Systems Hong-Mei Chen, University of Hawaii Serhiy Haziyev , SoftServe Inc. Olha Hrytsay, SoftServe Inc. Rick Kazman , University of Hawaii and SEI/CMU
Agenda Small & Big Types of RASP Model Discussion Data Systems Prototypes
Agenda Small & Big Types of RASP Model Discussion Data Systems Prototypes
Small (Traditional) vs Big Data Systems Traditional Analytics (BI) Big Data Analytics Reporting/BI Reporting/BI Machine Learning Data OLAP Cubes MPP Data Warehouses Hadoop/Map Warehouses Reduce NoSQL Search Engines Databases ETL Semi- Unstructured Structured Data Structured Data Structured Data Data
Big Data Systems Challenges • The five V’s of big data (Volume, Variety, Velocity, Veracity, and Value) • Paradigm shifts (Data Lake, Polyglot Persistence, Lambda Architecture, etc.) • The short history of big data system development • Rapid technology changes • The difficulty of selecting big data technology
Agenda Small & Big Types of RASP Model Discussion Data Systems Prototypes
Why Prototyping is Important? They said they didn’t need a prototype Typical motivations for prototyping: • Requirements are uncertain • Technologies are new • No comparable system has been previously developed • No full buy-in from the business Architectural analysis alone can not address these risks!
Types of Prototypes Minimum Viable Vertical Throwaway Prototype Horizontal Prototype Product (MVP) Evolutionary Prototype (Proof-of-Concept)
When and Why to Prototype? Goals (Why?): Initial Identification of missing, conflicting or ambiguous architectural requirements • Architecture Creation of initial architecture design and selection of candidate technologies • Project timeline (When?) Analysis Confirmation of user interface requirements and system scope • Rapid Horizontal Demonstration version of the system to obtain buy-in from the business • Prototype Validation of technologies and scenarios that pose risks • PoC PoCs Vertical Evolutionary Integration of selected technologies • Prototype Clarification of complex requirements • Testing critical functionality and quality attribute scenarios • Getting early feedback from end users and updating the product accordingly • Presentation of a working version to a trade show or customer event MVP • Evaluation of team progress and alignment • Find more info at: “Strategic Prototyping for Developing Big Data Systems”, IEEE Software, March-April, 2016
Agenda Small & Big Types of RASP Model Discussion Data Systems Prototypes
The RASP Model We created RASP (Risk-based Architecture-centric Strategic Prototyping) to provide guidance on how to employ strategic prototyping for risk management. The RASP model can stand alone, but is ideally integrated with an architecture-centric agile development methodology, such as BDD (Big Data Design).
BDD and RASP
RASP Model: A strategic-prototyping decision flowchart I nitial Evolutionary Architecture Prototype Analysis I s there a Assess existing Go to throwaway I s this a green significant risk system and identify prototype chart for field system with a selected gaps that pose risks each technology No Yes design? technology? for new requirements that poses a risk Yes No Perform architecture I ntegrate Perform architecture analysis, update the technologies, test analysis, and select a existing architecture, critical reference and select candidate functionality and architecture and technologies to quality attribute candidate replace or augment scenarios technologies existing technologies Reconsider the Are there issues technology discovered with selection, and go Yes any technologies? to throwaway prototype chart No
RASP Model: A strategic-prototyping decision flowchart Throwaway Prototype Are there new I s the selected requirements that Change candidate technology new on the pose technology technology market? No No risks? Yes Yes Was it used successfully before for similar Yes scenarios? No Have trusted sources “proven” the technology for specific Yes scenarios? No Create rapid (throwaway) prototype to mitigate risks Yes No Are there more I s it possible to Does the prototype configurations, data “soften” any satisfy the specific models, resources to No requirements? scenarios? No try? Yes
Validation 9 case studies were used to validate RASP Diverse domains: Network Security/Intrusion Prevention, Online Coupon Web Analytics Platform, Cloud-based Mobile App Development Platform, Telecom E-tailing platform, etc. Diverse BD requirements: data volume, latency, scalability Details available at: http://itm-vm.shidler.hawaii.edu/BDCases/CaseStudies.htm
Summary: 7 Guidelines 1. Architecture analysis might be the only feasible option when you need to make early decisions 2. Architecture analysis alone is insufficient to prove many important system properties 3. Architecture analysis complements vertical evolutionary prototyping 4. An evolutionary prototype can effectively mitigate risk if you implement it as a skeleton 5. Vertical evolutionary prototyping might need to be augmented with throwaway prototypes 6. Throwaway prototypes work best when you need to quickly evaluate a technology 7. MVP is more of a business decision than a decision driven simply by technological risk
Agenda Small & Big Types of RASP Model Discussion Data Systems Prototypes
Conclusions Risk analysis has been advocated for controlling the evolutionary prototyping process. Our research results demonstrate that an architecture-centric design methodology such as BDD makes risk management explicit, systematic and cost- effective. It provides a basis for value discovery with stakeholders, for reasoning about risks, for planning and estimating cost and schedule, and for supporting experimentation. The RASP model offers practical guidelines for strategic prototyping, combining architecture analysis and a variety of prototyping techniques.
Recommend
More recommend
