Model-Driven Continuous Deployment for Quality DevOps D. A. - - PowerPoint PPT Presentation

▶
model driven continuous
SMART_READER_LITE
LIVE PREVIEW

Model-Driven Continuous Deployment for Quality DevOps D. A. - - PowerPoint PPT Presentation

Oct 22, 2023 363 likes •623 views

Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri , E. Di Nitto, M. Guerriero, M. Artac , T. Borov s ak DICE Horizon 2020 Research & Innovation Action Grant Agreement no. 644869 Funded by the Horizon 2020

slide-1
SLIDE 1

DICE Horizon 2020 Research & Innovation Action Grant Agreement no. 644869 http://www.dice-h2020.eu

Funded by the Horizon 2020 Framework Programme of the European Union

Model-Driven Continuous Deployment for Quality DevOps

  • D. A. Tamburri, E. Di Nitto, M. Guerriero,
  • M. Artacˇ, T. Borovs̍ak
slide-2
SLIDE 2

Roadmap

  • Intro: The Rise of Big Data
  • Research Solution: Project DICE
  • DICE Meta-models
  • M2M and TOSCA in DICE

2

slide-3
SLIDE 3

Intro: The Rise of Big Data

  • 3 -
  • Software market rapidly

shifting to Big data

  • Expected 32% compound

annual growth rate in EU through 2016

  • Just 35% of Big data

projects are successful [CapGemini’15]

slide-4
SLIDE 4

Intro: Building blocks for DIAs today

4 Coordinator (Kafka) Orchestrator (Hadoop Cluster) Data Store Batch Layer Speed Layer Serving Layer Serving Layer Serving Layer Data Source Data Source Distributed computation Data streaming HDFS Distributed storage Lambda architecture Cloud infrastructure … … …

slide-5
SLIDE 5

Intro: example challenges!

Legacy Systems and Processes

  • 5 -

Big Data Analytics Adoption Learning curves Initial prototype Risk of failure

(+ others…)

slide-6
SLIDE 6

DICE Objectives

  • Tackling skill shortage and steep learning curves
  • High-degree of MDE automation via DICE tools
  • Shorten time to market for DIAs
  • Push out new products, without sacrificing quality
  • Decrease development and testing costs
  • Fast iterative definition of application prototypes
  • Reduce number and severity of quality incidents
  • DevOps-fashioned Iterative refinement of application design

6

slide-7
SLIDE 7

DICE incremental modeling and analysis[6]

7 DICE Platform Independent Model (DPIM) DICE Technology Specific Model (DTSM) DICE Deployment Specific Model (DDSM)

is implemented by is deployed onto

TOSCA blueprint

Analysis Analysis Analysis & Optimization

M2M transformation M2M transformation M2T transformation

  • To provide sound

abstractions and UML Profile for DIA

  • To provide a sound

method for:

  • DICE-based

Continuous DIA Architecting

  • DICE-based

Continuous DIA Deployment

slide-8
SLIDE 8

DICE incremental modeling and analysis[6]

8 DICE Platform Independent Model (DPIM) DICE Technology Specific Model (DTSM) DICE Deployment Specific Model (DDSM)

is implemented by is deployed onto

TOSCA blueprint

Analysis Analysis Analysis & Optimization

M2M transformation M2M transformation M2T transformation

  • To provide sound

abstractions and UML Profile for DIA

  • To provide a sound

method for:

  • DICE-based

Continuous DIA Architecting

  • DICE-based

Continuous DIA Deployment

In a DevOps Fashion!

Architec t Deploy Monitor Analyse Re- Deploy

slide-9
SLIDE 9

Where does this come from?

  • 9 -

Dev Area Views (Development structure, business logic, quality verification) Ops Area Views (Deployment, process, monitoring)

slide-10
SLIDE 10

Where does this come from? DICE!

10 Featuring the DICE H2020 EU Project Ops Area Views (Deployment,…) Dev Area Views (Development structure, business logic,…)

slide-11
SLIDE 11

Where does this come from? DICE!

11 Featuring the DICE H2020 EU Project Deployment and Operations area of

  • ur architecture framework
slide-12
SLIDE 12

Where does this come from? DICE!

12 Featuring the DICE H2020 EU Project

…

slide-13
SLIDE 13

Where does this come from? DICE!

13 Featuring the DICE H2020 EU Project

…

slide-14
SLIDE 14

DICE Meta-models - DTSM

14 Storm Package HadoopMR Package Spark Package <<Uses>> <<Uses>> Oryx 2 Package <<Uses>> Core DTSM Package <<Uses>>

...

DICE extensibility

*Metamodels here are illustrative mockups – You can think of using ADOxx to reproduce them!

slide-15
SLIDE 15

DICE Meta-models* - DTSM Core

  • 1. Common to all DICE tech-packs;
  • 2. Distinguishing analysable elements;
  • 3. Distinguishing properties;
  • 4. Simplified property definition;

15

*Metamodels here are illustrative mockups – You can think of using ADOxx to reproduce them!

slide-16
SLIDE 16

DICE Meta-models – DDSM[5]

16

MODACloudsML Specification TOSCA4DICE Specification Automated transformation MODA4DICE:: HadoopMR MODA4DICE:: Oryx2

MODACloudsML4DICE

TOSCA Blueprint *.yaml

... DDSM Metamodels DDSM Metamodel

Automated transformation

DICER M2T Tool

slide-17
SLIDE 17

Where does this come from? DICE!

17 Featuring the DICE H2020 EU Project

…

slide-18
SLIDE 18

DICE Meta-models – DDSMTOSCA

18 “Topology and Orchestration Specification for Cloud Applications”

  • 1. OASIS Standard;
  • 2. Cloud Infrastructure-

as-code language;

  • 1. Type-strength Intent

modeling;

  • 2. In DICE? Big-Data

TOSCA “profile”; Service template

slide-19
SLIDE 19

DICE Meta-models – DDSMTOSCA

19

Tier (Group Type)

“TOSCA is used to describe the deployment view for cloud applications”

source_resource

Node_Type_A

target_resource

Node_Type_B

Requirement connect_relationship ConnectsTo Capability

Nodes - are the resources or components that will be materialized or consumed in the deployment topology Relationships express the dependencies between the nodes (not the traffic flow) Requirement - Capability Relationships can be customized to match specific source requirements to target capabilities Groups Create Logical, Management or Policy groups (1 or more nodes)

 Node templates to describe components in the topology structure  Relationship templates to describe connections, dependencies, deployment ordering

slide-20
SLIDE 20

With M2M a TOSCA blueprint happens!

20

……….. "storm_nimbus_host": { "relationships": [ { "type": "cloudify.openstack.server_connected_to_floating_ip", "target": "storm_floating_ip" }, { "type": "cloudify.openstack.server_connected_to_security_group", "target": "storm_security_group" } ], "type": "dice.medium_host" }, "zookeeper": { "relationships": [ { "type": "cloudify.relationships.contained_in", "target": "zookeeper_host", "source_interfaces": { "cloudify.interfaces.relationship_lifecycle": { "preconfigure": "scripts/connect_zookeeper_servers.sh" } } } ], ………..

slide-21
SLIDE 21

And after that… DevOps!

21 TOSCA! TOSCA Deployment!

In a DevOps Fashion!

Architec t Deploy Monitor Analyse Re- Deploy

slide-22
SLIDE 22

Back to basics… DICE Views and Tools!

  • 4. Deploy improved-by-verification

architecture 19

slide-23
SLIDE 23
  • 5. Monitor and continuously

enhance/architect/deploy DIA

Back to basics… DICE Views and Tools!

  • 23 -

20

slide-24
SLIDE 24

Conclusions, take-home messages

  • MDE+TOSCA = DevOps Synergy!
  • Big-Data and TOSCA are fit for each other…
  • but technologies are not mature, yet!

24

slide-25
SLIDE 25

Bibliography

  • [1] A. Rajbhoj, V. Kulkarni, and N. Bellarykar. Early experience with model-driven development of mapreduce

based big data application. In Software Engineering Conference (APSEC), 2014 21st Asia-Pacific, volume 1, pages 94–97, Dec 2014.

  • [2] S. Santurkar, A. Arora, and K. Chandrasekaran. Stormgen - a domain specific language to create ad-hoc

storm topologies. In Computer Science and Information Systems (FedCSIS), 2014 Federated Conference on, pages 1621–1628, Sept 2014.

  • [3] Abel Gômez, Josê Merseguer, Elisabetta Di Nitto, Damian A. Tamburri “Towards a UML Profile for Data

Intensive Applications”, Proceedings of 2nd “Quality Aware DevOps” (QUDOS) workshop, to appear

  • [4] Elisabetta Di Nitto, Pooyan Jamshidi, Michele Guerriero, Ilias Spais, Damian A. Tamburri “A Software

Architecture Framework for Quality-Aware DevOps”, Proceedings of 2nd “Quality Aware DevOps” (QUDOS) workshop, to appear

  • [5] Matej Artacˇ, Tadej Borovs̍ak, Elisabetta Di Nitto, Michele Guerriero, Damian A. Tamburri, “Model-Driven

Continuous Deployment for Quality DevOps”, Proceedings of 2nd “Quality Aware DevOps” (QUDOS) workshop, to appear

  • [6] Guerriero, Michele; Tajfar, Saeed; Tamburri, Damian A. & Di Nitto, Elisabetta (2016), “Towards a model-

driven design tool for big data architectures.”, in 'BIGDSE@ICSE' , ACM, , pp. 37-43 .

  • 25 -