Name: Thomas Flynn ID: 16117743 Course: Information & Network - - PowerPoint PPT Presentation

Name: Thomas Flynn ID: 16117743 Course: Information & Network Security MEng Supervisor: Dr. Sean McGrath Project Title: Docker Containers Deployed Using Bluemix Containerization platform

➢ Linux OS ➢ Open source ➢ Packages application code

Platform as a service

➢ Integrates Docker ➢ Bare metal depoyment ➢ Container lifecycle management

Introduction

• DevOps and Infrastructure as code
• Requirements and analysis
• Technology components
• Overall architecture
• Use case diagram
• Mosca broker implementation
• HAProxy implementation
• Docker-compose implementation
• Jenkins pipeline implementation (demo)
• Node-RED and OpenWhisk implementation (demo)
• Conclusion

What is DevOps?

• DevOps is a philosophy that brings together operations and

development.

• It is an approach that promotes closer collaboration between

lines of business, development and IT operations

• It is an enterprise capability that enables the continuous

delivery, continuous deployment and continuous monitoring

• f applications

Infrastructure as code

• Infrastructure as code is a concept of describing your

infrastructure and its configuration as a script

• This allows environments to be replicated in a much less

error-prone manner

• Infrastructure automation brings agility to both development

and operations because any authorized team member can modify the scripts while applying good development practices

Requirements and Analysis

A student, who has been given the task of creating a smart campus analytics application that uses information obtained from university campus sensors, wants to use a microservices architecture to develop a public API. The student should be able to:

➢ Have all the necessary infrastructure in place to start writing code ➢ Obtain sensor data from the database ➢ Make code changes that trigger an automated build ➢ Have automated integration tests that prevent insecure commits ➢ See their application automatically deployed once all tests have passed

Step 1: Identify the business process

➢ Public API development automation process

Step 2: Identify IT proccesses that support the business processess Step 3: Identify Activities within each of the IT processes ITP1: Automated Infrastructure build process A1: Container build activity A2: Integration server trigger build activity ITP2: Automated Integration testing process A1: Set up testing environment activity A2: Run tests activity ITP3: Automated deployment process process A1: Image security scan activity A2: Container orchestration deployment activity

Technologies

➢ Open source MQTT broker written in Javascript ➢ MQTT 3.1 and 3.1.1 compliant ➢ QoS 0 and QoS 1 ➢ Various storage options for QoS 1 offline packets, and subscriptions ➢ Usable inside any other Node.js app ➢ Open source software load balancer ➢ Written in C ➢ Session consists of two TCP connections ➢ One from the client to the load balancer ➢ One from the load balancer to the server ➢ Loadbalancing policy specified in config file ➢ Open source, BSD licensed ➢ In-memory data store ➢ Can be used as a high-performance

database, a cache, and a message broker

➢ Various clients written in several languages ➢ Log aggregation ➢ Various data structures

OpenWhisk and Storage

➢ Managed NoSQL JSON database service ➢ Cloudant Geo ➢ Index – efficiently via algorithms optimized for spatial data ➢ Query – using complex polygons and geometric relations ➢ Visualize – with interactive maps, powered by Mapbox, directly in the Cloudant dashboard ➢ Open source, written in Go ➢ Time-series Database ➢ SQL like language Select count (schuman) from student_events Group by time(5m), schuman Where time > now() - 30m ➢ Serverless architecture ➢ Abstracts away infrastructure ➢ Makes it simple to deploy microservices ➢ Eliminates the need to manage your own message broker or deploy your own worker servers ➢ Triggers: A class of events emitted by event sources ➢ Actions: Encapsulates the actual code to be executed ➢ Rules: An association between a trigger and an action ➢ Packages: Describe external services in a uniform manner

Trigger, Action, Rules

OpenWhisk

Architecture

Use case diagram

Cark Park 1 Polygon

Schuman Building Polygon

OpenWhisk: Get Mobile Users Location(Schuman) Mobile-iotp topic: Car-iotp topic: Microservice: Update Schuman Building Continuous query(Schuman)

Schuman Polygon:

• NW 52.673497, -8.578025
• NE 52.673514, -8.577711
• SE 52.672793, -8.577627
• SW 52.672785, -8.577985

Mobile Topic IoT Device Topic

Management Services Sensor API

P:1883 Logging and Monitoring Subscribe P:6379 MQQT Broker Load Balancing

Web Socket

Publish

Load Balancing Architecture

Data cache P:1883 P:1883

Mosca Broker Implementation

Mosquitto Client publishes to “car-iotp” and “mobile-iotp” Mobile Topic Car Topic Mosca Broker running on Bluemix

Publisher Subscribers

HAProxy

HAProxy Config File Dockerfile Broker 2 Configured to have MQTT work on top of TCP Add restart command in order to

• veride default configurations

Define working directory for container “haproxy-start” will be the default command run when container is started Expose ports to other containers

• n same local network

Builds Container image Broker 1 Chooses the broker with the least amount of connections HAProxy listens for traffic on port 1883 Add HAProxy config file to image

Docker-compose

• A tool for defining and running multi-container Docker applications.
• With a Compose file you can configure your application’s services

3 step process

• 1: Define your app’s environment with a Dockerfile so it can be reproduced anywhere
• 2: Define the services that make up your app in docker-compose.yml so they can be run

together in an isolated environment.

• 3: Lastly, run docker-compose up and Compose will start and run your entire app

Docker-compose

Jenkins 2

• Jenkins Pipeline is a suite of plugins which supports implementing and

integrating continuous delivery pipelines into Jenkins

• Pipeline provides an extensible set of tools for modeling simple-to-

complex delivery pipelines "as code" via the Pipeline DSL

• IBM Cloud DevOps plugin

Jenkins Master

Jenkinsfile

Key Concepts

• Flows are a collection of nodes wired together to

exchange messages.

• Messages passed between nodes in Node-RED

are, by convention, JavaScript Objects called msg

• Nodes are the primary building block of Node-RED
• When messages are generated, they are consumed

and processed by nodes

• Wires define the connections between node input

and output endpoints in a flow.

Node-RED Flow

Format MQTT message OpenWhisk http request node config MQTT Broker Encode to Base64 Configure payload frequency Format header Convert to json Store in Cloudant

Msg.payload.response.result.payload

OpenWhisk & Cloudant

OpenWhisk Base64 Decode Action OpenWhisk Action REST Endpoint Extract msg.payload

Future work

• Redis and InfluxDB functionality
• Node-RED app linked to HAProxy
• Automate new IP address for HAProxy
• Logging and monitoring
• Load balance testing
• Mosca web application functionality
• Geospatial microservice feature
• HAProxy encryption
• Organise MQTT topic structure
• Deploy application using Kubernetes
• Perform A/B feature testing using Istio