Microservice Deployment Software Engineering II Sharif University of Technology MohammadAmin Fazli
Topics Continuous Integration & Microservices Continuous Delivery Deployment Artifacts Custom Images Environments Service Configuration Service-to-Host Mapping Physical to Virtual Reading: Building Microservices-Sam Newman-Chapter VI Deployment 2
Continuous Integration With CI, the core goal is to keep everyone in sync with each other, which we achieve by making sure that newly checked-in code properly integrates with existing code. A CI server detects that The code has been committed Checks it out Carries out some verification Making sure the code compiles Making sure tests pass As part of this process, we often create artifact(s) that are used for further validation, such as deploying a running service to run tests against it. Deployment 3
Continuous Integration Benefits Fast Feedback about code quality Automatic binary artifacts ’ creation All the code is version controlled and all artifacts can be recreated Traceability from deployed artifacts to the code Traceability over what tests were run over codes and artifacts CI checks: Jez Humble ’ s three questions he asks people to test if they really understand what CI is about Do you check in to mainline once per day? You need to make sure your code integrates. Do you have a suite of tests to validate your changes? Check if our code semantically integrates When the build is broken, is it the #1 priority of the team to fix it? Please avoid piling up non-integrated changes Deployment 4
CI & Microservices When thinking about microservices & CI, we need to think about how our CI builds map to individual microservices. 1 st solution: One repository+Monolithic build Deployment 5
CI & Microservices 1 st solution: Benefits: Fewer repositories to worry about Simpler build Downsides: A one-line change to a single service cause all the other services get verified and built and can be very time consuming If this one-line change to a service breaks the build, no other changes can be made to the other services until that break is fixed. If a change cause break, which team is in charge? Deployment 6
CI & Microservices 2 nd Solution: One code repository + Individual Build Check in/Check out process can be simpler as we have one repository to worry about Can be easily get into the habit of checking in source code for multiple services at once which can slip into making changes that couple services together. Deployment 7
CI & Microservices 3 rd Solution: Individual Repository + Individual Build When making a change, we run only the build and tests I need to. We get a single artifact to deploy. Alignment to team ownership is more clear. If you own the service, you own the repository and the build. Making changes across repositories can be more difficult in this world Deployment 8
Continuous Delivery The build pipeline concept gives us a nice way of tracking the progress of our software as it clears each stage, helping give us insight into the quality of our software. Ex: Running fast and slow tests together does not get us a fast feedback. If fast tests fail waiting for slow tests is useless. So it is better to run them in different stages. CD builds according to the pipeline concept. Continuous delivery is the approach whereby we get constant feedback on the production readiness of each and every check-in, and furthermore treat each and every check-in as a release candidate. Deployment 9
Continuous Delivery In CD we must model the process of getting our software from check-in to production and know where any given version of the software is. Different software stages must be modeled both manual or automated We really want a tool that embraces CD as a first-class concept. Tools that fully support CD allow you to define and visualize these pipelines, modeling the entire path to production for your software. By modeling the entire path to production for our software, we greatly improve visibility of the quality of our software, and can also greatly reduce the time taken between releases Deployment 10 10
Platform Specific Artifacts Most technology stacks have some sort of first-class artifact, along with tools to support creating and installing them. Rubby gems, Java Jars, Python eggs Depending on the technology stack, these artifacts may not be enough by themselves. Example: sometimes they need Apache/Nginx installation and configuration We may need some way of installing and configuring other software that we need in order to deploy and launch our artifacts. This is where automated configuration management tools like Puppet and Chef can help. These artifacts are specific to a certain technology stack. This may make deployment more difficult when we have a mix of technologies in play Automation can go a long way toward hiding the differences in the deployment mechanisms of the underlying artifacts. Deployment 11 11
Operating System Artifacts One way to avoid the problems associated with technology-specific artifacts is to create artifacts that are native to the underlying operating system. Rpm packages for CentOS, MSI for windows, deb for Ubuntu Advantages of using OS-specific artifacts: From a deployment point of view we don ’ t care what the underlying technology is. We just use the tools native to the OS to install the package. The OS tools can also help us uninstall and get information about the packages too, The OS tools may even provide package repositories that our CI tools can push to Downsides: Difficulty in creating the packages There is a great overhead when deploying on different OSes Deployment 12 12
Custom Images Preparing machines for a service is very time consuming even if automated configuration tools (Puppet, Chef, Ansible) are used. Installing a JVM needs some minutes Automatic tools can be smart and will avoid installing software that is already present. This does not mean that running the scripts on existing machines will always be fast, unfortunately, as running all the checks takes time. Moreover, we don ’ t want to allow for too much configuration drift. So we should have many machine preparations and it is a real drag. Configuration Drift is the phenomenon where running servers in an infrastructure become more and more different as time goes on, due to manual ad-hoc changes and updates, and general entropy. Deployment 13 13
Custom Images One approach to reducing this spin-up time is to create a virtual machine image that bakes in some of the common dependencies we use. When we want to deploy our software, we spin up an instance of this custom image, and all we have to do is install the latest version of our service. Because you build the image only once, when you subsequently launch copies of this image you don ’ t need to spend time installing your dependencies, as they are already there. Deployment 14 14
Custom Images Drawbacks: Building images can take a long time. This means that for developers you may want to support other ways of deploying services to ensure they don ’ t have to wait half an hour just to create a binary deployment. Some of the resulting images can be large. This could be a real problem if you ’ re creating your own images. Moving a huge image around a network is a real problem The tool chain required to build such an image varied from platform to platform. Building a VMWare image is different from building an AWS AMI, a Vagrant image, or a Rackspace image. Some of the tools are multi-platform like packer.io which has support for VMWare, AWS, Racksapce Cloud, Digital Ocean, Vagrant and … Deployment 15 15
Immutable Servers By storing all our configuration in source control, we are trying to ensure that we can automatically reproduce services and hopefully entire environments at will. But once we run our deployment process, what happens if someone comes along changes things independently of what is in source control? Configuration Drift-the code in source control no longer reflects the configuration of the running host. Immutable Servers Pattern: we can ensure that no changes are ever made to a running server. Instead, any change, no matter how small, has to go through a build pipeline in order to create a new machine. We can disable SSH Deployment 16 16
Environments Different environments in CD stages: Development Environments and Unit Testing One for Slower Tests One for UAT One for Performance Testing One for Production Deployment 17 17
Recommend
More recommend
