Jenkins as a code ukasz Szczsny & Marcin Zajczkowski FOSDEM, - - PowerPoint PPT Presentation

▶
jenkins as a code
SMART_READER_LITE
LIVE PREVIEW

Jenkins as a code ukasz Szczsny & Marcin Zajczkowski FOSDEM, - - PowerPoint PPT Presentation

Apr 10, 2023 173 likes •771 views

Jenkins as a code ukasz Szczsny & Marcin Zajczkowski FOSDEM, 30-31th January 2016 ukasz Szczsny & Marcin Zajczkowski @wybczu & @SolidSoftBlog About ukasz Software engineer @ Uber FOSS and Open Hardware lover

slide-1
SLIDE 1

Jenkins as a code

Łukasz Szczęsny & Marcin Zajączkowski

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

FOSDEM, 30-31th January 2016

slide-2
SLIDE 2

About Łukasz

Software engineer @ Uber FOSS and Open Hardware lover Co-organizer of the Warsaw Linux User Group

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-3
SLIDE 3

About Marcin

Areas of expertise Automatic Testing / TDD Software Craftsmanship / Code Quality Java8 / Groovy Concurrency / Parallel Computing / Reactive Systems Deployment Automation / Continuous Delivery FOSS projects author and contributor, blogger Leads a small software house - Codearte targeted at clients who care about the quality

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-4
SLIDE 4

Agenda

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-5
SLIDE 5

Agenda

Manual Jenkins maintenance

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-6
SLIDE 6

Agenda

Manual Jenkins maintenance Job configuration as code Jenkins Job DSL

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-7
SLIDE 7

Agenda

Manual Jenkins maintenance Job configuration as code Jenkins Job DSL Infrastructure as code

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-8
SLIDE 8

Agenda

Manual Jenkins maintenance Job configuration as code Jenkins Job DSL Infrastructure as code Case study - Continuous Delivery in Jenkins

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-9
SLIDE 9

Manual Jenkins maintenance

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-10
SLIDE 10

Manual Jenkins maintenance

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-11
SLIDE 11

Manual Jenkins maintenance

configuration via GUI does not scale slow, error prone, and boring

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-12
SLIDE 12

Manual Jenkins maintenance

configuration via GUI does not scale slow, error prone, and boring problematic with dozens of jobs and plugins

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-13
SLIDE 13

Manual Jenkins maintenance

configuration via GUI does not scale slow, error prone, and boring problematic with dozens of jobs and plugins mission impossible with several microservices deployed in several countries for multiple products using deployment pipeline with several steps each

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-14
SLIDE 14

Automation to the rescue!

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-15
SLIDE 15

Jenkins Job DSL

Job configuration in code

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-16
SLIDE 16

(Jenkins) Job DSL - 2 parts

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-17
SLIDE 17

(Jenkins) Job DSL - 2 parts

Domain Specific Language to specify job configuration

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-18
SLIDE 18

(Jenkins) Job DSL - 2 parts

Domain Specific Language to specify job configuration Jenkins plugin to transform configuration DSL into real jobs in Jenkins

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-19
SLIDE 19

Job DSL - part 1 - configuration

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-20
SLIDE 20

Job DSL - part 1 - configuration

Groovy based DSL (Domain Specific Language) job/view/dashboard configuration developed as "normal" code in IDE with auto-completion type check Groovy magic if needed

  • utside Jenkins instance

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-21
SLIDE 21

Job DSL - simple example

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-22
SLIDE 22

Job DSL - simple example

job('FOSDEM-website-publish') { scm { github('FOSDEM/website') } triggers { scm('*/15 * * * *') } steps { rake('publish') } }

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-23
SLIDE 23

Job DSL - dynamic example

String repo = 'FOSDEM/mobile-app' URL branchUrl = "https://api.github.com/repos/$repo/branches".toURL() List branches = new JsonSlurper().parseText(branchUrl.text) branches.each { branch -> String safeBranchName = branch.name.replaceAll('/', '-') job("$repo-$safeBranchName-build") { scm { github repo, branch.name } triggers { scm 'H/10 * * * *' } steps { gradle 'check' } } }

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-24
SLIDE 24

Job DSL - features

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-25
SLIDE 25

Job DSL - features

comprehensive support for Jenkins Core stuff extensive support for additional plugins 177 plugins in version 1.42 active community - continuous flow of new pull requests

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-26
SLIDE 26

Job DSL - features

comprehensive support for Jenkins Core stuff extensive support for additional plugins 177 plugins in version 1.42 active community - continuous flow of new pull requests powerful configuration block for not yet supported features custom stuff virtually everything possible in XML should be achievable

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-27
SLIDE 27

Job DSL - part 2 - Jenkins plugin

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-28
SLIDE 28

Job DSL - part 2 - Jenkins plugin

installed in Jenkins instance used in seed jobs on Jenkins leverages DSL configuration updates jobs & views in Jenkins to bring them to desired state by XML configuration files modification

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-29
SLIDE 29

Job DSL - benefits

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-30
SLIDE 30

Job DSL - benefits

source code instead of XML or GUI single source of truth manageable jobs and views backed by SCM reviewable - possibly with pull requests

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-31
SLIDE 31

Job DSL - benefits

source code instead of XML or GUI single source of truth manageable jobs and views backed by SCM reviewable - possibly with pull requests testable automatic "unit" testing pre-production environment

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-32
SLIDE 32

Job DSL - benefits

source code instead of XML or GUI single source of truth manageable jobs and views backed by SCM reviewable - possibly with pull requests testable automatic "unit" testing pre-production environment scalable hundreds of jobs created/modified in seconds*

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-33
SLIDE 33

Job DSL - drawbacks/limitations

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-34
SLIDE 34

Job DSL - drawbacks/limitations

quite steep learning curve can become hard to understand for complex configurations

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-35
SLIDE 35

Job DSL - drawbacks/limitations

quite steep learning curve can become hard to understand for complex configurations small error in DSL can remove all jobs can be easily recreated, but without execution history

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-36
SLIDE 36

Job DSL - drawbacks/limitations

quite steep learning curve can become hard to understand for complex configurations small error in DSL can remove all jobs can be easily recreated, but without execution history

  • ccasional backward compatibility issues with new

versions

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-37
SLIDE 37

Job DSL - drawbacks/limitations

quite steep learning curve can become hard to understand for complex configurations small error in DSL can remove all jobs can be easily recreated, but without execution history

  • ccasional backward compatibility issues with new

versions very old Groovy 1.8.9 - version bundled in Jenkins 1.x

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-38
SLIDE 38

Job DSL - drawbacks/limitations

quite steep learning curve can become hard to understand for complex configurations small error in DSL can remove all jobs can be easily recreated, but without execution history

  • ccasional backward compatibility issues with new

versions very old Groovy 1.8.9 - version bundled in Jenkins 1.x not suitable for global Jenkins configuration management credentials, machine provisioning, Jenkins and plugin update, ...

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-39
SLIDE 39

Infrastructure

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-40
SLIDE 40

Infrastructure challenges

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-41
SLIDE 41

Infrastructure challenges

install and configure Jenkins master install and configure all required dependencies install and configure plugins create and connect slaves add JDK installation configure authentication create credentials ...

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-42
SLIDE 42

Infrastructure toolbelt

configuration management tools Ansible Puppet Chef Salt etc. Groovy console Jenkins CLI

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-43
SLIDE 43

Infrastructure toolbelt

Slave management Swarm plugin Docker plugin SSH slaves

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-44
SLIDE 44

Continuous Delivery in Jenkins

Case study

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-45
SLIDE 45

Continuous Delivery

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-46
SLIDE 46

Continuous Delivery

Clearly defined way how to transform source code into project deployed to production a set of steps arranged into pipeline unified way for various projects/variants/realms

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-47
SLIDE 47

Continuous Delivery in Jenkins

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-48
SLIDE 48

Continuous Delivery in Jenkins

not a first class citizen in Jenkins 1.x bunch of jobs triggering each other can be emulated with various plugins Delivery Pipeline Plugin, Build Flow Plugin, Pipeline Plugin, ...

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-49
SLIDE 49

Continuous Delivery in Jenkins

not a first class citizen in Jenkins 1.x bunch of jobs triggering each other can be emulated with various plugins Delivery Pipeline Plugin, Build Flow Plugin, Pipeline Plugin, ... no easy (and unified) way to setup usually even harder to maintain

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-50
SLIDE 50

Continuous Delivery - case study

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-51
SLIDE 51

Continuous Delivery - case study

custom Continuous Delivery framework

  • n top of Jenkins Job DSL
  • ne standardized way for Continuous Delivery

reused in all projects in the company

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-52
SLIDE 52

Continuous Delivery - case study

custom Continuous Delivery framework

  • n top of Jenkins Job DSL
  • ne standardized way for Continuous Delivery

reused in all projects in the company Ansible for infrastructure management Rundeck for deployment

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-53
SLIDE 53

Continuous Delivery - case study

custom Continuous Delivery framework

  • n top of Jenkins Job DSL
  • ne standardized way for Continuous Delivery

reused in all projects in the company Ansible for infrastructure management Rundeck for deployment

  • pen sourced to make live easier to others

jenkins-pipeline-dsl - core library sample-jenkins-microservice-pipeline - sample pipeline

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-54
SLIDE 54

Live demo

The whole delivery pipeline with one click!

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-55
SLIDE 55

Summary

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-56
SLIDE 56

Summary

automation is good for you! Jenkins DSL is great but there are other tools jenkins-job-builder Pipeline (formerly Workflow) plugin ... looking forward to Jenkins 2.0 release!

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-57
SLIDE 57

Questions?

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog

slide-58
SLIDE 58

Marcin Zajączkowski

@SolidSoftBlog http://blog.solidsoft.info m.zajaczkowski@gmail.com IRC: szpak@freenode

Łukasz Szczęsny

@wybczu https://wybcz.pl luk@{uber.com,wybcz.pl} IRC: wybczu@freenode

Thank you

Łukasz Szczęsny & Marcin Zajączkowski @wybczu & @SolidSoftBlog