Getting a System to Production … and keeping it there Eoin Woods, Endava
Getting a System to Production and keeping it there Eoin Woods, - - PowerPoint PPT Presentation
Getting a System to Production and keeping it there Eoin Woods, - - PowerPoint PPT Presentation
Getting a System to Production and keeping it there Eoin Woods, Endava Content Introducing Production Systems What Goes Wrong in Production? Solutions for Production Systems Conclusions 3 Production Systems 4 What is a production
Content
Introducing Production Systems What Goes Wrong in Production? Solutions for Production Systems Conclusions
3
Production Systems
4
What is a production system?
5
Any system being used for real work
Why is Productionisation Hard?
No one teaches you about production
who do you talk to? what do they want? what is the definition of “done” ?
Production is difficult for developers
hard to access, interrogate, debug, change, ...
6
A new cast of characters
7
Developers
Development
Users
A new cast of characters
7
Production
Users Developers Auditors Operations Acquirers Infrastructure Business Management
Production is constrained
Highly controlled Content is all valuable Change can be difficult
8
Production is unpredictable
9
Production is highly visible!
10
You don’t own production
11
What goes wrong?
12
Performance surprises
Interactive load Batch time surprises System abusers!
“all transactions this year”, “average since 1967”, ...
13
Environment bombshells
Constraints and contention Unexpected behaviour Integration points
14
Failures happen
Software defects Platform failures Environment failures
15
Security tangles
Security is simple in Development Much more complex in Production!
16
Finding Solutions
17
Key requirements for production
Functionally correct
does what the business process requires
Stability
behaves predictably in all situations
Capacity
can process the workload required (at all times)
Security
limits access to those who are authorised to have it
18
Solution Framework
Correctness Stability Capacity Security Design Principles Technology Processes
19
Solution Framework
Correctness Stability Capacity Security Design Principles Technology Processes
Simplicity
19
Solution Framework
Correctness Stability Capacity Security Design Principles Technology Processes
Simplicity Resource Governor
19
Solution Framework
Correctness Stability Capacity Security Design Principles Technology Processes
Simplicity Resource Governor Threat Modelling
19
General Principles
One Team Automate Measure and Improve (feedback loops) Good Enough over Perfection
20
Timeless principles … today led by CD and DevOps
Example: Achieving Stability
21
Stability - design principles
Fail quickly
fail fast, timeouts
Isolate problems
flow control, circuit breakers, bulkheads, asynchronous integration
Ensure steady state operation
housekeeping, predictable resource allocation, governors, throttling
22
Stability - technology solutions
23
Stability - technology solutions
Fail fast
23
Stability - technology solutions
Fail fast Bulkhead
23
Stability - technology solutions
Timeouts Fail fast Bulkhead
23
Stability - technology solutions
Timeouts Fail fast Bulkhead Governor
23
Stability - technology solutions
Timeouts Circuit Breaker Fail fast Bulkhead Governor
23
Stability - technology solutions
Timeouts Circuit Breaker Fail fast Bulkhead Governor Housekeeping
23
Example - Circuit Breaker
Clear Checking Tripped
err_returned timeout err_returned && err_count > 10 err_returned
24
Stability - process principles
Repeatability
defined processes, practice scenarios, prelive environments
Automation
automate the routine, automate the difficult allow the human back in the loop on demand
Transparency
logging, monitoring, alerts, trends
25
Stability - process automation
Logging & Metrics Monitoring Automation
26
Summary
27
Production is just different
it’s not yours and you need to respect that
Production is demanding
Correctness Stability Capacity Security
Summary
28
Summary (ii)
Solutions for each requirement by area
principles technologies processes
29
Summary (iii)
Production requirements and principles go back to the age of the mainframe CD and DevOps the latest incarnation welcome attention from developers new tech enabling new possibilities breaking down silos to make it happen
30
Books
Software Systems Architecture
Second Edition
NICK ROZANSKI • EOIN WOODS
Working with Stakeholders Using Viewpoints and PerspectivesSecond Edition
31
Thanks !
34
Acknowledgements
http://www.icons-land.com http://www.alamy.com/ http://www.42u.com