Building Evolutionary #OSCON Architectures S UPPORT C ONSTANT C - - PowerPoint PPT Presentation

▶
building evolutionary
SMART_READER_LITE
LIVE PREVIEW

Building Evolutionary #OSCON Architectures S UPPORT C ONSTANT C - - PowerPoint PPT Presentation

Dec 21, 2023 28 likes •1.51k views

Building Evolutionary #OSCON Architectures S UPPORT C ONSTANT C HANGE Mike Mason Zhamak Dehghani @mikemasonca @zhamakd OSCON Networking Opportunity What is Software Architecture? requirements Time g ility n accessibility reliability

slide-1
SLIDE 1

Building Evolutionary Architectures

SUPPORT CONSTANT CHANGE

@mikemasonca @zhamakd

#OSCON

Mike Mason Zhamak Dehghani

slide-2
SLIDE 2

OSCON Networking Opportunity

slide-3
SLIDE 3

What is Software Architecture?

slide-4
SLIDE 4
slide-5
SLIDE 5

requirements

Time

slide-6
SLIDE 6
slide-7
SLIDE 7

s a m p l i n g

accessibility reliability repeatability accountability extensibility reproducibility accuracy failure transparency resilience adaptability fault-tolerance responsiveness administrability fidelity reusability affordability flexibility robustness agility inspectability safety auditability installability scalability autonomy integrity seamlessness availability interchangeability self-sustainability compatibility interoperability serviceability composability learnability supportability configurability maintainability securability correctness manageability simplicity credibility mobility stability customizability modifiability standards compliance debugability modularity survivability degradability

  • perability

sustainability determinability

  • rthogonality

tailorability demonstrability portability testability dependability precision timeliness deployability predictability traceability discoverability process capabilities transparency distributability producibility ubiquity durability provability understandability effectiveness recoverability upgradability efficiency relevance usability

https://en.wikipedia.org/wiki/List_of_system_quality_attributes

ility

slide-8
SLIDE 8

evolvability

accessibility reliability repeatability accountability extensibility reproducibility accuracy failure transparency resilience adaptability fault-tolerance responsiveness administrability fidelity reusability affordability flexibility robustness agility inspectability safety auditability installability scalability autonomy integrity seamlessness availability interchangeability self-sustainability compatibility interoperability serviceability composability learnability supportability configurability maintainability securability correctness manageability simplicity credibility mobility stability customizability modifiability standards compliance debugability modularity survivability degradability

  • perability

sustainability determinability

  • rthogonality

tailorability demonstrability portability testability dependability precision timeliness deployability predictability traceability discoverability process capabilities transparency distributability producibility ubiquity durability provability understandability effectiveness recoverability upgradability efficiency relevance usability

evolvability

slide-9
SLIDE 9

Change

slide-10
SLIDE 10

Change

requirements ecosystem

slide-11
SLIDE 11

Business Change

slide-12
SLIDE 12

Change

requirements ecosystem ecosystem

slide-13
SLIDE 13

Dynamic Equilibrium Dynamic Equilibrium

slide-14
SLIDE 14

How is long term planning possible when things constantly change in unexpected ways?

slide-15
SLIDE 15
slide-16
SLIDE 16

Once I’ve built an architecture, how can I prevent it from gradually degrading over time?

slide-17
SLIDE 17

Building Evolutionary Architectures

GROUP ICEBREAKER - 5 MINS #OSCON

In each group, choose a system that

  • ne of your members has worked on.

Explain the architecture to the rest of the group — we’ll use this to anchor today’s workshop exercises. Be ready to share at the end!

slide-18
SLIDE 18

Building Evolutionary Architectures

DEFINING EVOLUTIONARY ARCHITECTURE #OSCON

slide-19
SLIDE 19

Evolutionary Architecture

An evolutionary architecture supports guided, incremental change across multiple dimensions.

slide-20
SLIDE 20

Evolutionary Architecture

An evolutionary architecture supports guided, incremental change across multiple dimensions. guided

slide-21
SLIDE 21

guided

evolutionary computing fitness function: a particular type of objective function that is used to summarize…how close a given design solution is to achieving the set aims.

slide-22
SLIDE 22

guided

slide-23
SLIDE 23

guided

An architectural fitness function provides an

  • bjective integrity assessment of some

architectural characteristic(s).

slide-24
SLIDE 24

Evolutionary Architecture

An evolutionary architecture supports guided, incremental change across multiple dimensions. incremental guided

slide-25
SLIDE 25

incremental

Production

survey

shipping

catalog star rating

improved star rating

commit/ unit test

01001001010101 01010101010101 00101010010010 00100100010001

functional test UAT staging

database

unit tested code functionally tested code deployed code

slide-26
SLIDE 26

Evolutionary Architecture

An evolutionary architecture supports guided, incremental change across multiple dimensions. multiple dimensions incremental

slide-27
SLIDE 27

multiple dimensions

slide-28
SLIDE 28

Evolutionary Architecture

An evolutionary architecture supports guided, incremental change across multiple dimensions. multiple dimensions

slide-29
SLIDE 29

Why evolutionary?

continual agile incremental emergent reactive

slide-30
SLIDE 30

Why evolutionary?

adaptable?

slide-31
SLIDE 31

Building Evolutionary Architectures

#OSCON

Penultima⬆e

slide-32
SLIDE 32

EA Spreadsheet

Penultima⬆e

✔ definition ! verification

slide-33
SLIDE 33

EA Spreadsheet

Penultima⬆e

✔ definition ! verification ✔ verification

slide-34
SLIDE 34

Building Evolutionary Architectures

FITNESS FUNCTIONS:

CATEGORIES AND EXAMPLES

#OSCON

slide-35
SLIDE 35

Categories of Fitness Functions

atomic holistic

run against a singular context and exercise one particular aspect of the architecture. run against a shared context and exercise a combination of architectural aspects such as security and scalability

slide-36
SLIDE 36

Categories of Fitness Functions

run based on a particular event, such as a developer executing a unit test, a deployment pipeline running unit tests, or a QA person performing exploratory testing. don’t run on a schedule, but instead execute constant verification of architectural aspect(s) such as transaction speed.

triggered continuous

slide-37
SLIDE 37

Categories of Fitness Functions

have a fixed result, such as the binary pass/fail of a unit test. rely on a shifting definition based on extra

  • context. Some values may be contingent on

circumstances, and most architects will accept lower performance metrics when operating at high scale.

dynamic static

slide-38
SLIDE 38

Categories of Fitness Functions

tests and other verification mechanism that run without human interaction. must involve at least one human.

automated manual

slide-39
SLIDE 39

Categories of Fitness Functions

architects may want to build a time component into assessing fitness

temporal break on upgrade

  • verdue library update
slide-40
SLIDE 40

Categories of Fitness Functions

Some architectures have specific concerns, such as special security or regulatory requirements

domain-specific

slide-41
SLIDE 41

Categories of Fitness Functions

architects will define most fitness functions at project inception as they elucidate the characteristics of the architecture… …some fitness functions will emerge during development of the system

emergent intentional

slide-42
SLIDE 42

Fitness Function

atomic holistic triggered continuous

slide-43
SLIDE 43

Fitness Function

atomic holistic triggered continuous

slide-44
SLIDE 44

Cyclic Dependency Function

clarkware.com/software/JDepend.html application

slide-45
SLIDE 45

Consumer Driven Contracts

martinfowler.com/articles/consumerDrivenContracts.html

slide-46
SLIDE 46

Fitness Function

atomic holistic triggered continuous

slide-47
SLIDE 47

Fitness Function

atomic holistic triggered continuous

slide-48
SLIDE 48

holistic triggered

Holistic fitness functions must run in a specific (shared) context.

slide-49
SLIDE 49

Fitness Function

atomic holistic triggered continuous

slide-50
SLIDE 50

Fitness Function

atomic holistic triggered continuous

slide-51
SLIDE 51

atomic continuous

monitoring logging

slide-52
SLIDE 52

Synthetic Transactions

Sam Newman

Building Microservices

DESIGNING FINE-GRAINED SYSTEMS

Use synthetic transactions to test production systems.

slide-53
SLIDE 53

Fitness Function

atomic holistic triggered continuous

slide-54
SLIDE 54

Fitness Function

atomic holistic triggered continuous

slide-55
SLIDE 55
slide-56
SLIDE 56

System-wide Fitness Function

slide-57
SLIDE 57

Building Evolutionary Architectures

FITNESS FUNCTIONS:

TESTING AND AUTOMATION

#OSCON

slide-58
SLIDE 58

Testable

persistence web util

packages/namespaces

slide-59
SLIDE 59

Testable

persistence web util

packages/namespaces

slide-60
SLIDE 60

Testable

https://github.com/clarkware/jdepend/

slide-61
SLIDE 61

https://blog.jdriven.com/2017/10/implementing-architectural-fitness-functions-using-gradle-junit-code-assert/

slide-62
SLIDE 62
slide-63
SLIDE 63
slide-64
SLIDE 64
slide-65
SLIDE 65

ArchUnit

https://www.archunit.org/

slide-66
SLIDE 66

ArchUnit

https://www.archunit.org/

coding rules

slide-67
SLIDE 67

ArchUnit

https://www.archunit.org/

interface rules

slide-68
SLIDE 68

ArchUnit

https://www.archunit.org/

layer dependency

slide-69
SLIDE 69

ArchUnit

https://www.archunit.org/

governance

slide-70
SLIDE 70

Fitness Function Katas

http://evolutionaryarchitecture.com/ffkatas/

slide-71
SLIDE 71

Building Evolutionary Architectures

AFTERNOON TEA

@mikemasonca @zhamakd

#OSCON

Mike Mason Zhamak Dehghani

slide-72
SLIDE 72

Building Evolutionary Architectures

HYPOTHESIS AND DATA- DRIVEN DEVELOPMENT #OSCON

slide-73
SLIDE 73

Experiments to Perform

More Listings Better Structure Better Prioritization

slide-74
SLIDE 74

vision, strategy, business goals ideation portfolio

  • f experiments

selected experiments:

pivot fold double down

slide-75
SLIDE 75

Building Evolutionary Architectures

MOVE FAST AND FIX THINGS #OSCON

slide-76
SLIDE 76

https://github.com/github/scientist

slide-77
SLIDE 77

▫︎ It decides whether or not to run the try block, ▫︎ Randomizes the order in which use and try blocks are

run,

▫︎ Measures the durations of all behaviors, ▫︎ Compares the result of try to the result of use, ▫︎ Swallows (but records) any exceptions raised in the try

block

▫︎ Publishes all this information.

slide-78
SLIDE 78
slide-79
SLIDE 79
slide-80
SLIDE 80
slide-81
SLIDE 81

Building Evolutionary Architectures

ARCHITECTURAL CHARACTERISTICS #OSCON

slide-82
SLIDE 82
slide-83
SLIDE 83

Architecture Characteristics

reliability scalability

performance

availability

slide-84
SLIDE 84

accessibility evolvability repeatability accountability extensibility reproducibility accuracy failure transparency resilience adaptability fault-tolerance responsiveness administrability fidelity reusability affordability flexibility robustness agility inspectability safety auditability installability scalability autonomy integrity seamlessness availability interchangeability self-sustainability compatibility interoperability serviceability composability learnability supportability configurability maintainability securability correctness manageability simplicity credibility mobility stability customizability modifiability standards compliance debugability modularity survivability degradability

  • perability

sustainability determinability

  • rthogonality

tailorability demonstrability portability testability dependability precision timeliness deployability predictability traceability discoverability process capabilities transparency distributability producibility ubiquity durability provability understandability effectiveness recoverability upgradability efficiency relevance usability reliability

https://en.wikipedia.org/wiki/List_of_system_quality_attributes

Architecture Characteristics

slide-85
SLIDE 85

Architecture Characteristics

For each of the following business challenges, decide on the appropriate “ilities” that would be necessary in the system you learned about in the icebreaker. For each characteristic, how would you measure it in the example system?

slide-86
SLIDE 86

“our business is constantly changing to meet new demands of the marketplace” extensibility, maintainability, agility, modularity

Architecture Characteristics

slide-87
SLIDE 87

“due to new regulatory requirements, it is imperative that we complete end-

  • f-day processing in time”

performance, scalability, availability, reliability

Architecture Characteristics

slide-88
SLIDE 88

“we need faster time to market to remain competitive” maintainability, agility, modularity, deployability, testability

Architecture Characteristics

slide-89
SLIDE 89

“our plan is to engage heavily in mergers and acquisitions in the next three years” scalability, extensibility, openness, standards-based, agility, modularity

Architecture Characteristics

slide-90
SLIDE 90

“we have a very tight timeframe and budget for this project” feasibility

Architecture Characteristics

slide-91
SLIDE 91

architecture patterns help define the basic characteristics and behavior of the application

slide-92
SLIDE 92

Domain Driven Design

slide-93
SLIDE 93

Bounded Context

+

slide-94
SLIDE 94

Architectural Quantum

quantum

component external library code Data code code module code code code

slide-95
SLIDE 95

Microservices Quantum

Checkout

component component component

database

component component component

reporting package search

component component component component component component

microservice container

slide-96
SLIDE 96

Architectural Quantum

An architectural quantum is an independently deployable component with high functional cohesion, which includes all the structural elements required for the system to function properly.

slide-97
SLIDE 97

Architectural Quantum

An architectural quantum is an independently deployable component with high functional cohesion, which includes all the structural elements required for the system to function properly.

Checkout

component component component

database

component component component

reporting package search

component component component component component component

microservice container

slide-98
SLIDE 98

Building Evolutionary Architectures

EVOLVABILITY OF ARCHITECTURAL STYLES #OSCON

slide-99
SLIDE 99

For Each Pattern:

incremental change guided change via fitness functions appropriate coupling architectural quantum

slide-100
SLIDE 100

Big Ball of Mud

slide-101
SLIDE 101

Big Ball of Mud

Rippling side effects for any change difficult because no clearly defined partitioning exists Epitome of inappropriate coupling the entire system

slide-102
SLIDE 102

Monoliths

slide-103
SLIDE 103

Unstructured Monoliths

user interface

Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class

slide-104
SLIDE 104

Unstructured Monoliths

user interface

Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class Class

the entire system Large quantum size hinders incremental change because high coupling requires deploying large chunks of the application. difficult but not impossible functionally almost as bad as the Big Ball of Mud

slide-105
SLIDE 105

Layered Monolith

presentation business rules persistence database

component component component component component component component component component

slide-106
SLIDE 106

Layered Monolith

presentation business rules persistence database

component component component component component component component component component

the entire system Selectively easy based on partitioning easier because structure is more apparent — good technical architecture partitioning

slide-107
SLIDE 107

Modular Monoliths

user interface

module

Class Class Class Class

module

Class Class Class

module

Class Class Class Class Class Class

module

Class Class

slide-108
SLIDE 108

Modular Monoliths

user interface

module

Class Class Class Class

module

Class Class Class

module

Class Class Class Class Class Class

module

Class Class

the entire system, with selective better granularity the degree of deployability

  • f the components determines the rate of

incremental change. Each component is functionally cohesive, with good interfaces between them and low coupling. easier to design and implement in this architecture because of good separation of components

slide-109
SLIDE 109

Microservices

checkout module module

database

ship module module

database

inventory module module

database

listing module module

database

accounts module module

database

bulk txn module module

database

service module module

database

routing module module

database

API layer

client requests client requests client requests

slide-110
SLIDE 110

Microservices

checkout module module database ship module module database inventory module module database listing module module database accounts module module database bulk txn module module database service module module database routing module module database API layer

client requests client requests client requests

extremely decoupled fine-grained quanta with well defined boundaries robust testing & fitness function culture deployment pipeline considered standard extremely fine grained 21st century DevOps practices

slide-111
SLIDE 111

“Serverless” Architecture

FaaS Function as a Service

slide-112
SLIDE 112

N/A critical

“Serverless” Architecture

deployment pipeline integration challenging redeploy code

slide-113
SLIDE 113

Building Evolutionary Architectures

BUILDING EVOLVABLE ARCHITECTURES: MECHANICS #OSCON

slide-114
SLIDE 114

Mechanics

  • 1. Identify dimensions affect by evolution
slide-115
SLIDE 115
  • 1. Identify dimensions affect by evolution
slide-116
SLIDE 116

Mechanics

  • 1. Identify dimensions affect by evolution
  • 2. Define Fitness Function(s) for Each

Dimension

slide-117
SLIDE 117
  • 2. Define Fitness Function(s) for Each

Dimension

slide-118
SLIDE 118

Mechanics

  • 1. Identify dimensions affect by evolution
  • 2. Define Fitness Function(s) for Each

Dimension

  • 3. Use Deployment Pipelines to Automate

Fitness Functions

slide-119
SLIDE 119
  • 3. Use Deployment Pipelines to Automate

Fitness Functions

commit/ unit test

01001001010101 01010101010101 00101010010010 00100100010001

functional test UAT holistic fitness functions

database

atomic fitness functions

unit tested code functionally tested code architecturally tested code deployed quantum integration environment

slide-120
SLIDE 120

Mechanics

  • 1. Identify dimensions affect by evolution
  • 2. Define Fitness Function(s) for Each

Dimension

  • 3. Use Deployment Pipelines to Automate

Fitness Functions

slide-121
SLIDE 121

Mechanics

  • 1. Identify dimensions affect by evolution
  • 2. Define Fitness Function(s) for Each

Dimension

  • 3. Use Deployment Pipelines to Automate

Fitness Functions

slide-122
SLIDE 122

Building Evolutionary Architectures

BUILDING EVOLVABLE ARCHITECTURES: GREENFIELD & BROWNFIELD ARCHITECTURES #OSCON

slide-123
SLIDE 123

Greenfield Projects

slide-124
SLIDE 124

— implement incremental change at project inception — fitness functions definition easier before implementation — architects don’t have to untangle legacy coupling points — protective fitness functions from project outset — choose architecture patterns that support evolution

Greenfield Projects

slide-125
SLIDE 125

Brownfield Projects

slide-126
SLIDE 126

Appropriate Coupling & Cohesion

presentation business rules persistence database

component component component component component component component component component

checkout module module database ship module module database inventory module module database listing module module database accounts module module database bulk txn module module database service module module database routing module module database API layer client requests client requests client requests
slide-127
SLIDE 127

Understand the business problem before choosing an architecture.

slide-128
SLIDE 128

Improve Engineering Practices

slide-129
SLIDE 129

What About COTS?*

— quantum size: the package — incremental change: generally scores poorly — appropriate coupling: generally scores poorly — fitness functions: generally scores poorly

*Commercial Off-the-shelf Software

very ^

slide-130
SLIDE 130

Work diligently to hold integration points to your level of maturity…

slide-131
SLIDE 131

…if that isn’t possible, realize that some parts of the system will be easier for developers to evolve than others.

slide-132
SLIDE 132

Building Evolutionary Architectures

PUTTING EVOLUTIONARY ARCHITECTURE INTO PRACTICE #OSCON

slide-133
SLIDE 133

Organizational Factors

slide-134
SLIDE 134

Incidentally Coupled Teams

user interface server-side DBA

slide-135
SLIDE 135

Autonomous Teams

Orders Shipping Inverse Conway Maneuver Catalog

slide-136
SLIDE 136

Autonomous Teams…

Orders Shipping Catalog

…Organized around Business Capability

slide-137
SLIDE 137

Product over Project

slide-138
SLIDE 138

Autonomous Teams…

Orders Shipping Catalog

slide-139
SLIDE 139

Product over Project

slide-140
SLIDE 140

Product over Project

— Projects are ephemeral — Projects isolate developers from operational aspects — Products live forever — Products have owners — Products consist of cross-functional teams

slide-141
SLIDE 141

Product over Project

Long-term company buy-in

slide-142
SLIDE 142

Team Coupling Characteristics

slide-143
SLIDE 143

Culture

— Does everyone on the team know what fitness functions are and consider the impact

  • f new tool or product choices on the ability to evolve new fitness functions?

— Are teams measuring how well their system meets their defined fitness functions? — Do engineers understand cohesion and coupling? — Are there conversations about what domain and technical concepts belong together? — Do teams choose solutions not based on what technology they want to learn, but

based on its ability to make changes?

— How are teams responding to business changes?

slide-144
SLIDE 144

Culture of Experimentation

— Bring ideas from outside — Encouraging explicit improvement — Spike and stabilize — Creating innovation time — Following set-based development — Connecting engineers with end-users

slide-145
SLIDE 145

Building Enterprise Fitness Functions

commit/ unit test

01001001010101 01010101010101 00101010010010 00100100010001

functional test UAT holistic fitness functions

database

atomic fitness functions

unit tested code functionally tested code architecturally tested code deployed quantum integration environment

slide-146
SLIDE 146

Legality of Open Source Libraries

Penultima⬆e

slide-147
SLIDE 147

commit/ unit test

01001001010101 01010101010101 00101010010010 00100100010001

functional test UAT holistic fitness functions

database

atomic fitness functions

unit tested code functionally tested code architecturally tested code deployed quantum integration environment

slide-148
SLIDE 148

Building Evolutionary Architectures

For more information: http://evolutionaryarchitecture.com

#OSCON