In Integration solution in the Clo loud Berlin Expert Days, 16. - - PowerPoint PPT Presentation

▶
in
SMART_READER_LITE
LIVE PREVIEW

In Integration solution in the Clo loud Berlin Expert Days, 16. - - PowerPoint PPT Presentation

Aug 29, 2023 466 likes •1.09k views

Will trade an ESB for an agile In Integration solution in the Clo loud Berlin Expert Days, 16. September 2016, Urania Berlin @KayLerch | Engineering Manager | Immobilien Scout BEDCon 2016 | @KayLerch Agenda Disruptive forces and what

slide-1
SLIDE 1

Will trade an ESB for an agile

In Integration solution in the Clo loud

Berlin Expert Days,

  • 16. September 2016, Urania Berlin

@KayLerch | Engineering Manager | Immobilien Scout

slide-2
SLIDE 2

Agenda

BEDCon 2016 | @KayLerch

 Disruptive forces and what they do with enterprise IT  An ideal integration platform  AWS Simple Workflows (SWF) in a nutshell  Demo time  Leveraging SWF to get rid of a classical ESB solution  Reclaim process ownership and end-2-end-autonomy  Drawing the big picture of a hybrid integration solution

slide-3
SLIDE 3

Competitive forces and what they do with Enterprise IT

Competitive forces

Market IT of an E-commerce business Fast-paced feature development (“Lean IT”) Core business systems and services (“Classical IT”)

Innovation / Agility / Speed Agile teams Microservices / modularized DevOps Efficiency / compliance / reliability Project organization Legacy / monolithic Operations management Release-driven Deployments API-led Connectivity Proprietary Interfaces

Boosting time to market CLD

Continuous Deployment Product

BEDCon 2016 | @KayLerch

slide-4
SLIDE 4

“Two speed IT” parts heavily depend on each other not only for the good

Competitive forces

Market A digital business Fast-paced feature development (“Lean IT”) Core business systems and services (“Classical IT”)

Innovation / Agility / Speed Agile teams Microservices / modularized DevOps Continuous Deployment Efficiency / compliance / reliability Project organization Legacy / monolithic Operations management Release-driven Deployments API-led Connectivity Proprietary Interfaces

Boosting time to market CLD

Organizational clash Technical clash

„Two Speed IT“

Product

BEDCon 2016 | @KayLerch

slide-5
SLIDE 5

Overcome technical burden with Integration strategy (SOA?)

Competitive forces

Market A digital business Fast-paced feature development (“Lean IT”) Core business systems and services (“Classical IT”)

Innovation / Agility / Speed Agile teams Microservices / modularized DevOps Continuous Deployment Efficiency / compliance / reliability Project organization Legacy / monolithic Operations management Release-driven Deployments API-led Connectivity Proprietary Interfaces

Boosting time to market CLD

„Two Speed IT“

Product

BEDCon 2016 | @KayLerch

Organizational clash Technical clash

ICC ESB

slide-6
SLIDE 6

Well, not … Integration projects tend to be traditional (often) for good reason

Competitive forces

Market A digital business Fast-paced feature development (“Lean IT”) Core business systems and services (“Classical IT”)

Innovation / Agility / Speed Agile teams Microservices / modularized DevOps Continuous Deployment Efficiency / compliance / reliability Project organization Legacy / monolithic Operations management Release-driven Deployments API-led Connectivity Proprietary Interfaces

Boosting time to market CLD

„Two Speed IT“

Product

BEDCon 2016 | @KayLerch

Organizational clash Technical clash

ICC ESB

slide-7
SLIDE 7

An integration strategy should follow a “bimodal” approach where so- called hybrid integration platforms (HIP) strive for Self Service Integration

Competitive forces

Market A digital business Fast-paced feature development (“Lean IT”) Core business systems and services (“Classical IT”)

Innovation / Agility / Speed Agile teams Microservices / modularized DevOps Continuous Deployment Efficiency / compliance / reliability Project organization Legacy / monolithic Operations management Release-driven Deployments API-led Connectivity Proprietary Interfaces

Boosting time to market CLD

„Two Speed IT“

Product

ISG HIP

BEDCon 2016 | @KayLerch

slide-8
SLIDE 8

The digital ecosystem brings a lot of new potential but also threat

Mobile IoT User Data Cloud Social Networks Digital marketplace

Digital ecosystem

discruptive BEDCon 2016 | @KayLerch

slide-9
SLIDE 9

Businesses need to adopt and integrate these potential

Mobile IoT User Data Cloud Social Networks Digital marketplace

Digital ecosystem

discruptive BEDCon 2016 | @KayLerch

slide-10
SLIDE 10

An HIP got its name from being the gateway for those new stuff

Mobile IoT User Data Cloud Social Networks Digital marketplace

Digital ecosystem

discruptive BEDCon 2016 | @KayLerch

slide-11
SLIDE 11

HIP is a concept whereas iPaaS, iSaaS are (commercial) solutions in the cloud

Mobile IoT User Data Cloud Social Networks Digital marketplace

Digital ecosystem

discruptive

iPaaS iSaaS

BEDCon 2016 | @KayLerch

slide-12
SLIDE 12

Ok, you got your BINGO! Let’s move on in the real world …

BEDCon 2016 | @KayLerch

slide-13
SLIDE 13

Agenda

BEDCon 2016 | @KayLerch

 Disruptive forces and what they do with enterprise IT  An ideal integration platform  AWS Simple Workflows (SWF) in a nutshell  Demo time  Leveraging SWF to get rid of a classical ESB solution  Reclaim process ownership and end-2-end-autonomy  Drawing the big picture of a hybrid integration solution

slide-14
SLIDE 14

Given an ESB in charge of syncing business data across the enterprise

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

BEDCon 2016 | @KayLerch

slide-15
SLIDE 15

This centralized hub is key for processes arching over multiple domains

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

BEDCon 2016 | @KayLerch

slide-16
SLIDE 16

It encapsulates the burden of integrating system interfaces with brokers

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article { } { } { } { }

BEDCon 2016 | @KayLerch

slide-17
SLIDE 17

Teams depend on those brokers in order to change their interfaces

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article { } { } { } { }

API API API API

BEDCon 2016 | @KayLerch

slide-18
SLIDE 18

It would be great to push responsibility (integration logic) to the edges

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

API API API API

{ } { } { } { }

BEDCon 2016 | @KayLerch

slide-19
SLIDE 19

It would also be great to delegate process ownership (orchestration logic)

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

API API API API

{ } { } { } { }

BEDCon 2016 | @KayLerch

slide-20
SLIDE 20

ESB left over for “dirty work” (messaging, tracking, governance, …)

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

API API API API

{ } { } { } { }

BEDCon 2016 | @KayLerch

slide-21
SLIDE 21

It should also be in the cloud to not be isolated from the digital ecosystem

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

API API API API

{ } { } { } { }

BEDCon 2016 | @KayLerch

slide-22
SLIDE 22

It should also be in the cloud to not be isolated from the digital ecosystem

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

API API API API

{ } { } { } { }

BEDCon 2016 | @KayLerch

slide-23
SLIDE 23

Why can’t this thing just be like an HIP?

HIP

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

API API API API

{ } { } { } { }

BEDCon 2016 | @KayLerch

slide-24
SLIDE 24

As our company moves to AWS anyway, let’s give SWF a try

  • There is PAYG
  • AWS ecosystem is huge, it serves all the hip stuff like serverless architecture (Lambda, API Gateway),

IoT, Mobile integration, Messaging, Elastic computing, Container deployments and more

  • There‘s a big community around AWS
  • SWF (Simple Workflows) is used by NASA for processing data from the Mars-Rover on earth

You got me at „Mars“…

BEDCon 2016 | @KayLerch

slide-25
SLIDE 25

Agenda

BEDCon 2016 | @KayLerch

 Disruptive forces and what they do with enterprise IT  An ideal integration platform  AWS Simple Workflows (SWF) in a nutshell  Demo time  Leveraging SWF to get rid of a classical ESB solution  Reclaim process ownership and end-2-end-autonomy

slide-26
SLIDE 26

SWF is a workflow engine scheduling tasks for all the workflow participants

BEDCon 2016 | @KayLerch

slide-27
SLIDE 27

A workflow starter simply kicks off a workflow with some input via API

1

BEDCon 2016 | @KayLerch

{ "childPolicy": "string", "domain": "string", "executionStartToCloseTimeout": "string", "input": "string", "lambdaRole": "string", "tagList": [ "string" ], "taskList": { "name": "string" }, "taskPriority": "string", "taskStartToCloseTimeout": "string", "workflowId": "string", "workflowType": { "name": "string", "version": "string" } } { "runId": "string" }

slide-28
SLIDE 28

SWF has no clue what comes next so it schedules a “decision task”

2

BEDCon 2016 | @KayLerch

slide-29
SLIDE 29

A decider owns the actual workflow logic. It is an application polling for tasks via API and returns decisions to the SWF engine

3

BEDCon 2016 | @KayLerch

{ "domain": "string", "identity": "string", "maximumPageSize": number, "nextPageToken": "string", "reverseOrder": boolean, "taskList": { "name": "string" } } { "decisions": [ { "cancelTimerDecisionAttributes": { … }, "cancelWorkflowExecutionDecisionAttributes": { … }, "completeWorkflowExecutionDecisionAttributes": { … }, "continueAsNewWorkflowExecutionDecisionAttributes": { … }, "recordMarkerDecisionAttributes": { … }, "requestCancelActivityTaskDecisionAttributes": { … }, "requestCancelExternalWorkflowExecutionDecisionAttributes": { … }, "scheduleActivityTaskDecisionAttributes": { … }, "scheduleLambdaFunctionDecisionAttributes": { … }, "signalExternalWorkflowExecutionDecisionAttributes": { … }, "startChildWorkflowExecutionDecisionAttributes": { … }, "startTimerDecisionAttributes": { … } } ], "executionContext": "string", "taskToken": "string" }

slide-30
SLIDE 30

SWF fulfills the remotely given decisions –> e.g. it schedules an activity task

4

BEDCon 2016 | @KayLerch

slide-31
SLIDE 31

An activity worker polls for tasks from a task list, works on it and returns a result

5

BEDCon 2016 | @KayLerch

{ "domain": "string", "identity": "string", "taskList": { "name": "string" } } { "result": "string", "taskToken": "string" } { "details": "string", "reason": "string", "taskToken": "string" }

slide-32
SLIDE 32

SWF receives the result – again has no clue how to go on – so it schedules another decision task

6

BEDCon 2016 | @KayLerch

slide-33
SLIDE 33

The decider receives the task. It now contains a detailed protocol of what happened in this workflow-execution. Based on that it gives the next decision(s)

7

BEDCon 2016 | @KayLerch

slide-34
SLIDE 34

SWF schedules the next task – according to the last decision by the decider. Activity workers can be serverless Lambda-functions as well

8

BEDCon 2016 | @KayLerch

slide-35
SLIDE 35

This goes on and on until the decider aka workflow worker decides for completing the workflow. What again is executed by SWF itself.

BEDCon 2016 | @KayLerch

slide-36
SLIDE 36

You keep track of your workflow executions in AWS console

BEDCon 2016 | @KayLerch

slide-37
SLIDE 37

AWS CloudWatch provides performance metrics for your workflows

BEDCon 2016 | @KayLerch

slide-38
SLIDE 38

Agenda

BEDCon 2016 | @KayLerch

 Disruptive forces and what they do with enterprise IT  An ideal integration platform  AWS Simple Workflows (SWF) in a nutshell  Demo time  Leveraging SWF to get rid of a classical ESB solution  Reclaim process ownership and end-2-end-autonomy  Drawing the big picture of a hybrid integration solution

slide-39
SLIDE 39

SWF-powered IoT-Photobooth

BEDCon 2016 | @KayLerch

2) Shoot a photo View progress on breadboard 3) View on Slack 1) Press a button

slide-40
SLIDE 40

AWS Lambda AWS SWF

Lambda Task

Lambda Upload to S3 & Start Workflow Lambda Update Shadow

Lambda Task

Lambda Get S3 Photo and Post Message to Slack

Lambda Task Lambda Task Lambda Task

Lambda Shorten Url with Bitly Lambda Update Shadow Lambda Update Shadow

AWS EC2

WorkflowWorker DeciderImpl

Raspberry Pi

AWS IoT

Thing Shadow Python Script GPIO Rule

AWS S3

Bucket

Put File Start WF Invoke MQTT Publish MQTT Receive Update Shadow Poll Tasks Shoot Impulse Impulse

slide-41
SLIDE 41

Agenda

BEDCon 2016 | @KayLerch

 Disruptive forces and what they do with enterprise IT  An ideal integration platform  AWS Simple Workflows (SWF) in a nutshell  Demo time  Leveraging SWF to get rid of a classical ESB solution  Reclaim process ownership and end-2-end-autonomy  Drawing the big picture of a hybrid integration solution

slide-42
SLIDE 42

So let’s use SWF for replacing the ESB in our company.

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article { } { } { } { }

API API API API

BEDCon 2016 | @KayLerch

slide-43
SLIDE 43

Client-side worker integrate their services by pulling tasks from SWF

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

API API API API

wor ker wor ker wor ker wor ker

Acti vity Acti vity Acti vity Acti vity

AWS SWF

BEDCon 2016 | @KayLerch

slide-44
SLIDE 44

Workers poll for tasks in task lists associated with declarative activity type

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

API API API API

Acti vity Acti vity Acti vity Acti vity Task list Task list Task list Task list

HTTP REST HTTP REST HTTP REST HTTP REST

AWS SWF

wor ker wor ker wor ker wor ker

BEDCon 2016 | @KayLerch

slide-45
SLIDE 45

Workers can be Lambda functions as well in charge of the edge team

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

API API

Acti vity Acti vity Acti vity Acti vity Task list Task list

HTTP REST HTTP REST

AWS SWF

wor ker wor ker lam bda

AWS

lam bda

AWS

Event Trigger Event Trigger

BEDCon 2016 | @KayLerch

slide-46
SLIDE 46

That’s nice, but there’s still a central orchestration controlled by one decider

ESB

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

API API

Acti vity Acti vity Acti vity Acti vity Task list Task list

HTTP REST HTTP REST

AWS SWF

wor ker wor ker lam bda

AWS

lam bda

AWS

Event Trigger Event Trigger

BEDCon 2016 | @KayLerch

slide-47
SLIDE 47

Delegate parts of the orchestration in child workflows reclaims process ownership

SINA

Salesforce

CRM Sales CC

ERP

AMG Finance

Checkout Service

ARTE

Fraud Service

SHIELD

Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article

API API

HTTP REST

AWS SWF

wor ker lam bda

AWS

Event Trigger Event Trigger HTTP REST

AWS

Acti vity Task list

wor ker

Sub WF Acti vity

BEDCon 2016 | @KayLerch

Task list

slide-48
SLIDE 48

Agenda

BEDCon 2016 | @KayLerch

 Disruptive forces and what they do with enterprise IT  An ideal integration platform  AWS Simple Workflows (SWF) in a nutshell  Demo time  Leveraging SWF to get rid of a classical ESB solution  Reclaim process ownership and end-2-end-autonomy  Drawing the big picture of a hybrid integration solution

slide-49
SLIDE 49

Reclaim process ownership leads to real end-2-end-autonomy of teams

Sales

BEDCon 2016 | @KayLerch

slide-50
SLIDE 50

End-2-End autonomy along the whole lifecycle of a worker

Sales

BEDCon 2016 | @KayLerch

slide-51
SLIDE 51

We call them Microworker as this is what they really are

Sales

BEDCon 2016 | @KayLerch

slide-52
SLIDE 52

Agenda

BEDCon 2016 | @KayLerch

 Disruptive forces and what they do with enterprise IT  An ideal integration platform  AWS Simple Workflows (SWF) in a nutshell  Demo time  Leveraging SWF to get rid of a classical ESB solution  Reclaim process ownership and end-2-end-autonomy  Drawing the big picture of a hybrid integration solution

slide-53
SLIDE 53
slide-54
SLIDE 54
slide-55
SLIDE 55
slide-56
SLIDE 56

Q & A

BEDCon 2016 | @KayLerch

It‘s time to ask questions and give feedback. Thanks for joining my session ...

slide-57
SLIDE 57

Backup (SWF at JPL Datacenter of NASA)

Sales

BEDCon 2016 | @KayLerch

slide-58
SLIDE 58

Backup (SWF-powered processing of images from Mars rover)

Sales

BEDCon 2016 | @KayLerch

slide-59
SLIDE 59

Backup (SWF-powered contract data distribution at Scout24)

Sales

BEDCon 2016 | @KayLerch

slide-60
SLIDE 60

Backup (SWF-powered contract data distribution at Scout24)

Sales

BEDCon 2016 | @KayLerch