3 Common Pitfalls in Microservice Integration and How to Avoid Them @berndruecker
Bernd Ruecker Co-founder and Developer Advocate of Camunda Berlin, Germany bernd.ruecker@camunda.com @berndruecker
REST, SOAP, Cloud, Saas, Microservices, SCS, FaaS, Serverless, …
Distributed systems
https://www.infoworld.com/article/3254777/application-development/ 3-common-pitfalls-of-microservices-integrationand-how-to-avoid-them.html
Distributed systems Communication Challenges of Distributed is complex asynchronicity T ransactions
Some Service Some Some Service Service Some Service But keep it local! Some Service Failure will happen. Be resilient. Accept it! Some Some Service Service
Let‘s start with a simple example Credit Payment Card REST
Live hacking https://github.com/berndruecker/flowing-retail
Circuit Breaker Photo by CITYEDV, available under Creative Commons CC0 1.0 license.
Fai ail f l fast is important
Fai ail f l fast is important but not enough!
Photo by Tookapic, available under Creative Commons CC0 1.0 license.
„ There was an error while sending your boarding pass“
Current situation Web-UI Me Check-in
Current situation Web-UI Me Check-in Barcode Output Generator Mgmt
Current situation Web-UI Me Check-in Barcode Output Generator Mgmt
Current situation Circuit breaker Web-UI Me Check-in Barcode Output Generator Mgmt
Current situation – the bad part Web-UI Me Check-in Barcode Output Generator Mgmt
Current situation – the bad part Web-UI Me Check-in Barcode Output Generator Mgmt
Current situation – the bad part Web-UI Me Check-in Barcode Output Generator Stateful Mgmt Retry
We are having some technical difficulties and cannot present you your boarding pass right away. But we do actively retry ourselves, so lean back, relax and we will send it on time.
Possible situation – much better! Web-UI Me Check-in Barcode Output Generator Mgmt
Possible situation – much better! Web-UI Stateful Retry Me Check-in Barcode Output Generator Mgmt
Possible situation – much better! Web-UI Stateful The failure Retry never leaves Me Check-in this scope! Barcode Output Generator Mgmt
Handling State Persist thing State machine or (Entity, Document , Actor, …) workflow engine Complex, proprietary, DIY = effort, T ypical Scheduling, Versioning, heavyweight, slow, accidental concerns operating, visibility, don‘t scale, complexity scalability , … developer adverse
Workflow engines, state machines It is relevant in modern architectures
Workflow engines, CADENCE state machines Silicon valley has recognized
Workflow engines, CADENCE state machines
Workflow engines, CADENCE state machines also at scale
Workflow engines, CADENCE state machines for todays demo
Live hacking https://github.com/berndruecker/flowing-retail
Now you have a state machine! Credit Payment Card REST
Client Service Provider has to implement has to implement Retry Idempotency
It is a business We are processing your payment. problem Do not leave this page. anyway! And for god sake – do not reload!
It is a business We are currently processing your request. We are processing your payment. problem Don‘t worry, it will happen safely – Do not leave this page. even if you loose connection. anyway! And for god sake – do not reload! Feel free to reload this page any time!
Requirement: Idempotency of services! Photo by pixabay, available under Creative Commons CC0 1.0 license.
Requirement: Idempotency of services! Photo by Chr.Späth , available under Public Domain.
Distributed systems introduce complexity you have to tackle! Credit Payment Card REST
Distributed systems
It is impossible to Client Service differentiate certain Provider failure scenarios. Independant of communication style!
Distributed systems introduce complexity you have to tackle! Credit Payment Card REST
Distributed systems introduce complexity you have to tackle! Credit Payment Card REST
BPMN Business Process Model and Notation ISO Standard
Widespread BPMN Executable Easy to and mature understand* *(for Biz, Dev, and Ops)
Proper Operations Visibility + Context
Compare to e.g. Step Functions
Workflows live within service boundaries Credit Payment Card REST
Workflows live within service boundaries Web-UI Me Check-in Barcode Generato Output r Mgmt
Workflows live within service boundaries Web-UI Me Check-in Barcode Generato Output r Mgmt
Manigfold architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
Manigfold architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
Manigfold architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
Manigfold architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
Check-in generateBoardingPass HTTP 200 OK HTTP 202 ACCEPTED Happy case: Synchronous response Otherwise: asynchronous
„The customer wants a synchronous response “ Web-UI Me Check-in „Eh – no !“ Barcode Output Generator Mgmt
Synchronous communication is the crystal meth of distributed programming T odd Montgomery and Martin Thompson in “How did we end up here” at GOT O Chicago 2015
Challenges of asynchronicity
Asynchronous communication Web-UI You need to monitor timeouts Me Check-in Barcode Output Generator Mgmt
Remember … Web-UI The failure never leaves Me Check-in this scope! Barcode Output Generator Mgmt
Workflow…
Workflow…
Client Service Provider has to implement has to implement Timeout, Retry Idempotency
Who uses a message bus?
Who has no problems operating a message bus? Dead messages | No context | Inaccesible payload | Hard to redeliver | Home- grown message hospitals | …
Manigfold architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
Manigfold architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
Manigfold architecture options https://blog.bernd-ruecker.com/architecture-options-to-run-a-workflow-engine-6c2419902d91
Avoids th thundering ering her erds Photo by Gopal Vijayaraghavan, available under Creative Commons BY 2.0 license.
https://aws.amazon.com/de/message/5467D2/ Photo by Gopal Vijayaraghavan, available under Creative Commons BY 2.0 license.
Distributed Distributed T T ransactions ransactions
Distributed systems
“ Grown- Ups Don’t Use Distributed T ransactions Pat Helland Distributed Systems Guru Worked at Amazon, Microsoft & Salesforce
Distributed transactions using compensation * Compensation
Eventual consistency T emporarily inconsistent state But only temporarily!
Live hacking https://github.com/berndruecker/flowing-retail
Client Service Provider has to implement has to offer Timeout, Retry, Compensation Compensation has to implement Idempotency
Client Service Provider Don‘t forget about state has to implement has to offer Timeout, Retry, Compensation Compensation has to implement Idempotency
Event-driven example also available Checkout Order Payment Inventory Shipping Monitor Human T asks H2 H2 https://github.com/flowing/flowing-retail/
Workflows live inside service boundaries
Sales-Order & Order-Fulfillment via Camunda for every order worldwide (Q2 2017: 22,2 Mio )
# Be aware of complexity of distributed systems # Know strategies and tools to handle it e.g. Circuit breaker (Hystrix) Workflow engine for stateful retry, waiting, timeout and compensation (Camunda)
Thank you!
Contact: mail@berndruecker.io @berndruecker Slides: https://berndruecker.io Blog: https://medium.com/berndruecker Code: https://github.com/berndruecker https://www.infoworld.com/article/3254777/ application-development/ 3-common-pitfalls-of-microservices- integrationand-how-to-avoid-them.html https://www.infoq.com/articles/events- workflow-automation https://thenewstack.io/5-workflow-automation- use-cases-you-might-not-have-considered/
Recommend
More recommend
