SLIDE 1
Failing Gracefully As A Feature
Lorne Kligerman
Director of Product, Gremlin
@lklig
2
Failing Gracefully As A Feature Lorne Kligerman Director of - - PowerPoint PPT Presentation
Failing Gracefully As A Feature Lorne Kligerman Director of - - PowerPoint PPT Presentation
Failing Gracefully As A Feature Lorne Kligerman Director of Product, Gremlin @lklig 2 3 T-Ho 2017 Hey team bit of a spill but Im fine. Be down in 10! 4 We Expect Technology To Just Work 5 Black Friday Failures Technical
SLIDE 2
SLIDE 3
3
SLIDE 4
4
Be down in 10! T-Ho 2017 Hey team… bit of a spill but I’m fine.
SLIDE 5
5
We Expect Technology To Just Work™
SLIDE 6
Technical Issues Likely Cost Retailers Billions
12.01.16
Macy’s, Lowe’s hit by Black Friday technical glitches
11.27.17
Retail outages online leave shoppers frustrated on Black Friday
11.23.18
People.com
Black Friday Failures
@lklig
SLIDE 7
Wells Fargo accidentally foreclosed hundreds of homeowners
8.7.18
Customers report difficulty accessing Chase Bank mobile and online
2.16.19
Citibank Website down, not working
2.28.19
Investopedia
Breaking Banks
@lklig
SLIDE 8
Computer Problems Blamed For Flight Delays
4.1.19
Major US Airlines hit by delays after glitch at vendor
4.1.19
Pilots of doomed Boeing 737 MAX fought the plane’s software and lost
4.4.19
Airline Incidents
@lklig
SLIDE 9
9
Technology is fragile. When it breaks, we shouldn’t notice.
@lklig
SLIDE 10
10
Plan ahead to keep your users happy
FAILURE GRACEFUL DEGRADATION
@lklig
SLIDE 11
11
Why Are Failures So Common?
SLIDE 12
12
Legacy Systems
@lklig
SLIDE 13
13
Lack of Testing
Failure UI End to end Integration Unit
@lklig
SLIDE 14
14
With Scale Comes Complexity
@lklig
SLIDE 15
@lklig
SLIDE 16
@lklig
SLIDE 17
17
What Can We Do About It?
SLIDE 18
18
Design For Failure
SLIDE 19
19
@lklig
SLIDE 20
20
Designing For Failure
Key User Stories & Features Edge Cases From Unexpected User Behaviour Dependency Failures
@lklig
SLIDE 21
21 21
@lklig
SLIDE 22
22
@lklig
Loading Screens Are Not Graceful
SLIDE 23
23
Inject Failure By Breaking Things On Purpose
@lklig
SLIDE 24
Inject failure one service at a time. Maintain critical functionality.
24
@lklig
SLIDE 25
Common Failures Modes
25
Errors
HTTP 400, 401, 402 500, 503
Blackhole Latency
@lklig
THAT DEGRADE THE USER EXPERIENCE
@lklig
SLIDE 26
26
Degrade Gracefully
SLIDE 27
27
Graceful Degradation
- Provide the best possible experience
- All but the most critical
functionality can fall off
- Don’t give up on your users, hold
state as long as possible
@lklig
SLIDE 28
28
@lklig
When one dependency fails, users are
- ften affected
Storage Auth User Data Content Cache Feature 1 Feature 2
SLIDE 29
29
Implemented As Designed
@lklig
SLIDE 30
30
Added Latency
@lklig
SLIDE 31
31
Blocked Video Link
@lklig
SLIDE 32
32
Blocked JQuery Request
@lklig
SLIDE 33
33
@lklig
SLIDE 34
34
Delight Your Users
SLIDE 35
35 35
Graceful Degradation Done Right
@lklig
SLIDE 36
36
Positive Business Impact
Product Launch
Delight users with new features
Success Metrics
Quantitative goals of the launch
Product Landing
Were the goals achieved? Why or why not? What got in way?
@lklig
SLIDE 37
37
@lklig
Maintain release velocity Deliver a positive user experience Engineers spend less time in war rooms
Plan Experiments Early
@lklig
SLIDE 38
38 RELIABILITY THROUGH CHAOS ENGINEERING
Design for Failure
Identify the most critical end user functionality.
Inject Failure
Impact your system to be sure your user experience isn’t impacted.
Degrade Gracefully
Plan for non critical functionality not to get in the way.
Delight Your Users
Your product metrics will show behaviour, no matter the condition.
Graceful Degradation As a Feature
@lklig
SLIDE 39
USE LORNE FOR 20% OFF
SLIDE 40
40
gremlin.com/lorne
SLIDE 41
Q&A
Lorne Kligerman
Director of Product, Gremlin
@lklig