Presenters Phillipa Avery Robert Reta Senior Software Engineer - PowerPoint PPT Presentation

Presenters Phillipa Avery Robert Reta Senior Software Engineer Senior Software Engineer Project Technical Lead and Engineer Event Sourcing System Architect Downloads License Accounting Downloads License Accounting pavery@Netflix.com / @PhillipaAvery rreta@netflix.com / @rreta04

Phillipa Robert

Overview 1. Why do we need a Downloads License Accounting Service? 2. Event Sourcing overview. 3. Deep dive into the Event Sourcing Architecture. 4. What’s it like working with the License Accounting Service after release?

Streaming Playback Lifecycle CREATE PLAYBACK CONTEXT LICENSE SESSION EVENTS (START, PAUSE, RESUME, KEEPALIVE)

Streaming Playback Lifecycle RELEASE STOP LICENSE SESSION EVENT (STOP)

Downloads Playback Lifecycle CREATE PLAYBACK CONTEXT LICENSE DOWNLOADED ENCODED MOVIE

Downloads Playback Lifecycle SESSION EVENTS (START, PAUSE, RESUME, KEEPALIVE, STOP) RELEASE LICENSE RENEW LICENSE? X ~ 1 year

Download Business Requirements ● Devices with downloads

Download Business Requirements ● Devices with downloads ● Downloads per studio

Download Business Requirements ● Devices with downloads ● Downloads per studio ● Movie downloads or playback over year ~ 1 year

License Accounting Service Requirements ❏ Flexible y t i l i b F ❏ Debuggable a i l l e e R x i b i l ❏ Reliable i t y S ❏ Scalable c a l a b Debuggability i l i t y

License Accounting Service Requirements... ✓ Flexible: data model can be changed ❏ RDBMS ❏ Document Model

License Accounting Service Requirements... ✓ Flexible: data model can be changed ❏ RDBMS ❏ Document Model ✓ Debuggable ❏ Event Sourcing

License Accounting Service Requirements... ✓ Flexible: data model can be changed ❏ RDBMS ❏ Document Model ✓ Debuggable ❏ Event Sourcing ✓ Reliable ❏ Fallbacks

License Accounting Service Requirements... ✓ Flexible: data model can be changed ❏ RDBMS ❏ Document Model ✓ Debuggable ❏ Event Sourcing ✓ Reliable ❏ Fallbacks ✓ Scalable

Event Sourcing

Domain Model

Commands Make Purple Command

Commands Command Handler Make Purple Command

Commands Added Red Event Added Blue Command Event Handler Make Purple Command

Events Event Handler Added Red Event Added Blue Event

Events Event Handler Added Blue Event

Domain Model

Aggregate

Commands Command Handler Make Red Command

Commands Command Handler Make Red Command

Aggregate Aggregate REST Service Repository Query Query Query Event Store Endpoint

Event Store Row ID Events

Event Store Row ID Events

Event Store Events Row ID e 2 e 3 e 4 e 5 e 6 e 7 e 1 Aggregate id 1 Aggregate id 2

Aggregate Event Store Query Repository

Aggregate Event Store Query Repository select * from events where rowId =

e 2 e 3 e 4 e 5 e 6 e 7 e 1 Aggregate Event Store Repository Response

Aggregate Repository e 2 e 3 e 4 e 5 e 6 e 7 e 1 Aggregate id 1 Aggregate id 2 e 2 e 5 e 6 e 3 e 4 e 7 e 1

Aggregate Repository Aggregate id 1 e 2 e 5 e 6 e 1 Aggregate 1 Aggregate id 2 Aggregate 2 e 3 e 4 e 7

Aggregate Repository Aggregate id 1 e 2 e 5 e 6 Aggregate 1 e 1 Aggregate id 2 Aggregate 2 e 3 e 4 e 7

Aggregate Repository Aggregate id 1 e 5 e 6 Aggregate 1 e 2 Aggregate id 2 Aggregate 2 e 3 e 7

Aggregate Repository Aggregate id 1 e 6 Aggregate 1 e 5 Aggregate id 2 Aggregate 2 e 7

Aggregate Repository Aggregate id 1 Aggregate 1 e 6 Aggregate id 2 Aggregate 2

Aggregate Repository Aggregate 1 Aggregate 2

Aggregate Repository Aggregate 1 Command Handler Command

Aggregate Repository Command e 8 e 9 Handler

Aggregate Repository Aggregate 1 e 8 e 9

Aggregate Repository Aggregate 1 e 8 e 9

Aggregate Repository Aggregate 1 e 9

Aggregate Repository Aggregate 1

Aggregate Update Event Store Repository append events e 8 e 9 where rowId =

Event Store Events Row ID e 2 e 3 e 4 e 5 e 6 e 7 e 8 e 9 e 1 Aggregate id 1 Aggregate id 2

Get all aggregates for a customer Aggregate Aggregate Service Repository Query

Aggregate 1 Aggregate 2 Aggregate Aggregate Service Repository Response

Aggregate License Downloaded Aggregate Aggregate

customerId: Bayley titleId: GlowS01E01 Acquire License License Query Service Endpoint

isAllowed? customerId: Bayley titleId: GlowS01E01 License Downloaded Service Service Query

getAggregates Downloaded Downloaded customerId: Bayley Service Repository titleId: GlowS01E01 since: 6/27/2016 Query

Downloaded Downloaded Aggregate Aggregate customerId: Bayley customerId: Bayley titleId: GlowS01E01 titleId: GlowS01E01 date: 2/15/2017 date: 5/25/2017 Downloaded Downloaded Repository Service Response

Downloaded Service isAllowed? License Service Response aggregates.size() < yearly limit

Downloaded Service isAllowed? License True Service Response aggregates.size() < yearly limit (2) < (3)

Create Aggregate License Aggregate customerId: titleId: expires: released: License License Service Repository Create License Command customerId: Bayley titleId: GlowS01E01 date: 6/27/2017 Update

License Repository License Aggregate customerId: titleId: expires: released: Command Handler Create License Command customerId: Bayley titleId: GlowS01E01 date: 6/27/2017

License Repository License Aggregate customerId: titleId: expires: License Created released: Event Command customerId: Bayley Handler Create License titleId: GlowS01E01 Command date: 6/27/2017 customerId: Bayley titleId: GlowS01E01 date: 6/27/2017

License Repository License Aggregate customerId: titleId: expires: License Aggregate released: Event customerId: Bayley Handler License Created titleId: GlowS01E01 Event expires: 7/27/2017 released: false customerId: Bayley titleId: GlowS01E01 date: 6/27/2017

License Created Event customerId: Bayley titleId: GlowS01E01 date: 6/27/2017 License Save Repository Event Store

Create Aggregate Acquire License License Response License Created Service Repository Endpoint

isAllowed? License License Service Repository aggregates.filter(GlowS01E01) Get all licenses .size() < yearly limit Response

Table Partition Keys Clustering Columns Columns

Event Table rowId Aggregate Id Event Time Event Data Event Mapper text text timestamp blob String

● A fast and efficient object graph serialization framework for Java ○ https://github.com/EsotericSoftware/kryo ● Able to preconfigure the library to your data models for extra compaction ● Custom serializers

rowId AggregateId Event Time Event Data text text timestamp blob House of Cards 0 LicenseCreated Matt:LicenseAggregate Buddy 1 LicenseCreated Thunderstruck The Hardy Boys 2 LicenseCreated Jeff:LicenseAggregate

rowId AggregateId Event Time Event Data text text timestamp blob House of Cards 0 LicenseCreated Matt:LicenseAggregate House of Cards 3 LicenseReleased Buddy 1 LicenseCreated Thunderstruck The Hardy Boys 2 LicenseCreated Jeff:LicenseAggregate

rowId AggregateId Event Time Event Data text text timestamp blob House of Cards 0 LicenseCreated Matt:LicenseAggregate House of Cards 3 LicenseReleased Buddy 1 LicenseCreated Thunderstruck The Hardy Boys 2 LicenseCreated Jeff:LicenseAggregate

rowId AggregateId Event Time Event Data text text timestamp blob House of Cards 0 LicenseCreated Matt:LicenseAggregate House of Cards 3 LicenseReleased Buddy 1 LicenseCreated Thunderstruck The Hardy Boys 2 LicenseCreated Jeff:LicenseAggregate

Snapshotting Matt:LicenseAggregate House of Cards 0 LicenseCreated House of Cards 3 LicenseReleased Buddy Thunderstruck 1 LicenseCreated Master of None 0 LicenseCreated Master of None 3 LicenseReleased Bojack 1 LicenseCreated Lucha Underground 0 LicenseCreated Lucha Underground 3 LicenseReleased Grace & Frankie 1 LicenseCreated House of Cards 0 LicenseCreated

Snapshot Table rowId version Snapshot Data text int blob

rowId Version Event Data text int blob 1 binaryData Matt:LicenseAggregate

rowId AggregateId Event Time Event Data text text timestamp blob House of Cards 0 LicenseCreated Matt:LicenseAggregate:0 Jessica Jones 1 LicenseCreated House of Cards 3 Matt:LicenseAggregate:1 LicenseReleased The Hardy Boys 2 LicenseCreated Jeff:LicenseAggregate:0

Working With Event Sourcing

Flexibility in Practice ● Changes to the Data Model are trivial! ○ Device deactivation

New Device Deactivation Requirement Device Deactivate Device Device Aggregate Device Service Repository Endpoint Device Deactivated Event Deactivate Device Command