Microservice Layout in Netflix Polyglot Persistence Powering - - PowerPoint PPT Presentation

▶
microservice layout in netflix polyglot persistence
SMART_READER_LITE
LIVE PREVIEW

Microservice Layout in Netflix Polyglot Persistence Powering - - PowerPoint PPT Presentation

Apr 03, 2024 191 likes •952 views

Microservice Layout in Netflix Polyglot Persistence Powering Microservices Roopa Tangirala Engineering Manager Netflix Agenda 5 Use Cases Challenges Current Approach Takeaway AWS S3 CDE Search, Analyze and visualize in

slide-1
SLIDE 1
slide-2
SLIDE 2
slide-3
SLIDE 3

Microservice Layout in Netflix

slide-4
SLIDE 4
slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7
slide-8
SLIDE 8

Polyglot Persistence Powering Microservices

Roopa Tangirala Engineering Manager Netflix

slide-9
SLIDE 9

Agenda

  • 5 Use Cases
  • Challenges
  • Current Approach
  • Takeaway
slide-10
SLIDE 10
slide-11
SLIDE 11
slide-12
SLIDE 12
slide-13
SLIDE 13

AWS S3

CDE

slide-14
SLIDE 14

Search, Analyze and visualize in near real time Distributed in-memory caching solution based on memcached Distributed NOSQL database to handle large datasets providing high availability. Distributed dynamo layer for different storage engines and protocols supporting Redis, memcached, RocksDB TitanDB is scalable graph database optimized for storing and querying graph datasets.

slide-15
SLIDE 15

Us-west-2 Oregon Us-east-1 North Virginia Eu-west-1 Ireland

slide-16
SLIDE 16

CDN URL

slide-17
SLIDE 17
slide-18
SLIDE 18

Requirements - CDN URL

  • High availability
  • Very low latency reads/writes (less than 1ms)
  • High Throughput per node
slide-19
SLIDE 19

GUESS?

slide-20
SLIDE 20

Distributed In Memory Very low Latency responses

slide-21
SLIDE 21

Playback Error

slide-22
SLIDE 22

PLAYBACK CONTEXT (Tracks + Track Urls)

NETFLIX OPENCONNECT CDN URLS

slide-23
SLIDE 23

PLAYBACK CONTEXT

COUNTRY USER PREFERENCES TITLE METADATA DEVICE NETWORK

slide-24
SLIDE 24

Requirements - Playback Error

  • Quick Incident Resolution
  • Interactive Dashboards
  • Near realtime Search
  • Ad Hoc Queries
slide-25
SLIDE 25

GUESS?

slide-26
SLIDE 26

Powerful Search & Analytics Interactive Dashboards

slide-27
SLIDE 27

Interactive Exploration

slide-28
SLIDE 28

Top N queries

slide-29
SLIDE 29

Incident To Resolution Time

2+ Hours Under 10 Minutes

slide-30
SLIDE 30

Viewing History

slide-31
SLIDE 31
slide-32
SLIDE 32

Requirements - Viewing History

  • Time series dataset
  • Support high writes
  • Cross region replication
  • Large dataset
slide-33
SLIDE 33

Growth of Viewing History

33

slide-34
SLIDE 34

GUESS?

slide-35
SLIDE 35

Multi-datacenter, multi-directional replication Highly availability and scalability

slide-36
SLIDE 36

Data Model

36

slide-37
SLIDE 37

New Data Model

37

slide-38
SLIDE 38

Artwork Display Set B Display Set A Character Movie Video Track PNG Webp Audio Track JPG Video Seg Es Sub Fr Dub Montage TEXT Track Trailer Fr Sub Person

Digital Asset Management

slide-39
SLIDE 39

Requirements - DAM

  • One backend plane for all asset metadata
  • Storage of relationships/connected data
  • Searchable
slide-40
SLIDE 40

GUESS?

slide-41
SLIDE 41

Distributed GraphDB Support for various storage backends

slide-42
SLIDE 42

Distributed Delayed Queues

slide-43
SLIDE 43

Requirements - Delayed Queues

  • Distributed
  • Highly concurrent
  • At-least-once delivery semantics
  • Delayed queue
  • Priorities within the shard
slide-44
SLIDE 44

GUESS?

slide-45
SLIDE 45

Pluggable datastore supporting Redis Multi-datacenter replication

slide-46
SLIDE 46

Data Model

For each queue three set of Redis data structures are maintained:

  • 1. A Sorted Set containing queued elements by score.
  • 2. A Hash set that contains message payload, with key as

message ID.

  • 3. A Sorted Set containing messages consumed by client but

yet to be acknowledged. Un-ack set.

slide-47
SLIDE 47
slide-48
SLIDE 48
slide-49
SLIDE 49

Prediction @ CDE

slide-50
SLIDE 50
slide-51
SLIDE 51
slide-52
SLIDE 52

Current Approach

slide-53
SLIDE 53

Subject Matter Expert

slide-54
SLIDE 54
slide-55
SLIDE 55

CDE Service

Empowering CDE to provide datastores as a service

slide-56
SLIDE 56

CDE Service

  • Thresholds/SLAs
  • Cluster metadata
  • Self Service
  • Contact information
  • Maintenance windows
slide-57
SLIDE 57

Architecture

PAGE CDE

Alert if needed

S C H E D U L E R R E M E D I A T I O N CDE SERVICE Alert Atlas Mantis Datastore Cluster Metadata Cluster Metadata/ Advisor Maintenance Remediation MONITORING

slide-58
SLIDE 58

SLA

cass_xyz abc@netflix.com cde

abc@netflix.com

slide-59
SLIDE 59

10

6 4

cas_xyz

cass_abc cass_test abc@netf lix.com abc@netf lix.com xyz@netfl ix.com xyz@netfl ix.com 343

34

4 34 2 3 34 5 6 65 5 34 65 56 55 56 2

slide-60
SLIDE 60
slide-61
SLIDE 61
slide-62
SLIDE 62

Machine learning

Automic (UC4)
slide-63
SLIDE 63

Pattern in Disk usage

slide-64
SLIDE 64
slide-65
SLIDE 65

Cde Channel

slide-66
SLIDE 66
slide-67
SLIDE 67
slide-68
SLIDE 68
slide-69
SLIDE 69
slide-70
SLIDE 70

Common Approach

CRON System Job Runner Job Runner Job Runner Job Runner

slide-71
SLIDE 71

Streaming micro-services

Source Stage Sink Source Stage Sink Source Stage Sink Source - input, handles backpressure Stage - business logic Sink - output, handles backpressure Source Stage Sink Mantis Job

slide-72
SLIDE 72

Real Time Dash (Macro View)

cluster_3 cluster_6

slide-73
SLIDE 73

Real Time Dash (Cluster View)

1

slide-74
SLIDE 74

Takeaway

slide-75
SLIDE 75