Where is my cache? Architectural patterns for caching microservices - - PowerPoint PPT Presentation

▶
where is my cache architectural patterns for caching
SMART_READER_LITE
LIVE PREVIEW

Where is my cache? Architectural patterns for caching microservices - - PowerPoint PPT Presentation

Jan 05, 2023 145 likes •1.09k views

Where is my cache? Architectural patterns for caching microservices by example Rafa Leszko Cloud Software Engineer Hazelcast About me Cloud Software Engineer at Hazelcast Worked at Google and CERN Author of the book

slide-1
SLIDE 1

Where is my cache? Architectural patterns for caching microservices by example

Rafał Leszko Cloud Software Engineer Hazelcast

slide-2
SLIDE 2

About me

  • Cloud Software Engineer at Hazelcast
  • Worked at Google and CERN
  • Author of the book "Continuous Delivery with

Docker and Jenkins"

  • Trainer and conference speaker
  • Live in Kraków, Poland
slide-3
SLIDE 3

About Hazelcast

  • Distributed Company
  • Open Source Software
  • 140+ Employees
  • Hiring (Remote)!
  • Recently Raised $21M
  • Products:

○ Hazelcast IMDG ○ Hazelcast Jet ○ Hazelcast Cloud

@Hazelcast www.hazelcast.com

slide-4
SLIDE 4
  • Introduction
  • Caching Architectural Patterns

○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar

  • Summary

Agenda

slide-5
SLIDE 5

Why Caching?

  • Performance

○ Decrease latency ○ Reduce load

  • Resilience

○ High availability ○ Lower downtime

slide-6
SLIDE 6

Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby

Microservice World

slide-7
SLIDE 7

Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby

Microservice World

cache cache cache cache

slide-8
SLIDE 8

Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby

Microservice World

cache cache cache

slide-9
SLIDE 9

Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby

Microservice World

cache cache cache

slide-10
SLIDE 10
  • Introduction
  • Caching Architectural Patterns

○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar

  • Summary

Agenda

slide-11
SLIDE 11
  • 1. Embedded
slide-12
SLIDE 12

Application Load Balancer Cache Application Cache Request

Embedded Cache

slide-13
SLIDE 13

private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; }

Embedded Cache (Pure Java implementation)

slide-14
SLIDE 14

private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; }

Embedded Cache (Pure Java implementation)

slide-15
SLIDE 15
  • No Eviction Policies
  • No Max Size Limit

(OutOfMemoryError)

  • No Statistics
  • No built-in Cache Loaders
  • No Expiration Time
  • No Notification Mechanism

Java Collection is not a Cache!

slide-16
SLIDE 16

CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration.ofMinutes(10)) .maximumSize(1000) .build();

Embedded Cache (Java libraries)

slide-17
SLIDE 17

Embedded Cache (Java libraries)

CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration.ofMinutes(10)) .maximumSize(1000) .build();

slide-18
SLIDE 18

Caching Application Layer

@Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } }

slide-19
SLIDE 19

Caching Application Layer

@Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } }

Be Careful, Spring uses ConcurrentHashMap by default!

slide-20
SLIDE 20

Application Load Balancer Cache Application Cache Request

Embedded Cache

slide-21
SLIDE 21

1*. Embedded Distributed

slide-22
SLIDE 22

Application Application Load Balancer Cache Cache Request Hazelcast Cluster

Embedded Distributed Cache

slide-23
SLIDE 23

@Configuration public class HazelcastConfiguration { @Bean CacheManager cacheManager() { return new HazelcastCacheManager( Hazelcast.newHazelcastInstance()); } }

Embedded Distributed Cache (Spring with Hazelcast)

slide-24
SLIDE 24

DEMO

slide-25
SLIDE 25

Hazelcast Discovery Plugins

slide-26
SLIDE 26

Hazelcast Discovery Plugins

slide-27
SLIDE 27

Hazelcast Discovery Plugins

slide-28
SLIDE 28

Application Application Load Balancer Cache Cache Request

Embedded Distributed Cache

Hazelcast Cluster

slide-29
SLIDE 29

Embedded Cache Pros Cons

  • Simple configuration /

deployment

  • Low-latency data access
  • No separate Ops Team needed
  • Not flexible management (scaling,

backup)

  • Limited to JVM-based

applications

  • Data collocated with applications
slide-30
SLIDE 30
  • Introduction
  • Caching Architectural Patterns

○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar

  • Summary

Agenda

slide-31
SLIDE 31
  • 2. Client-Server
slide-32
SLIDE 32

Application Load Balancer Application Request Cache Server

Client-Server Cache

slide-33
SLIDE 33

Application Load Balancer Application Request Cache Server

Client-Server Cache

slide-34
SLIDE 34

Application Load Balancer Application Request Cache Server

Client-Server Cache

Separate Management:

  • backups
  • (auto) scaling
  • security

Ops Team

slide-35
SLIDE 35

Application Load Balancer Application Request Cache Server

Client-Server Cache

slide-36
SLIDE 36

Application Load Balancer Application Request Cache Server

Client-Server Cache

slide-37
SLIDE 37

Client-Server Cache

slide-38
SLIDE 38

Client-Server Cache

slide-39
SLIDE 39

Client-Server Cache

$ ./start.sh

Starting Hazelcast Cache Server (standalone)

slide-40
SLIDE 40

Client-Server Cache

Starting Hazelcast Cache Server (Kubernetes)

$ helm install hazelcast/hazelcast

slide-41
SLIDE 41

Client-Server Cache

Hazelcast Client (Kubernetes):

@Configuration public class HazelcastClientConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getKubernetesConfig() .setEnabled(true); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } }

Starting Hazelcast Cache Server (Kubernetes)

$ helm install hazelcast/hazelcast

slide-42
SLIDE 42

Application Load Balancer Application Request Cache Server

Client-Server Cache

Ops Team Separate Management:

  • backups
  • (auto) scaling
  • security
slide-43
SLIDE 43

2*. Cloud

slide-44
SLIDE 44

Application Load Balancer Application Request

Cloud (Cache as a Service)

slide-45
SLIDE 45

Application Load Balancer Application Request

Cloud (Cache as a Service)

Management:

  • backups
  • (auto) scaling
  • security

Ops Team

slide-46
SLIDE 46

Application Load Balancer Application Request

Cloud (Cache as a Service)

Management:

  • backups
  • (auto) scaling
  • security

Ops Team

slide-47
SLIDE 47

Application Load Balancer Application Request

Cloud (Cache as a Service)

slide-48
SLIDE 48

@Configuration public class HazelcastCloudConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getCloudConfig() .setEnabled(true) .setDiscoveryToken("KSXFDTi5HXPJGR0wRAjLgKe45tvEEhd"); clientConfig.setGroupConfig( new GroupConfig("test-cluster", "b2f984b5dd3314")); return new HazelcastCacheManager( HazelcastClient.newHazelcastClient(clientConfig)); } }

Cloud (Cache as a Service)

slide-49
SLIDE 49

DEMO

cloud.hazelcast.com

slide-50
SLIDE 50

Client-Server (Cloud) Cache Pros

  • Data separate from applications
  • Separate management (scaling,

backup)

  • Programming-language agnostic

Cons

  • Separate Ops effort
  • Higher latency
  • Server network requires

adjustment (same region, same VPC)

slide-51
SLIDE 51
  • Introduction
  • Caching Architectural Patterns

○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar

  • Summary

Agenda

slide-52
SLIDE 52
  • 3. Sidecar
slide-53
SLIDE 53

Kubernetes Service (Load Balancer) Request Kubernetes POD Application Container Cache Container Application Container Cache Container Kubernetes POD

Sidecar Cache

Hazelcast Cluster

slide-54
SLIDE 54

Sidecar Cache

Similar to Embedded:

  • the same physical machine
  • the same resource pool
  • scales up and down together
  • no discovery needed (always localhost)

Similar to Client-Server:

  • different programming language
  • uses cache client to connect
  • clear isolation between app and cache
slide-55
SLIDE 55

@Configuration public class HazelcastSidecarConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig() .addAddress("localhost:5701"); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } }

Sidecar Cache

slide-56
SLIDE 56

Sidecar Cache

apiVersion: apps/v1 kind: Deployment ... spec: template: spec: containers:

  • name: application

image: leszko/application

  • name: hazelcast

image: hazelcast/hazelcast

slide-57
SLIDE 57

Sidecar Cache Pros Cons

  • Simple configuration
  • Programming-language agnostic
  • Low latency
  • Some isolation of data and

applications

  • Limited to container-based

environments

  • Not flexible management (scaling,

backup)

  • Data collocated with application

PODs

slide-58
SLIDE 58
  • Introduction
  • Caching Architectural Patterns

○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar

  • Summary

Agenda

slide-59
SLIDE 59
  • 4. Reverse Proxy
slide-60
SLIDE 60

Application Load Balancer Cache Application Request

Reverse Proxy Cache

slide-61
SLIDE 61

Reverse Proxy Cache

http { ... proxy_cache_path /data/nginx/cache keys_zone=one:10m; ... }

slide-62
SLIDE 62
  • Only for HTTP
  • Not distributed
  • No High Availability
  • Data stored on the disk

NGINX Reverse Proxy Cache Issues

slide-63
SLIDE 63

NGINX Reverse Proxy Cache Issues

  • Only for HTTP
  • Not distributed
  • No High Availability
  • Data stored on the disk
slide-64
SLIDE 64

4*. Reverse Proxy Sidecar

slide-65
SLIDE 65

Kubernetes Service (Load Balancer) Request Kubernetes POD Application Container Reverse Proxy Cache Container Application Container Reverse Proxy Cache Container Kubernetes POD

Reverse Proxy Sidecar Cache

Hazelcast Cluster

slide-66
SLIDE 66
slide-67
SLIDE 67

Good

slide-68
SLIDE 68

Reverse Proxy Sidecar Cache

Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby

slide-69
SLIDE 69

Reverse Proxy Sidecar Cache

Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby

slide-70
SLIDE 70

Reverse Proxy Sidecar Cache

apiVersion: apps/v1 kind: Deployment ... spec: template: spec: initContainers:

  • name: init-networking

image: leszko/init-networking containers:

  • name: caching-proxy

image: leszko/caching-proxy

  • name: application

image: leszko/application

slide-71
SLIDE 71

Reverse Proxy Sidecar Cache

Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby

slide-72
SLIDE 72
slide-73
SLIDE 73

Reverse Proxy Sidecar Cache (Istio)

slide-74
SLIDE 74

Bad

slide-75
SLIDE 75
slide-76
SLIDE 76

Reverse Proxy Cache

@CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {}

Application Cache:

slide-77
SLIDE 77

Reverse Proxy Cache

@CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {}

Application Cache: Proxy Cache:

http { ... location / { add_header Cache-Control public; expires 86400; etag on; } }

slide-78
SLIDE 78

Reverse Proxy (Sidecar) Cache Pros Cons

  • Configuration-based (no need to

change applications)

  • Programming-language agnostic
  • Consistent with containers and

microservice world

  • Difficult cache invalidation
  • No mature solutions yet
  • Protocol-based (e.g. works only

with HTTP)

slide-79
SLIDE 79
  • Introduction
  • Caching Architectural Patterns

○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar

  • Summary

Agenda

slide-80
SLIDE 80

Summary

slide-81
SLIDE 81

application-aware?

slide-82
SLIDE 82

application-aware? containers? no

slide-83
SLIDE 83

application-aware? containers? Reverse Proxy no no

slide-84
SLIDE 84

application-aware? containers? Reverse Proxy Reverse Proxy Sidecar no yes no

slide-85
SLIDE 85

application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? yes no yes no

slide-86
SLIDE 86

application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? yes no yes no no

slide-87
SLIDE 87

application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) yes no yes no no no

slide-88
SLIDE 88

application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar yes no yes yes no no no

slide-89
SLIDE 89

application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar cloud? yes no yes yes yes no no no

slide-90
SLIDE 90

application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar cloud? Client-Server yes no yes yes yes no no no no

slide-91
SLIDE 91

application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar cloud? Client-Server Cloud yes no yes yes yes yes no no no no

slide-92
SLIDE 92

Resources

  • Hazelcast Sidecar Container Pattern: https://hazelcast.com/blog/hazelcast-

sidecar-container-pattern/

  • Hazelcast Reverse Proxy Sidecar Caching Prototype:

https://github.com/leszko/caching-injector

  • Caching Best Practices: https://vladmihalcea.com/caching-best-practices/
  • NGINX HTTP Reverse Proxy Caching:

https://www.nginx.com/resources/videos/best-practices-for-caching/

slide-93
SLIDE 93

Thank You!

Rafał Leszko @RafalLeszko