Cloud Native Go Building Scalable, Resilient Microservices for the - - PDF document
Cloud Native Go 6/28/17, 11)02 AM Cloud Native Go Building Scalable, Resilient Microservices for the Cloud in Go 1 / 29 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1 Page 1 of 38 Cloud Native Go 6/28/17, 11)02 AM Agenda
6/28/17, 11)02 AM Cloud Native Go Page 1 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
1 / 29
6/28/17, 11)02 AM Cloud Native Go Page 2 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
2 / 29
1 / 29
6/28/17, 11)02 AM Cloud Native Go Page 3 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Wrote a few books (2 fantasy, 15+ .NET/C#, Go) Maker, Tinkerer, Linguist Taught people how migrate/build cloud native for Pivotal Lead Software Engineer for Capital One Twitter: @KevinHoffman, github autodidaddict Go to gopherize.me gopherize.me for your Gopher Avatar 3 / 29
2 / 29
6/28/17, 11)02 AM Cloud Native Go Page 4 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Fast Low memory, CPU, and Disk footprint Docker images from SCRATCH Fit more containers per node/VM Single, no-dependency binary Beautiful, simple language Short learning curve 4 / 29
Wrote a few books (2 fantasy, 15+ .NET/C#, Go) Maker, Tinkerer, Linguist Taught people how migrate/build cloud native for Pivotal Lead Software Engineer for Capital One Twitter: @KevinHoffman, github autodidaddict Go to gopherize.me gopherize.me for your Gopher Avatar 3 / 29
6/28/17, 11)02 AM Cloud Native Go Page 5 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
API First Dependency Management Design, Build, Release, Run Config, Credentials, and Code Logs Disposability Backing Services Environment Parity Administrative Processes Port Binding Stateless Processes Concurrency Telemetry Authentication and Authorization 5 / 29
Fast Low memory, CPU, and Disk footprint Docker images from SCRATCH Fit more containers per node/VM Single, no-dependency binary Beautiful, simple language Short learning curve 4 / 29
6/28/17, 11)02 AM Cloud Native Go Page 6 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
HTTP Server Routing URL path variables Query strings JSON Encode/Decode Middleware (logging, security, etc) 6 / 29
API First Dependency Management Design, Build, Release, Run Config, Credentials, and Code Logs Disposability Backing Services Environment Parity Administrative Processes Port Binding Stateless Processes Concurrency Telemetry Authentication and Authorization 5 / 29
6/28/17, 11)02 AM Cloud Native Go Page 7 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
package main import ( ... ) func main() { port := os.Getenv("PORT") if len(port) == 0 { port = "8080" } mux := http.NewServeMux() mux.HandleFunc("/", hello) n := negroni.Classic() n.UseHandler(mux) hostString := fmt.Sprintf(":%s", port) n.Run(hostString) } func hello(res http.ResponseWriter, req *http.Request) { fmt.Fprintln(res, "Hello from Go!") }
7 / 29
HTTP Server Routing URL path variables Query strings JSON Encode/Decode Middleware (logging, security, etc) 6 / 29
6/28/17, 11)02 AM Cloud Native Go Page 8 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Routing with Gorilla Mux
r := mux.NewRouter() r.HandleFunc("/products/{key}", ProductHandler) r.HandleFunc("/articles/{category}/", ArticlesCategoryHandler) r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler) s := r.PathPrefix("/zombies").Subrouter() s.HandleFunc("/", ZombiesHandler) s.HandleFunc("/sightings", NewSightingHandler).Methods("POST") s.HandleFunc("/{key}", QueryZombieHandler).Methods("GET")
URLs: /zombies /zombies/sightings (POST) /zombies/bob (GET) 8 / 29
package main import ( ... ) func main() { port := os.Getenv("PORT") if len(port) == 0 { port = "8080" } mux := http.NewServeMux() mux.HandleFunc("/", hello) n := negroni.Classic() n.UseHandler(mux) hostString := fmt.Sprintf(":%s", port) n.Run(hostString) } func hello(res http.ResponseWriter, req *http.Request) { fmt.Fprintln(res, "Hello from Go!") }
7 / 29
6/28/17, 11)02 AM Cloud Native Go Page 9 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Accessing Route Data
vars := mux.Vars(req) zombieID := vars["zombie-key"]
Building URLs
r := mux.NewRouter() r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler). Name("article") ... url, err := r.Get("article").URL("category", "technology", "id", "42")
Builds - /articles/technology/42 9 / 29
Routing with Gorilla Mux
r := mux.NewRouter() r.HandleFunc("/products/{key}", ProductHandler) r.HandleFunc("/articles/{category}/", ArticlesCategoryHandler) r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler) s := r.PathPrefix("/zombies").Subrouter() s.HandleFunc("/", ZombiesHandler) s.HandleFunc("/sightings", NewSightingHandler).Methods("POST") s.HandleFunc("/{key}", QueryZombieHandler).Methods("GET")
URLs: /zombies /zombies/sightings (POST) /zombies/bob (GET) 8 / 29
6/28/17, 11)02 AM Cloud Native Go Page 10 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Read JSON from Body:
payload, _ := ioutil.ReadAll(req.Body) var newMatchRequest newMatchRequest err := json.Unmarshal(payload, &newMatchRequest)
Write JSON to Response:
var mr newMatchResponse mr.copyMatch(newMatch) w.Header().Add("Location", "/matches/"+newMatch.ID) formatter.JSON(w, http.StatusCreated, &mr)
10 / 29
Accessing Route Data
vars := mux.Vars(req) zombieID := vars["zombie-key"]
Building URLs
r := mux.NewRouter() r.HandleFunc("/articles/{category}/{id:[0-9]+}", ArticleHandler). Name("article") ... url, err := r.Get("article").URL("category", "technology", "id", "42")
Builds - /articles/technology/42 9 / 29
6/28/17, 11)02 AM Cloud Native Go Page 11 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
apiRouter := mux.NewRouter() apiRouter.HandleFunc("/api/post", apiPostHandler(formatter)).Methods("POST") router.PathPrefix("/api").Handler(negroni.New( negroni.HandlerFunc(isAuthorized(formatter)), negroni.Wrap(apiRouter), )) func isAuthorized(formatter *render.Render) negroni.HandlerFunc { return func(w http.ResponseWriter, r *http.Request, next http.HandlerFunc) { providedKey := r.Header.Get(APIKey) if providedKey == "" { formatter.JSON(w, http.StatusUnauthorized, struct{ Error string }{"Unauthorized." } else if providedKey != apikey { formatter.JSON(w, http.StatusForbidden, struct{ Error string }{"Insufficient credentials." } else { next(w, r) } } }
11 / 29
Read JSON from Body:
payload, _ := ioutil.ReadAll(req.Body) var newMatchRequest newMatchRequest err := json.Unmarshal(payload, &newMatchRequest)
Write JSON to Response:
var mr newMatchResponse mr.copyMatch(newMatch) w.Header().Add("Location", "/matches/"+newMatch.ID) formatter.JSON(w, http.StatusCreated, &mr)
10 / 29
6/28/17, 11)02 AM Cloud Native Go Page 12 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
✅ HTTP Server ✅ Routing ✅ URL path variables ✅ Query strings ✅ JSON Encode/Decode ✅ Middleware (logging, security, etc) 12 / 29
apiRouter := mux.NewRouter() apiRouter.HandleFunc("/api/post", apiPostHandler(formatter)).Methods("POST") router.PathPrefix("/api").Handler(negroni.New( negroni.HandlerFunc(isAuthorized(formatter)), negroni.Wrap(apiRouter), )) func isAuthorized(formatter *render.Render) negroni.HandlerFunc { return func(w http.ResponseWriter, r *http.Request, next http.HandlerFunc) { providedKey := r.Header.Get(APIKey) if providedKey == "" { formatter.JSON(w, http.StatusUnauthorized, struct{ Error string }{"Unauthorized." } else if providedKey != apikey { formatter.JSON(w, http.StatusForbidden, struct{ Error string }{"Insufficient credentials." } else { next(w, r) } } }
11 / 29
6/28/17, 11)02 AM Cloud Native Go Page 13 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
13 / 29
✅ HTTP Server ✅ Routing ✅ URL path variables ✅ Query strings ✅ JSON Encode/Decode ✅ Middleware (logging, security, etc) 12 / 29
6/28/17, 11)02 AM Cloud Native Go Page 14 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
A fox knows many things, but a hedgehog one important thing - Archilochus 14 / 29
13 / 29
6/28/17, 11)02 AM Cloud Native Go Page 15 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Ports and Adapters 15 / 29
A fox knows many things, but a hedgehog one important thing - Archilochus 14 / 29
6/28/17, 11)02 AM Cloud Native Go Page 16 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Build your Core Functionality First Decouple HTTP, JSON, Headers, Security from Core Function Should be able to test your core function in isolation Code is blissfully ignorant of source and nature of inputs and ultimate destination of replies 16 / 29
Ports and Adapters 15 / 29
6/28/17, 11)02 AM Cloud Native Go Page 17 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
17 / 29
Build your Core Functionality First Decouple HTTP, JSON, Headers, Security from Core Function Should be able to test your core function in isolation Code is blissfully ignorant of source and nature of inputs and ultimate destination of replies 16 / 29
6/28/17, 11)02 AM Cloud Native Go Page 18 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
API Gateways 17 / 29
17 / 29
6/28/17, 11)02 AM Cloud Native Go Page 19 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
API Gateways Aggregator Services 17 / 29
API Gateways 17 / 29
6/28/17, 11)02 AM Cloud Native Go Page 20 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
API Gateways Aggregator Services Message Brokers Topics Pub/Sub 17 / 29
API Gateways Aggregator Services 17 / 29
6/28/17, 11)02 AM Cloud Native Go Page 21 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
API Gateways Aggregator Services Message Brokers Topics Pub/Sub ACL / Translators 17 / 29
API Gateways Aggregator Services Message Brokers Topics Pub/Sub 17 / 29
6/28/17, 11)02 AM Cloud Native Go Page 22 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
API Gateways Aggregator Services Message Brokers Topics Pub/Sub ACL / Translators Dynamic Service Discovery 17 / 29
API Gateways Aggregator Services Message Brokers Topics Pub/Sub ACL / Translators 17 / 29
6/28/17, 11)02 AM Cloud Native Go Page 23 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
API Gateways Aggregator Services Message Brokers Topics Pub/Sub ACL / Translators Dynamic Service Discovery External Configuration 17 / 29
API Gateways Aggregator Services Message Brokers Topics Pub/Sub ACL / Translators Dynamic Service Discovery 17 / 29
6/28/17, 11)02 AM Cloud Native Go Page 24 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
API Gateways Aggregator Services Message Brokers Topics Pub/Sub ACL / Translators Dynamic Service Discovery External Configuration Adapter paths through system Gateways, Proxies Translators on Data Streams / Queues 17 / 29
API Gateways Aggregator Services Message Brokers Topics Pub/Sub ACL / Translators Dynamic Service Discovery External Configuration 17 / 29
6/28/17, 11)02 AM Cloud Native Go Page 25 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
API Gateways Aggregator Services Message Brokers Topics Pub/Sub ACL / Translators Dynamic Service Discovery External Configuration Adapter paths through system Gateways, Proxies Translators on Data Streams / Queues Hexagonal arch inside and outside of services 17 / 29
API Gateways Aggregator Services Message Brokers Topics Pub/Sub ACL / Translators Dynamic Service Discovery External Configuration Adapter paths through system Gateways, Proxies Translators on Data Streams / Queues 17 / 29
6/28/17, 11)02 AM Cloud Native Go Page 26 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Security Monitoring Log Aggregation Zero Downtime Deployment Build/Deploy Automation Tracing Metrics Audit Automatic / Dynamic Horizontal Scaling Configuration Secrets Management Discovery 18 / 29
API Gateways Aggregator Services Message Brokers Topics Pub/Sub ACL / Translators Dynamic Service Discovery External Configuration Adapter paths through system Gateways, Proxies Translators on Data Streams / Queues Hexagonal arch inside and outside of services 17 / 29
6/28/17, 11)02 AM Cloud Native Go Page 27 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Define what we're building Define why we're building it Who are we building it for? Build the core Add onion layers around the core Ports and Adapters Remember we're building an ecosystem, not hello world 19 / 29
Security Monitoring Log Aggregation Zero Downtime Deployment Build/Deploy Automation Tracing Metrics Audit Automatic / Dynamic Horizontal Scaling Configuration Secrets Management Discovery 18 / 29
6/28/17, 11)02 AM Cloud Native Go Page 28 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
20 / 29
Define what we're building Define why we're building it Who are we building it for? Build the core Add onion layers around the core Ports and Adapters Remember we're building an ecosystem, not hello world 19 / 29
6/28/17, 11)02 AM Cloud Native Go Page 29 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
go micro Framework https://micro.mu/ Ports and Adapters are services Not boilerplate inside service Ecosystem-aware from bottom-up Dynamic Service Discovery gRPC Transport RESTful Aggregate API ... and much more 21 / 29
20 / 29
6/28/17, 11)02 AM Cloud Native Go Page 30 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Opinionated vs boilerplate Spend boilerplate to insulate from opinions Mitigate ceremony with code generation? Go-kit - suite of libraries, incur BP cost Go-micro - opinionated, less BP Spring Boot - example of opinionated framework 22 / 29
go micro Framework https://micro.mu/ Ports and Adapters are services Not boilerplate inside service Ecosystem-aware from bottom-up Dynamic Service Discovery gRPC Transport RESTful Aggregate API ... and much more 21 / 29
6/28/17, 11)02 AM Cloud Native Go Page 31 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
API First is NOT: RESTful Endpoint First Swagger-First One API to Rule them All API First is: A strict, semantically versioned, interaction specification. A team and organization-wide discipline Different consumers, channels may need different APIs Ports and Adapters 23 / 29
Opinionated vs boilerplate Spend boilerplate to insulate from opinions Mitigate ceremony with code generation? Go-kit - suite of libraries, incur BP cost Go-micro - opinionated, less BP Spring Boot - example of opinionated framework 22 / 29
6/28/17, 11)02 AM Cloud Native Go Page 32 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Protocol Buffer IDL gRPC Service
syntax = "proto3"; package warehouse; service Warehouse { rpc GetWarehouseDetails(DetailsRequest) returns (DetailsResponse); } message DetailsRequest { string sku = 1; } message DetailsResponse { WarehouseDetails details = 1; } message WarehouseDetails { string sku = 1; uint32 stock_remaining = 2; string manufacturer = 3; string model_number = 4; }
24 / 29
API First is NOT: RESTful Endpoint First Swagger-First One API to Rule them All API First is: A strict, semantically versioned, interaction specification. A team and organization-wide discipline Different consumers, channels may need different APIs Ports and Adapters 23 / 29
6/28/17, 11)02 AM Cloud Native Go Page 33 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
package service import ( "github.com/autodidaddict/go-shopping/warehouse/proto" "golang.org/x/net/context" ) type warehouseService struct{} // NewWarehouseService returns an instance of a warehouse handler func NewWarehouseService() warehouse.WarehouseHandler { return &warehouseService{} } func (w *warehouseService) GetWarehouseDetails(ctx context.Context, request *warehouse.DetailsRequest, response *warehouse.DetailsResponse) error { response.Manufacturer = "TOSHIBA" response.ModelNumber = "T-1000" response.SKU = request.SKU response.StockRemaining = 35 return nil }
25 / 29
Protocol Buffer IDL gRPC Service
syntax = "proto3"; package warehouse; service Warehouse { rpc GetWarehouseDetails(DetailsRequest) returns (DetailsResponse); } message DetailsRequest { string sku = 1; } message DetailsResponse { WarehouseDetails details = 1; } message WarehouseDetails { string sku = 1; uint32 stock_remaining = 2; string manufacturer = 3; string model_number = 4; }
24 / 29
6/28/17, 11)02 AM Cloud Native Go Page 34 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
What could possibly go wrong? 26 / 29
package service import ( "github.com/autodidaddict/go-shopping/warehouse/proto" "golang.org/x/net/context" ) type warehouseService struct{} // NewWarehouseService returns an instance of a warehouse handler func NewWarehouseService() warehouse.WarehouseHandler { return &warehouseService{} } func (w *warehouseService) GetWarehouseDetails(ctx context.Context, request *warehouse.DetailsRequest, response *warehouse.DetailsResponse) error { response.Manufacturer = "TOSHIBA" response.ModelNumber = "T-1000" response.SKU = request.SKU response.StockRemaining = 35 return nil }
25 / 29
6/28/17, 11)02 AM Cloud Native Go Page 35 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Question Everything Beware of Protobuf Default Values Service-first development Code is the easiest and simplest part of Microservices NFRs are everything - logging, metrics, alerts, discovery, health, autoscale, async messaging, etc. Decide where you want to be on the opinion vs boilerplate curve Go is an ideal microservices development language 27 / 29
What could possibly go wrong? 26 / 29
6/28/17, 11)02 AM Cloud Native Go Page 36 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
Create Services Test everything Automate everything Deploy all the time Try out go-micro Try out go-kit Go Shopping http://github.com/autodidaddict/go-shopping These slides https://github.com/autodidaddict/cng-presentation 28 / 29
Question Everything Beware of Protobuf Default Values Service-first development Code is the easiest and simplest part of Microservices NFRs are everything - logging, metrics, alerts, discovery, health, autoscale, async messaging, etc. Decide where you want to be on the opinion vs boilerplate curve Go is an ideal microservices development language 27 / 29
6/28/17, 11)02 AM Cloud Native Go Page 37 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
29 / 29
Create Services Test everything Automate everything Deploy all the time Try out go-micro Try out go-kit Go Shopping http://github.com/autodidaddict/go-shopping These slides https://github.com/autodidaddict/cng-presentation 28 / 29
6/28/17, 11)02 AM Cloud Native Go Page 38 of 38 file:///Users/kimberlyamaral/Desktop/cng-presentation/pres.html#1
29 / 29