@olliwegner @stilkov
Oliver Wegner • Stefan Tilkov From Parts to a Whole: Modular Development of a Large-Scale e-Commerce Site QCon, London 06/03/2014
1. Reviewing architectures
Generic Architecture Review Results Deployment is Building way too Technical debt is features takes complicated well-known and too long and slow not addressed Architectural quality has degraded Scalability has reached its limit “-ility” problems Replacement would abound be way too expensive
Any architecture’s quality is directly proportional to the number of bottlenecks limiting its evolution, development, and operations http://www.flickr.com/photos/krissymayhew/5463349254, Krissy Mayh
Conway’s Law Organization → Architecture “Organizations which design systems are constrained to produce systems which are copies of the communication structures of these organizations.” – M.E. Conway
Reversal 1 Architecture → Organization Any particular architecture approach constraints organizational options – i.e. makes some organizational models simple and others hard to implement.
Reversal 2 Architecture → Organization Choosing a particular architecture can be a means of optimizing for a desired organizational structure.
2. Rebuilding Otto.de
For the last 15 years the E-Commerce business has become more important 4.200 Percentage of turnover Employees > 2.1 Billion € turnover > 2 Million items on Otto.de 80% turnover online e-Commerce Solutions & Technology Product 5 March 2014 Seite 10
But why rebuild Otto.de?
Goals Non-functional: Time To Simple Market Functional: Test-Driven Data-Driven Reliable Personalized Features Scalable Fast Realtime
How green is the green field? OTTO E-Commerce Frontend Infrastructure Articles Orders Customer OTTO Backend Infrastructure Product Customer Order Information Management Management Management
Start of the project LHOTSE Open Source as core technologies One Prototype to define the technology stack Project organization with autonomous teams Scrum as an agile development method Technical system architecture
3. A system-of-systems approach
Domain architecture Macro (technical) architecture
JRuby C# Groovy Scala Clojure Java
NoSQL RDBMS K/V RDBMS/ NoSQL RDBMS DWH DocDB
NoSQL RDBMS K/V RDBMS/ NoSQL RDBMS DWH DocDB Micro architecture
Module C Module B Module A Persistence Logic UI
UI UI UI Logic Logic Logic Persistence Persistence Persistence System A System B System C
4. The Otto architecture
Buying Process – as you already know it Customer Journey Discover Search Assess Order Check
The E-Commerce Business Architecture – Vertical and Horizontal aspects of the product Otto.de Discover Search Assess Order Check Usability Webanalytics and Testing Online and Performance Marketing Platform Engineering 1 Website = 1 Product = 1 System = 1 Engineering Team ?
System architecture is vertical UI UI UI UI UI Search Product Order User After … Sales
Organizational aspects UI UI UI User Auth After Sales 1 Team à N Systems 1 System à 1 Team
Architecture rules Micro-Architecture Macro-Architecture RESTful Architecture Buy when non core Shared Nothing Common Technologies Vertical Design Data Governance
The 3 Faces of Product Ownership Business Lead Product Owner Project Technical Lead Lead
Project start with distributed teams … Team Team Team Team Discover Search Check Order But how to deal with frontend integration?
5. Frontend Integration
Server-side integration options ESI (Portal server) Edge integration Homegrown REST RMI Backend call RPC Feeds Storage DB replication Chef, Puppet, … Deployment Build tools Asset pipeline Git/SVN submodules Gems Development Maven artifacts
Client-side integration options Client call SPA-style oEmbed Unobtrusive JS Replaced link ROCA-style Magical integration concept Link
Otto.de – detail view
Otto.de - Basket
Management of JS and CSS from each team System View Order User Personalization
Separate teams for horizontal aspects Team Search Team Team Discover Order Team Assetserver
Decoupling Versioned storage Asset Server Local SCM GitHub
6. A/B-Testing
AB-Testing in a distributed environment – What is an AB- Test 1% 99%
Solution with a centralized framework every team has to include in their repository User Request User Response Backoffice R Vertical (e.g. Search) E S UI for Managing Tests T Persistence Pull Experiment Data from Backoffice Testing specific and Vertical independent logic DB DB
Solution with a dedicated Vertical and loose coupling Request Response Testing Vertical Backoffice R R Vertical (e.g. Search) E E S S Implement and UI for Managing Tests T Testing Logic T Deliver Alternative A Persistence Persists all Tests and B ESI-Includes Pull Data Frontend- from Back- Proxy office DB DB DB
7. Conclusion
Results of the project LHOTSE 2 years in total Scaled to >100 people Finished in budget Finished in quality Finished before schedule: October, 24th à 4 months earlier Minimum Viable Product
Lessons learned in applying a system-of-systems approach Independent, autonomous systems for Prefer “pull” to maximum decoupling “push” sharing Strict macro architecture rules Minimize cross- functional concerns Address cross- functional concerns Minimize need Be skeptical of for coordination “easy” solutions Teams with their own decisions
@olliwegner @stilkov
BACKUP
Plattform Engineering delivers the basic infrastructure for all verticals … Team Team Team Team Discover Search Plattform Engineering Order Infrastructure Deployment Logging Provisioning Components
Microservices? Microservices Approach Vertical Systems Approach – Very small – Medium-sized – 100s – 10s – Does one thing only – Small # of related things – Separate client (?) – Includes UI
Recommend
More recommend
