Introduction Couchbase Server 2.0 Tugdual Grall - PowerPoint PPT Presentation

Introduction à Couchbase Server 2.0 Tugdual Grall TechnicalEvangelist @tgrall

{“about” : “me”} Tugdual “Tug” Grall Web • Couchbase @tgrall • • Technical Evangelist http://blog.grallandco.com • • eXo Tgrall • • CTO • Oracle NantesJUG co-founder • • Developer/Product Manager • Mainly Java/SOA Pet Project : • • Developer in consulting firms http://www.resultri.com • •

Agenda 1. Introduction to Couchbase Server 2. Cluster Management 3. Operation and Document Design 4. Introduction to Views and Queries 5. Cross Data Center Replication 6. Q&A Couchbase Developer Day Paris is this Friday! http://couchbase.com/events

Couchbase Server 2.0

Couchbase Server NoSQL Document Database

Couchbase Open Source Project • Leading NoSQL database project focused on distributed database technology and surrounding ecosystem • Supports both key-value and document-oriented use cases • All components are available under the Apache 2.0 Public Couchbase License Open Source Project • Obtained as packaged software in both enterprise and community editions.

Couchbase Server Easy Consistent High Scalability Performance Grow cluster without Consistent sub-millisecond application changes, without read and write response times downtime with a single click with consistent high throughput Always Flexible Data On J S O N Model J S O N J S ON J S ON J S O N 24x365 No downtime for software JSON document model upgrades, hardware with no fixed schema. maintenance, etc.

New in 2.0 JSON support Indexing and Querying J S O N J S O N J S ON J S ON J S ON Incremental Map Cross data center Reduce replication

Couchbase Server 2.0 Architecture 8092 11211 11210 Query API Memcapable 1.0 Memcapable 2.0 Moxi Query Engine vBucket state and replication manager REST management API/Web UI Memcached Global singleton supervisor Rebalance orchestrator Configuration manager Node health monitor Process monitor Heartbeat Couchbase EP Engine Data Manager Cluster Manager storage interface New Persistence Layer http on each node one per cluster Erlang/OTP HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 - 21199

Couchbase Server 2.0 Architecture 8092 11211 11210 Query API Memcapable 1.0 Memcapable 2.0 Moxi Query Engine vBucket state and replication manager REST management API/Web UI Object-level Cache Global singleton supervisor Rebalance orchestrator Configuration manager RAM Cache, Node health monitor Process monitor Server/Cluster Heartbeat Indexing & Couchbase EP Engine Management & Persistence Communication storage interface Management (Erlang) (C & V8) New Persistence Layer Disk Persistence http on each node one per cluster Erlang/OTP HTTP Erlang port mapper The Unreasonable Effectiveness of C by Damien Katz Distributed Erlang 8091 4369 21100 - 21199

Single node - Couchbase Write Operation Doc 1 App Server Couchbase Server Node 3 3 2 Managed Cache Replication To other node Doc 1 Doc 1 Queue Disk Queue Disk

Single node - Couchbase Read Operation Doc 1 GET App Server Couchbase Server Node 3 3 2 Managed Cache Replication To other node Queue Doc 1 Doc 1 Disk Queue Disk Doc 1

Basic Operation App Server 1 App Server 2 COUCHBASE Client COUCHBASE Client Library Library Cluster Map Cluster Map READ/WRITE/UPDATE • Docs distributed evenly across servers Server 1 Server 2 Server 3 • Each server stores both active Active Active Active and replica docs Doc 5 Doc Doc 4 Doc Doc 1 Doc Only one server active at a time Doc 2 Doc Doc 7 Doc Doc 2 Doc • Client library provides app with Doc 9 Doc Doc 8 Doc Doc 6 Doc simple interface to database • Cluster map provides map REPLICA REPLICA REPLICA to which server doc is on Doc 4 Doc Doc 6 Doc Doc 7 Doc App never needs to know Doc 1 Doc Doc 3 Doc Doc 9 Doc • App reads, writes, updates docs Doc 8 Doc Doc 2 Doc Doc 5 Doc • Multiple app servers can access Couchbase Server Cluster same document at same time User Configured Replica Count = 1

Add Nodes to Cluster App Server 1 App Server 2 COUCHBASE Client COUCHBASE Client Library Library Cluster Map Cluster Map • Two servers added READ/WRITE/UPDATE READ/WRITE/UPDATE One-click operation Server 1 Server 2 Server 3 Server 4 Server 5 • Docs automatically Active Active Active Active Active rebalanced across cluster Doc 5 Doc Doc 4 Doc Doc 1 Doc Even distribution of docs Doc 2 Doc Doc 7 Doc Doc 2 Doc Minimum doc movement Doc 9 Doc Doc 8 Doc Doc 6 Doc • Cluster map REPLICA REPLICA REPLICA REPLICA REPLICA updated Doc 4 Doc Doc 6 Doc Doc 7 Doc • App database calls now Doc 1 Doc Doc 3 Doc Doc 9 Doc distributed over larger number Doc 8 Doc Doc 2 Doc Doc 5 Doc of servers Couchbase Server Cluster User Configured Replica Count = 1

Fail Over Node App Server 1 App Server 2 COUCHBASE Client COUCHBASE Client Library Library Cluster Map Cluster Map App servers accessing • docs Server 1 Server 2 Server 3 Server 4 Server 5 Active Active Active Active Active Requests to Server 3 fail • Doc 5 Doc Doc 4 Doc Doc 1 Doc Doc 9 Doc Doc 6 Doc Cluster detects server • failed Doc 2 Doc Doc 7 Doc Doc 2 Doc Doc 8 Doc Doc Promotes replicas of docs to active Updates cluster map Doc 1 Doc 3 Requests for docs now • REPLICA REPLICA REPLICA REPLICA REPLICA go to appropriate server Doc 4 Doc Doc 6 Doc Doc 7 Doc Doc 5 Doc Doc 8 Doc Typically rebalance • would follow Doc 1 Doc Doc 3 Doc Doc 9 Doc Doc 2 Doc Couchbase Server Cluster User Configured Replica Count = 1

Demo Time

Operation & Document Design

Basic Store & Retrieve Operations • get(key) Retrieve a document • set(key, value) Store a document or replace if it exists • add(key, value) Store a document and fail if it exists • replace(key, value) Replace a document and fail if it does not exist

Lots of other Operations • Atomic Counters (increment, decrement) • Append/Prepend • CAS (Compare and Set) ­ Optimistic Locking • “Get with Lock” ­ Write Lock on Objects • Bulk Operations ­ Saves network overhead • Stats • View operations

Couchbase 2.0 Bonus Points: JSON • JSON is a lightweight format to represent document structure in a language-independent manner. • If JSON documents are stored, the View engine can be used. • Allows to build secondary indexes on your datasets. • Makes it possible to ask questions like “Give me all user documents by lastname”.

Flexible Data Model { “ID”: 1, “FIRST”: “Dipti”, “LAST”: “Borkar”, “ZIP”: “94040”, } JSON “CITY”: “MV”, JSON JSON “STATE”: “CA” JSON No need to worry about the database when changing • your application Records can have different structures, there is no fixed • schema Allows painless data model changes for rapid • application development

Aggregate View of Data http://martinfowler.com/bliki/AggregateOrientedDatabase.html

Store and Retrieve Aggregates order::1001 { uid: ji22jd, customer: Ann, line_items: [ { sku: 0321293533, quan: 3, unit_price: 48.0 }, { sku: 0321601912, quan: 1, unit_price: 39.0 }, { sku: 0131495054, quan: 1, unit_price: 51.0 } ], payment: { type: Amex, expiry: 04/2001, last5: 12345 } • Easier to Distribute } Data • More Flexibility • Reduced Latency

Demo Time

Official SDKs Ruby Python www.couchbase.com/develop Community SDKs Clojure Go

Indexing and Querying

Indexing and Querying – The basics • Define materialized views on JSON documents and then query across the data set • Using views you can define • Primary indexes • Simple secondary indexes (most common use case) • Complex secondary, tertiary and composite indexes • Aggregations (reduction) • Indexes are eventually indexed • Queries are eventually consistent • Built using Map/Reduce technology • Map and Reduce functions are written in Javascript

Indexing and Querying App Server 1 App Server 2 COUCHBASE Client COUCHBASE Client Library Library Cluster Map Cluster Map Query Indexing work is distributed Server 1 • Server 2 Server 3 amongst nodes Active Active Active Large data set possible • Doc 5 Doc Doc 5 Doc Doc 5 Doc Parallelize the effort • Doc 2 Doc Doc 2 Doc Doc 2 Doc Each node has index for data • Doc 9 Doc Doc 9 Doc Doc 9 Doc stored on it REPLICA REPLICA REPLICA Queries combine the results • from required nodes Doc 4 Doc Doc 4 Doc Doc 4 Doc Doc 1 Doc Doc 1 Doc Doc 1 Doc Doc 8 Doc Doc 8 Doc Doc 8 Doc Couchbase Server Cluster User Configured Replica Count = 1

Demo Time

Cross Data Center Replication (XDCR)

Cross Data Center Replication – The basics • Replicate your Couchbase data across clusters • Clusters may be spread across geos • Configured on a per-bucket (per-database) basis • Supports unidirectional and bidirectional operation • Application can read and write from both clusters ­ Active – Active replication • Replication throughput scales out linearly • Different from intra-cluster replication