Introduction Couchbase Server 2.0 Tugdual Grall - - PowerPoint PPT Presentation

Introduction à Couchbase Server 2.0 Tugdual Grall TechnicalEvangelist @tgrall

{“about” : “me”}

Tugdual “Tug” Grall

• Couchbase
• Technical Evangelist
• eXo
• CTO
• Oracle
• Developer/Product Manager
• Mainly Java/SOA
• Developer in consulting firms
• Web
• @tgrall
• http://blog.grallandco.com
• Tgrall
• NantesJUG co-founder
• Pet Project :
• http://www.resultri.com

Agenda

• 1. Introduction to Couchbase Server
• 2. Cluster Management
• 3. Operation and Document Design
• 4. Introduction to Views and Queries
• 5. Cross Data Center Replication
• 6. Q&A

Couchbase Developer Day Paris is this Friday! http://couchbase.com/events

Couchbase Server 2.0

Couchbase Server NoSQL Document Database

Couchbase Open Source Project

• Leading NoSQL database

project focused on distributed database technology and surrounding ecosystem

• Supports both key-value and

document-oriented use cases

• All components are available

under the Apache 2.0 Public License

• Obtained as packaged

software in both enterprise and community editions. Couchbase Open Source Project

Easy Scalability Consistent High Performance Always On 24x365

Grow cluster without application changes, without downtime with a single click Consistent sub-millisecond read and write response times with consistent high throughput No downtime for software upgrades, hardware maintenance, etc.

J S ON

J S O N J S O N

J S O N

J S ON

Flexible Data Model

JSON document model with no fixed schema.

Couchbase Server

New in 2.0

JSON support Indexing and Querying Cross data center replication Incremental Map Reduce

J S ON

J S O N J S O N

J S ON

J S ON

Heartbeat Process monitor Global singleton supervisor Configuration manager

• n each node

Rebalance orchestrator Node health monitor

• ne per cluster

vBucket state and replication manager

http REST management API/Web UI

HTTP

8091

Erlang port mapper

4369

Distributed Erlang

21100 - 21199

Erlang/OTP

storage interface

Couchbase EP Engine

11210

Memcapable 2.0

Moxi

11211

Memcapable 1.0

Memcached New Persistence Layer

8092

Query API

Query Engine

Data Manager Cluster Manager

Couchbase Server 2.0 Architecture

New Persistence Layer

storage interface

Couchbase EP Engine

11210

Memcapable 2.0

Moxi

11211

Memcapable 1.0

Object-level Cache Disk Persistence

8092

Query API

Query Engine

HTTP

8091

Erlang port mapper

4369

Distributed Erlang

21100 - 21199 Heartbeat Process monitor Global singleton supervisor Configuration manager

• n each node

Rebalance orchestrator Node health monitor

• ne per cluster

vBucket state and replication manager

http REST management API/Web UI Erlang/OTP

Server/Cluster Management & Communication (Erlang) RAM Cache, Indexing & Persistence Management (C & V8)

The Unreasonable Effectiveness of C by Damien Katz

Couchbase Server 2.0 Architecture

3 3 2

Single node - Couchbase Write Operation

Managed Cache

Disk Queue

Disk

Replication Queue

App Server Couchbase Server Node

Doc 1 Doc 1 Doc 1

To other node

GET Doc 1

3 3 2

Single node - Couchbase Read Operation

Disk Queue Replication Queue

App Server

Doc 1 Doc 1 Doc 1

Managed Cache Disk

To other node

Couchbase Server Node

Couchbase Server Cluster

Basic Operation

• Docs distributed evenly across

servers

• Each server stores both active

and replica docs

Only one server active at a time

• Client library provides app with

simple interface to database

• Cluster map provides map

to which server doc is on

App never needs to know

• App reads, writes, updates docs
• Multiple app servers can access

same document at same time

User Configured Replica Count = 1

READ/WRITE/UPDATE

Active Doc 5 Doc 2 Doc Doc Doc Server 1 Active Doc 4 Doc 7 Doc Doc Doc Server 2 Doc 8 Active Doc 1 Doc 2 Doc Doc Doc REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc REPLICA Doc 6 Doc 3 Doc 2 Doc Doc Doc REPLICA Doc 7 Doc 9 Doc 5 Doc Doc Doc Server 3 Doc 6

App Server 1

COUCHBASE Client Library Cluster Map COUCHBASE Client Library Cluster Map

App Server 2

Doc 9

Add Nodes to Cluster

• Two servers added

One-click operation

• Docs automatically

rebalanced across cluster

Even distribution of docs Minimum doc movement

• Cluster map

updated

• App database

calls now distributed

• ver larger number
• f servers

REPLICA Active Doc 5 Doc 2 Doc Doc Doc 4 Doc 1 Doc Doc Server 1 REPLICA Active Doc 4 Doc 7 Doc Doc Doc 6 Doc 3 Doc Doc Server 2 REPLICA Active Doc 1 Doc 2 Doc Doc Doc 7 Doc 9 Doc Doc Server 3 Server 4 Server 5 REPLICA Active REPLICA Active Doc Doc 8 Doc Doc 9 Doc Doc 2 Doc Doc 8 Doc Doc 5 Doc Doc 6

READ/WRITE/UPDATE READ/WRITE/UPDATE App Server 1

COUCHBASE Client Library Cluster Map COUCHBASE Client Library Cluster Map

App Server 2 Couchbase Server Cluster

User Configured Replica Count = 1

Fail Over Node

REPLICA Active Doc 5 Doc 2 Doc Doc Doc 4 Doc 1 Doc Doc Server 1 REPLICA Active Doc 4 Doc 7 Doc Doc Doc 6 Doc 3 Doc Doc Server 2 REPLICA Active Doc 1 Doc 2 Doc Doc Doc 7 Doc 9 Doc Doc Server 3 Server 4 Server 5 REPLICA Active REPLICA Active Doc 9 Doc 8 Doc Doc 6 Doc Doc Doc 5 Doc Doc 2 Doc 8 Doc Doc

• App servers accessing

docs

• Requests to Server 3 fail
• Cluster detects server

failed

Promotes replicas of docs to active Updates cluster map

• Requests for docs now

go to appropriate server

• Typically rebalance

would follow

Doc Doc 1 Doc 3

App Server 1

COUCHBASE Client Library Cluster Map COUCHBASE Client Library Cluster Map

App Server 2

User Configured Replica Count = 1

Couchbase Server Cluster

Demo Time

Operation & Document Design

Basic Store & Retrieve Operations

• get(key)

Retrieve a document

• set(key, value)

Store a document or replace if it exists

• add(key, value)

Store a document and fail if it exists

• replace(key, value)

Replace a document and fail if it does not exist

Lots of other Operations

• Atomic Counters (increment, decrement)
• Append/Prepend
• CAS (Compare and Set)

­ Optimistic Locking

• “Get with Lock”

­ Write Lock on Objects

• Bulk Operations

­ Saves network overhead

• Stats
• View operations

Couchbase 2.0 Bonus Points: JSON

• JSON is a lightweight format to represent document

structure in a language-independent manner.

• If JSON documents are stored, the View engine can be

used.

• Allows to build secondary indexes on your datasets.
• Makes it possible to ask questions like

“Give me all user documents by lastname”.

Flexible Data Model

• No need to worry about the database when changing

your application

• Records can have different structures, there is no fixed

schema

• Allows painless data model changes for rapid

application development

{ “ID”: 1, “FIRST”: “Dipti”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA”

}

JSON JSON JSON JSON

Aggregate View of Data

http://martinfowler.com/bliki/AggregateOrientedDatabase.html

Store and Retrieve Aggregates

• Easier to Distribute

Data

• More Flexibility
• Reduced Latency
• rder::1001

{ uid: ji22jd, customer: Ann, line_items: [ { sku: 0321293533, quan: 3, unit_price: 48.0 }, { sku: 0321601912, quan: 1, unit_price: 39.0 }, { sku: 0131495054, quan: 1, unit_price: 51.0 } ], payment: { type: Amex, expiry: 04/2001, last5: 12345 } }

Demo Time

Official SDKs Community SDKs

www.couchbase.com/develop

Python Ruby Go Clojure

Indexing and Querying

Indexing and Querying – The basics

• Define materialized views on JSON documents and

then query across the data set

• Using views you can define
• Primary indexes
• Simple secondary indexes (most common use case)
• Complex secondary, tertiary and composite indexes
• Aggregations (reduction)
• Indexes are eventually indexed
• Queries are eventually consistent
• Built using Map/Reduce technology
• Map and Reduce functions are written in Javascript

Couchbase Server Cluster

Indexing and Querying

User Configured Replica Count = 1

Active Doc 5 Doc 2 Doc Doc Doc Server 1 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc

App Server 1

COUCHBASE Client Library Cluster Map COUCHBASE Client Library Cluster Map

App Server 2

Doc 9

• Indexing work is distributed

amongst nodes

• Large data set possible
• Parallelize the effort
• Each node has index for data

stored on it

• Queries combine the results

from required nodes

Active Doc 5 Doc 2 Doc Doc Doc Server 2 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc Doc 9 Active Doc 5 Doc 2 Doc Doc Doc Server 3 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc Doc 9 Query

Demo Time

Cross Data Center Replication (XDCR)

Cross Data Center Replication – The basics

• Replicate your Couchbase data across clusters
• Clusters may be spread across geos
• Configured on a per-bucket (per-database) basis
• Supports unidirectional and bidirectional operation
• Application can read and write from both clusters

­ Active – Active replication

• Replication throughput scales out linearly
• Different from intra-cluster replication

Cross Data Center Replication (XDCR)

Couchbase Server Cluster US

Active Doc Doc 2 Server 1 Doc 9 Server 2 Server 3

RAM

Doc Doc Doc Active Doc Doc Doc

RAM

Active Doc Doc Doc

RAM DISK

Doc Doc Doc

DISK

Doc Doc Doc

DISK Couchbase Server Cluster EMEA

Active Doc Doc 2 Server 1 Doc 9 Server 2 Server 3

RAM

Doc Doc Doc Active Doc Doc Doc

RAM

Active Doc Doc Doc

RAM DISK

Doc Doc Doc

DISK

Doc Doc Doc

DISK

Couchbase Handles Real World Scale

Q & A

• Couchbase DeveloperDays Paris

­ This Friday ! ­ Presentations & Labs ­ Information

• http://www.couchbase.com/events

­ Use the promo code TUG

Thank you!

tug@couchbase.com @tgrall Get Couchbase Server at http://www.couchbase.com/download