Consistency of NoSQL Models Au Tran, Thy Nguyen, Chaz Chang, Vijaypal Singh, Timothy To, Akash Budholia
Introduction: From RBDMS to NoSQL In the past, ACID (Atomicity, Consistency, Isolation, and Durability) was a ● must have requirement for all traditional monolithic database systems. Strong consistency was a must have for any database system, which only offers vertical-scalability and prevents horizontal-scalability. As the demand grow, the need to scale for high availability become ● necessary. For this reason, strong consistency can no longer be enforced and databases must relax their consistency levels. Therefore, the NoSQL databases systems have emerged. Strong consistency High Availability
Introduction: Sub-categories of NoSQL Databases In this study, we compare the consistency models of five most popular non-cloud database systems: Redis, Cassandra, MongoDB, orientDB and Neo4j. Redis Key-value store Cassandra Column store MongoDB Document Neo4j Graph database OrientDB Multi-model
Introduction: Data-centric and Client-centric
Consistency Models We will review 8 main consistency models: Strong consistency ● Weak consistency ● Eventual consistency ● Causal consistency ● Read-your-writes consistency ● Session consistency ● Monotonic Reads consistency ● Monotonic Writes consistency ●
Strong consistency vs Weak consistency Strong consistency Weak consistency (a.k.a Linearization) Does not guarantee specific ● order of events Operation: must be ● Read Operation: does not ● committed immediately => guarantee to have the most events in order and same data updated value state for all clients Inconsistency window: The ● Read operation : After all write ● time period between the write commits are done => new operation and when every read version of data operation returns the updated value
Eventual consistency Weak Consistency Eventual Consistency Strong consistency Eventual consistency strengths Weak Consistency. ● In this model, it is possible for read operations to retrieve the ● older version instead of the latest one, like Weak Consistency while the replicas converge to the same data state However, after the inconsistency window, the latest data will be ● retrieved.
Causal Consistency Weak Eventual Causal Sequential Consistency Consistency Consistency Consistency If some process updates a given object: ● Processes acknowledge the update: get updated value ○ Processes do not acknowledge the update: follow Eventual Consistency Model ○
Read-your-writes Consistency Read-your-writes consistency allows ensuring that a replica is at least ● current enough to have the changes made by a specific transaction. If some process updates a ● given object, this same process will always consider the updated value. Other processes will ● eventually read the updated value after the inconsistency window
Session Consistency In the context of the existence of a session, read-your-writes ● consistency model will be applied. All reads are current writes from that session, but writes from other ● sessions may lag. Data from other sessions come in the correct order, just isn’t guaranteed ● to be current. Good performance and good availability at half the cost of strong ● consistency
Monotonic Reads Consistency ● After a process reads some value, all the successive reads will return that same value or a more recent one. Monotonic reads ensure that if a process ● performs read x1, then x2, then x2 cannot observe a state prior to the writes which were reflected in x1; intuitively, reads cannot go backward. Monotonic reads do not apply to ● operations performed by different processes, only reads by the same process.
Monotonic Writes Consistency ● A write operation invoked by a process on a given object needs to be completed before any subsequent write operation on the same object by the same process. Monotonic writes ensure that if a process ● performs write w1, then w2, then all processes observe w1 before w2. Monotonic writes do not apply to ● operations performed by different processes, only writes by the same process. -
Redis Description from the official website (https://redis.io/): "Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries"
Redis - Background Key-value store ● Optimizes data in memory by: ● prioritizing high performance ○ low computation complexity ○ high memory space efficiency ○ low application network traffic ○ Guarantees high availability by extending its architecture and introducing ● the Redis Cluster Strong consistent on a single instance configuration ● Eventual Consistent in a cluster when the client reads from replica nodes ●
Redis - Cluster specification High performance and linear scalability up to 1000 nodes. ● Relaxed write guarantees: Redis Cluster tries its best to retain all write ● operations issued by the application, but some of these operations can be lost. Availability: Redis Cluster survives network partitions as long as the ● majority of the master nodes are reachable and there is at least one reachable slave for every master node that is no longer reachable.
Redis - Keys and master-slave model Redis Cluster distributes keys into 16384 hash slots . ● Each master stores a subset of the 16384 slots. ● To compute the hash slot of a given key, the formula below is used ● (CRC16 used as a hash algorithm): HASH_SLOT = CRC16(key) mod 16384 Architecture implements a master-slave model without proxies which ● means that the application is redirected to the node that has the requested data. Redis nodes do not intermediate responses. Each master node holds a hash slot. This slot has 1 to N replicas (including ● the master and its replica nodes).
Redis - Hash tags Hash tags ensure that two keys are allocated in the same slot ● Allows for multi-key operations ● Part of the key has to be a common substring between the two keys and ● inside brackets These two keys end up in the same slot because only the substring inside ● the brackets will be hashed {user:1000}following {user:1000}followers
Redis - Redis Cluster Redis Cluster is formed by N nodes ● Node connected by TCP connections. Each node has N-1 outgoing ● connections and N-1 incoming TCP connection connections. A connection is kept alive as long ● as the two connected nodes live.
Redis - Asynchronous replication and writes When a master node receives ● OK to app an application issued request, it handles it and M W1 M W2 asynchronously propagates any changes to its replicas. master node by default ● W1 W1 S S acknowledges the application without an assured replication.
Redis - Asynchronous replication and writes On the asynchronous replication configuration (default), if the master ● node dies before replicating and after acknowledging the client, the data is permanently lost. Therefore, the Redis Cluster is not able to guarantee write persistence at all times. This behavior can be overwritten by explicitly making a request using the ● WAIT command, but this profoundly compromises performance and scalability— the two main strong points of using Redis.
Redis - Node failure A A Suppose we have a master node A M ● and a single replica A1. If A fails, A1 is promoted to master, ● and the cluster will continue to A1 A1 S M operate. However, if A has no replicas or A and ● A1 fail at the same time, the Redis Cluster will not be able to continue operating.
Redis - Network partition M A A S S S M S A1 A2 A1 A2
Redis - Network partition In the case of a network partition event, if the client is on the minority side ● with master A, while on the majority side resides its replicas A1 and A2, if the partition holds for too long (NODE_TIMEOUT) the majority side starts an election process to elect a new master among them, either A1 or A2. Node A is also aware of the timeout and its role change from master to ● slave. Consequently, it will refuse any further write operations from the client. In this case, Redis Cluster is not the best solution for applications that ● require high-availability, such as large network partition events.
Redis - Replica migration M A M B A M M B S S S S S S A1 A2 B1 A1 B1 B2
Redis - Replica migration Suppose that the majority side has N nodes and A and B and its replicas, ● A1, B1, and B2, respectively, and a network partition event occurs in such way that the replica A1 is separated from the rest. If the partition lasts long enough for assuming A1 as unreachable, Redis ● Cluster uses a strategy called replicas migration to reorganize the cluster and because B has multiple slaves, one of B’s replicas will now replicate from A and not from B.
Recommend
More recommend
