Glossary of a few terms you need to know as you read on: Sharding (or Horizontal partitioning) is a database design principle whereby the content of a database table are split across physical locations by rows instead of by columns (using referential integrity). Each partition forms part of a shard. Multiple shards together provide a complete data set but the partitioned shards are split logically to ensure faster reads and writes.
Database Replication is the process of sharing data between the primary and one or more redundant databases, to improve reliability, fault-tolerance, or accessibility. Typically, data is immediately copied over to backup location upon write so as to be available for recovery and/or read-only resources.
Riak
Riak is a key-value storage database management system. It is one of the databases that fall under the NoSQL umbrella. Inspired by Amazon’s Dynamo, it is designed to be horizontally scalable, distributed, providing fast reads and writes while being highly available through software and hardware failures.
Riak stores data as key-value pairs with the value being the data that can be encoded following the demands of the application. Keys are organized into buckets, which I will explain in more detail later in the post. Operations on data use HTTP verb like commands PUT, GET and DELETE. In addition to your own data, the value portion of the key-value pair stores metadata – both out of the box and custom. Another powerful feature of Riak is the ability to manage relationships via links which are HTTP Rest-style links. These links are easy to decipher and understand by machine and human alike.
Riak is especially suited when high availability is of primary importance. This is typically for logging, analytics data storage, and when performing aggregation and analytics specific queries. As a database built from the ground up, it is specifically designed for high read and write throughput, providing seamless failover and guaranteed up-time all the while running on commodity servers.
Riak is available as open-source or as an Enterprise version with additional features and technical support from Basho, the company behind Riak.
The Data Model
Riak is a key-value storage database. The key is a unique identifier. The value is an amorphous value that can be custom-encoded as the application requires. Associated with the key, along with the value, is metadata for the data. You can also add custom metadata to the document to supplement the Riak provided metadata. Keys are always associated with a bucket; thus making the bucket-key combination, the unique reference to any data stored.
A Bucket/Key pair is called a Riak object. This Riak object can also store links to other Bucket/Key pairs thus providing the relationships often desired in databases. Because access is via HTTP Rest-style addressing, the metadata is accessible via HTTP Headers when Riak objects are retrieved.
Listing A
“employees” bucket may contain multiple “employee” keys. Each “employee” key has value to represent that employee’s data.
Each employee is addressable as: employees/emp1001 where emp1001 is the employee id, a value that is unique to each employee.
The key emp1001 can now hold data and metadata as shown:
emp1001.content_type = “application/json” emp1001.data = {name: ‘Mike Smith’, dept:’FIN’,title:’Manager’}
The default storage back-end for Riak is Bitcask. Riak uses a pluggable API to address the storage back-end and thus can seamlessly address other supported back-ends as needed.
Querying and Secondary Indexes
Querying options in Riak include the three following techniques - writing MapReduce functions, full-text search on data or writing custom indexes and performing http GET operations.
Riak allows clients to query for data using HTTP GET operations. An R value can be set for each fetch where R signifies the minimum number of nodes that must successfully process the query for the query operation to be completed successfully. Riak can also return objects based on links stored on the object.
Riak supports MapReduce operations to execute complex query operations. A series of functions comprise a MapReduce job with the result of one function serving as input to the next in series until the result is finally served back to the client.
A new and improved feature in the recent release of Riak v 1.0 supports secondary indexes for efficient querying. The goal of this feature is to provide simple indexing options on data buckets at data write-time.
For example, when creating an employee record, you can create it within the "employee" bucket with a key which is a unique employee id, add data element to it (name, title) and finally, add index metadata for fields that are expected to be used in querying (skills, salary).
curl -XPUT -d"{name: 'Mike Smith', dept:'FIN',title:'Manager'}" -H"x-riak-index-skills_bin:accounting"\
-H"x-riak-index-salary_int:100000" http://127.0.0.1:8098/riak/employees/employee200
To then query for employees with "accounting' skills, you would perform the following operation:
curl http://127.0.0.1:8098/buckets/employees/index/skills_bin/accounting
To then query for employees whose salaries range between 50000 and 150000, you would perform the following operation:
curl http://127.0.0.1:8098/buckets/employees/index/salary_int/50000/150000
NOTE: In the above two employee records, there are two fields in addition to the basic data element containg name, department and title. These are: salary and skills. To add a secondary index, the field name must contain a suffix "_bin" for binary fields or "_int" for integer fields.
Of course, if you decided that you should be able to search your employees data by title, you could add another index like below in addition to maintaining the title as part of your basic data.
curl -XPUT -d"{name: 'Mike Smith', dept:'FIN',title:'Manager'}" -H"x-riak-index-skills_bin:accounting"\ -H"x-riak-index-salary_int:100000"-H"x-riak-index-title_bin:Manager" http://127.0.0.1:8098/riak/employees/employee200
Riak runs as a cluster by default. Each physical Riak server is called a node. Each node has multiple "virtual nodes" within it. In a node, the bucket/key pairs are stored on a ring-like configuration where the ring is segmented into the number of "virtual nodes" running within it. Each "virtual node" is responsible for a segment. A cluster of nodes does not have a designated Master node. All nodes are equal and can serve any request from a client. As I mentioned earlier, Riak stores copies of data across the nodes in the cluster. The number of copies stored is configurable and is consistent across nodes in a cluster. Riak promises seamless addition and removal of nodes to and from a cluster.
Riak aims for eventual consistency while focusing on complete availability. Eventual consistency means that your data will be eventually consistent on all nodes over time. The latency is not expected to be more than a few hundred milliseconds and thus is usually acceptable. Nodes focus on being up and available for reads and writes and after that, they replicate data to other nodes.
Data Integrity
Riak is Masterless and hence write and read operations are able to be fulfilled by any node in a cluster. Being a distributed system with multiple copies of data across nodes, Riak has to provide some sort of conflict resolution and write safety. For every update, Riak accepts a ‘W’ value which is a number that represents the minimum number of nodes in a cluster that must report a successful update before the operation is deemed successful. With this safety net, an application can write to a Riak cluster even if one node is (or more nodes are) down. This number, coupled with another configurable number to indicate number of copies of a value to be stored, can be used to optimize the cluster.
Riak also has a configurable option to handle conflicts when it encounters data updated by separate sources. It allows you to configure it so the most recent update wins and is propagated to all nodes or you can configure it to present the differences to the client application for resolution based on a custom algorithm or manual intervention.
What do you give up?
RDBMS have long rules the database world for everything from simple persistence to complex analytical database engines. It is inevitable that NoSQL databases are compared to relative to RDBMS features. Here are a few things of note:
By using Riak, you give up the notion of 100% consistent data in the database in a cluster. Since Riak works on the principle of eventual consistency, you will encounter situations where retrieved data is stale and you application has to be designed to handle such scenarios.
There is no concept equivalent to table joins and foreign key constraints in Riak.
Because content is not typed, queryability is not as efficient as RDBMS.
With the data model being a kay-value pair, you lose the concept of a rich domain with strongly typed data.
You have to have a certain tolerance for latency and conflict resolution for update operations.
By using Riak, you will give up some of the powerful SQL Querying constructs supported by RDBMS databases.
On the bright side, you give up complex administration of RDBMS, the expensive hardware and the potential single point of failure inherent to RDBMS solutions.
Summary
This was a brief introduction to Riak, a popular key-value storage NoSQL database designed and built for high availability applications that can accept eventual consistency. The latency for data consistency being at most a few 100 milliseconds makes it a palatable alternative.
However, it is important to know that a NoSQL database may not be the right option for every use case. And further, even if you have a strong case for a NoSQL database, Riak may still not be the right choice for you.
Consider the following as you determine a fit your usage scenarios:
- Riak is a key-value storage database.
- Riak can act as primary storage for your web apps but also excels as an ancillary database for your applications to store user session data for web applications, data for web services etc.
- It acts as a scalable low-latency database for mobile apps where application availability for data entry is critical no matter the situation.
- Riak is schema-less and light-weight. This allows you to model a domain whose attributes are expected to vary over time.
- Riak’s high availability feature is a good fit for systems that need to be up for writes under any and all circumstances and eventual consistency is acceptable. In such cases, the merge conflicts are typically easily resolved through the automatic algorithms in Riak so that the databases can self-repair conflicts.
- Riak’s scaling features allows you to support a large database and still meet speed and availability specifications typically required of web applications.
- Not having to worry much about DBA activities, developers control the data format which leads to seamless development and enhancements to existing features, all resulting in faster delivery.
- Being a key-value store databse, Riak is not best suited for ad-hoc querying since the data structure in the value is not clearly defined. This results in data-analytics queries directly on Riak to not perform as well as one may expect.
For documentation, instructions on installing and tutorials, visit www.basho.com.
No comments:
Post a Comment