The growth of Cloud, social, and mobile technology is driving increased use of NoSQL databases like Apache Cassandra, mongoDB, Amazon DynamoDB and Google Bigtable. A recent study by ESG revealed that 56% of companies have NoSQL databases in production or deployed as part of a pilot/proof of concept. A similar study by Couchbase revealed that 90% of companies consider NoSQL important or critical to their business.Once the province of Internet innovators like Google, Amazon, Facebook, LinkedIn, and Twitter, the use of NoSQL databases is now widespread across every major industry. Modern applications create new demands that developers and DBAs must address such as:

• Huge volumes of rapidly changing data including semi-structured, unstructured and polymorphic data.
• Applications are now delivered as services that must be always on, accessible from mobile devices and scaled globally to millions of users.
• Agile development methods where smaller teams work in sprints, iterate quickly and release code every few weeks, and in some cases daily.
• Scale-out architectures using open source software, commodity servers and Cloud computing.

The choice for many of these applications is a distributed database (NoSQL) that stores portions of the data on multiple computers (nodes) within the network. These databases scale rapidly by adding nodes to a cluster making them effective for both Cloud and on-premise use. Their schema-less design allows data of different formats to be added accommodating the large amount of unstructured data (documents, audio, video, images) in use today. In-memory processing and the use of direct attached storage can provide fast processing of queries and support real time analytics.

The innovations in performance, cost, and agility found in NoSQL databases come at a cost. These open source databases lack the robust internal tools necessary for effective data protection and disaster recovery (DR). They are not as operationally mature as their relational database counterparts. It took many years for IBM, Oracle, and Microsoft to build robust data management and protection capabilities into their products. Low cost and high performance don’t matter very much if databases are offline or contain corrupted data.  Historical approaches and mainstream data protection solutions used widely today are not suitable for NoSQL data protection, disaster recovery and copy management. Given the need for always-on resilient systems, NoSQL applications will require better tools for data protection, compliance, and disaster recovery. The issues include:

• NoSQL databases are eventually consistent and scale-out using locally attached storage (DAS). LUN-based snapshots do not produce application-consistent copies of the database suitable for restore or use in DR.
• Stopping database writes to allow the system to catch up and produce an application-consistent snapshot is not practical.
• Traditional backups of cluster nodes don’t describe the dependencies between VMs and their applications. A detailed understanding of application configurations is required to restore a NoSQL cluster from VM backups.
• NoSQL nodes can be dynamically added (and removed) causing data to move between the nodes. Backups using traditional products don’t reflect the current state of NoSQL nodes making them unusable for DR.
• Taking application-inconsistent snapshots and running a database repair (replay) process takes hours–days and is not practical for low RPO DR.

The main tools used today for NoSQL fall short of providing a robust, efficient solution for data protection and DR. Scripting is a common tool used in backup and recovery, however, it is labor intensive and prone to error as configurations constantly change. Replication is meant to address availability and is often not effective for data backup and DR. If the database becomes corrupted then bad data simply replicates itself across nodes. Keeping multiple replicas to support DR creates wasted storage space, management issues, and the need for some method to deduplicate the data.

Given the importance of NoSQL to mission critical applications it is not surprising new data protection and DR solutions have come to market. One such solution is from Datos IO (http://www.datos.io).  Datos IO is driven by a vision to make data recoverable at scale for next-generation databases. Their product solves the main issues described above. It creates application-consistent point in time backups (versioning) across all nodes of common NoSQL databases. These consistent copies remove manual effort during restores or the need for replaying activity logs. Datos IO allows point in time versions to be created as frequently as every 15 minutes. Backups can be stored on-premise (in NFS format) or in public cloud.

Backup is only a part of the Datos IO solution. Their solution also provides near instantaneous data restore. They do this by storing NoSQL backup data in native format. Datos IO provides backup of the database at granular levels providing the technology needed to meet low RTO-RPO scenarios. Backups are incremental forever limiting the bandwidth required to keep NoSQL backups current and lowering operational costs. The Datos IO backup becomes the single point of truth about the state of a NoSQL database.

Disaster recovery can be accomplished with a single click and is configurable to meet a variety of needs. The database can be restored to the same, or an entirely different cluster, an important feature for Cloud-based DR. This capability also allows Datos IO to support DevOps use cases by rapidly creating test/dev nodes or data migrations across Cloud environments. Datos IO is also space efficient and performs semantic deduplication of its backups saving customers up to 70% on the cost of recovery storage.

Data protection and disaster recovery technologies follow new innovations to market. The explosive growth of NoSQL databases requires the operational maturity of their relational database counterparts. Datos IO is bringing the robust quality of traditional data protection products to modern Cloud, big data, distributed applications.

To learn more about data protection, Cloud, and trends in resilience subscribe to my blog here