Nosql Systems

A NoSQL database is one that stores and queries data without the relational model's tables, fixed schemas, and SQL as its native interface.

8 frameworks, from 1965 to today · Databases · Computer Science

still a working view ended · each leads to its entry below

Overview

A NoSQL database is one that stores and queries data without the relational model's tables, fixed schemas, and SQL as its native interface1. The name, coined around 2009, is usually read as "not only SQL": the systems are not defined by a shared theory but by what they gave up and what they gained1. The relational model, described by E. F. Codd in 1970 and made practical by IBM's System R and the commercial systems that followed, had won the database world by the 1990s2. It offered a declarative query language, transactions with strong guarantees, and a clean separation between logical data and physical storage. What it struggled with, at the scale the web reached in the 2000s, was volume, distribution, and schema churn. NoSQL is the history of systems built to answer those pressures, and of the ideas — some old, some new — that they revived or invented.

Navigational stores and the pre-relational inheritance

Before the relational model, databases were navigational: programs followed pointers from record to record2. IBM's IMS (1960s) organized data hierarchically; the CODASYL network model linked records through sets23. These systems were fast and tightly coupled to application code, but queries had to be written as traversal programs, and changing the data structure meant changing the programs. Codd's relational model replaced pointers with values and set-based queries, and by the 1980s relational systems dominated commercial data processing2.

The key-value idea never disappeared, though. The associative array — a map from keys to values — is one of computing's oldest structures, and systems such as IBM's VSAM and later in-memory caches kept it in use1. What the 2000s added was distribution: a key-value store spread across many machines, with the mapping from key to machine handled by the system rather than the programmer.

The web-scale pressure and the CAP argument

The immediate trigger for NoSQL was the growth of large web services. Google's file system paper (2003) and its MapReduce paper (2004) described how to store and process data across thousands of commodity machines4. In 2006 Google published Bigtable, a sparse, distributed, sorted map indexed by row key, column name, and timestamp5. Bigtable's data model — rows, column families, and versioned cells — became the template for what is now called the wide-column store6. Around the same time, Amazon described Dynamo (2007), a highly available key-value store that used consistent hashing to partition data and let applications resolve conflicting writes rather than blocking on them78.

These systems were built against a theoretical backdrop. In 2000 Eric Brewer conjectured what became the CAP theorem: a distributed system facing a network partition cannot simultaneously guarantee consistency (every read sees the latest write) and availability (every request gets a response)9. Gilbert and Lynch formalized it in 20029. The theorem was widely read as a menu — pick two of consistency, availability, and partition tolerance — and many early NoSQL systems chose availability and partition tolerance, offering eventual consistency: replicas may diverge temporarily but converge if writes stop10. Dynamo's design, with vector clocks to detect conflicting versions and read-repair to reconcile them, was the canonical example. Later commentary, including Brewer's own 2012 reassessment, stressed that the trade-off is finer-grained and that partitions are rare, but the theorem had already shaped a generation of designs9.

Four data models take shape

The systems that followed clustered into families defined by their data model.

Key-value stores keep the simplest contract: put a value under a key, get it back. Redis (2009), created by Salvatore Sanfilippo, kept data in memory and added structures — lists, sets, sorted sets — on top of the basic map, which made it a common cache and session store11. Amazon's Dynamo and its descendants, including Riak (2009) and LinkedIn's Voldemort, emphasized availability across nodes18.

Document stores keep semi-structured records, usually JSON or a JSON-like format, and let queries reach inside them112. MongoDB, begun in 2007 by Dwight Merriman and Eliot Horowitz's company 10gen and released in 2009, stored BSON documents in collections and let developers index and query nested fields13. CouchDB (2005, Damien Katz) took a different route, emphasizing replication and a map/reduce view model14. The document model's appeal was direct: an application object could be stored roughly as the program already represented it, without decomposing it into tables.

Wide-column stores follow Bigtable: rows identified by a key, grouped into column families, with cells versioned by timestamp6. Apache HBase (2008) was an open-source implementation on top of Hadoop; Cassandra, begun at Facebook in 2008 and open-sourced in 2009, combined Bigtable's data model with Dynamo's distribution1516. These systems suited very large, sparse tables written at high volume.

Graph databases store nodes and edges as first-class entities and traverse them directly3. Neo4j, released in 2007 by Neo Technology, used a property-graph model — nodes and relationships carrying key-value properties — and a traversal-oriented query language317. The model's case was that some questions, such as "friends of friends who bought this," are cheap as pointer-chasing and expensive as repeated relational joins. Graph databases descend from older network and semantic-network ideas but were rebuilt for modern workloads3.

Polyglot persistence and the operational turn

As these systems matured, the practical lesson was that no single model fits every workload. The term polyglot persistence (popularized by Martin Fowler and others around 2011) named the pattern of using several stores in one application: a document store for product catalogs, a key-value cache for sessions, a graph store for recommendations, a relational database for billing18. The pattern was less a technology than an admission that the relational monopoly had ended and that choosing a store had become an architectural decision.

The early systems also exposed their limits. Many offered no multi-document transactions, weak or ad hoc query languages, and operational complexity. Over the 2010s they responded: MongoDB added a aggregation framework and, by 2018, multi-document transactions; Cassandra added a SQL-like query language (CQL); graph systems standardized around Cypher and, later, GQL313. The gap between NoSQL and relational systems narrowed from both sides.

NewSQL and the return of strong guarantees

The most consequential response came from a group of systems that kept SQL and ACID transactions but borrowed NoSQL's distribution techniques. NewSQL was coined around 2011 to describe this class19. Google's Spanner (2012 paper) was the landmark: it used synchronized clocks (TrueTime) to order transactions globally and offered externally consistent distributed transactions at scale20. CockroachDB (2015) and YugabyteDB built similar architectures on open-source foundations; VoltDB pursued in-memory partitioned transactions21. These systems argued that the CAP trade-off could be softened by better clocks, consensus protocols such as Paxos and Raft, and careful engineering — that strong consistency and horizontal scale were not mutually exclusive. Their success pushed the mainstream relational vendors toward distributed architectures as well.

Where the field stands

Today the boundary the name drew has largely dissolved. Cloud providers offer managed key-value, document, wide-column, and graph services alongside distributed SQL databases; PostgreSQL has absorbed JSON storage and extensions for graph and time-series work; and the term "NoSQL" is used more as a historical label than a technical one1. The enduring contributions are the data models themselves — the document, the wide-column table, the graph, the distributed key-value map — and the operational vocabulary of replication, partitioning, and tunable consistency that the field made ordinary163. The open questions are the old ones in new dress: how to give developers flexible schemas without losing correctness, how to make distributed transactions cheap enough to be the default, and how to reconcile the many stores a modern application now depends on.

References

  1. NoSQL (introduction, History, Key–value store, Types and examples, Document store), Wikipedia
  2. Database (History), Wikipedia
  3. Graph database (History, introduction), Wikipedia
  4. Apache Hadoop (History), Wikipedia
  5. Bigtable, Wikipedia
  6. Wide-column store, Wikipedia
  7. Dynamo (storage system), Wikipedia
  8. Riak, Wikipedia
  9. CAP theorem (introduction, History), Wikipedia
  10. Eventual consistency, Wikipedia
  11. Redis, Wikipedia
  12. Document-oriented database, Wikipedia
  13. MongoDB (introduction, History), Wikipedia
  14. Apache CouchDB (introduction, History), Wikipedia
  15. Apache HBase, Wikipedia
  16. Apache Cassandra (introduction, History), Wikipedia
  17. Neo4j (introduction, History), Wikipedia
  18. Polyglot persistence, Wikipedia
  19. NewSQL (History), Wikipedia
  20. Spanner (database) (introduction, History), Wikipedia
  21. CockroachDB (introduction, History), Wikipedia

The frameworks of Nosql Systems

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 2000-Present

    CAP theorem and eventual consistency

    What must a distributed system give up during a network partition?

    Fly to its tower in Nousfarer

  2. 2005-Present

    Document-oriented database

    How can semi-structured records be stored and queried directly?

    Fly to its tower in Nousfarer

  3. 2006-Present

    Wide-column store

    How can very large sparse tables be stored across many machines?

    Fly to its tower in Nousfarer

  4. 2007-Present

    Graph database

    How can relationships between records be traversed cheaply?

    Fly to its tower in Nousfarer

  5. 2007-Present

    Key–value database

    How can a simple map from keys to values span many machines?

    Fly to its tower in Nousfarer

  6. 2011-Present

    NewSQL

    Can strong transactions and horizontal scale coexist?

    Fly to its tower in Nousfarer

  7. 2011-Present

    Polyglot persistence

    How should an application combine several specialized data stores?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Navigational data modelsCAP theorem and eventual consistencyDocument-oriented databaseWide-column storeGraph databaseKey–value databaseNewSQLPolyglot persistence

Beyond the field

Where Nosql Systems's ideas came from in other fields, and what they led to there: the strongest 12 of 46 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Nosql Systems is a planet in Nousfarer

Its city has a tower for each of these 8 frameworks. Land, walk in, and climb one concept at a time.

Land on Nosql Systems

Related fields

Fields its ideas are linked to

More in Databases

See where Nosql Systems sits on the Map of Human Knowledge