Loading field map
Transaction processing is the discipline within database systems concerned with the reliable execution of operations that must be treated as indivisible units of work. A transaction is a sequence of database operations—reads, writes, computations—that is guaranteed to be atomic: either all of its effects are permanently recorded, or none of them are. The field studies how to provide this guarantee efficiently and correctly, especially when many transactions execute concurrently and when the system may fail at any moment.
The stakes are practical and high. Transaction processing underpins financial transfers, airline reservations, e-commerce orders, inventory management, and countless other applications where a partial or duplicated operation would be unacceptable. If a funds transfer debits one account but crashes before crediting another, the system has violated its contract. The discipline's central questions concern how to prevent such outcomes, how to detect them when prevention fails, and how to organize systems so that correctness does not come at an unbearable performance cost.
To understand the field, one must first see why transactions are necessary. A database is shared: many users and applications issue operations simultaneously. Without coordination, interleaved operations can produce incorrect results. Consider two transactions that both read a counter, increment it locally, and write it back. If they run concurrently, both may read the same initial value, both write the same incremented value, and one increment is lost. This is a lost update, one of several anomalies that arise from uncontrolled concurrency.
Failure poses a second, independent threat. A system can crash mid-operation, a disk can fill, a network can partition. If a transaction has written some but not all of its intended changes, the database is left in a state that never corresponded to any valid sequence of complete transactions. The system must be able to undo partial work or redo completed work to restore consistency.
The field's foundational achievement was recognizing that these two problems—concurrency and failure—can be addressed separately and then composed. The standard framework for this separation is the ACID properties, articulated in the early 1980s by Jim Gray and others. Atomicity requires that a transaction's effects are all-or-nothing. Consistency requires that a transaction transforms the database from one valid state to another, respecting declared integrity constraints. Isolation requires that concurrent transactions appear to execute serially, as if each ran alone. Durability requires that once a transaction commits, its effects survive subsequent failures.
ACID is best understood not as a single algorithm but as a specification of what correctness means. The field's history is largely the story of developing mechanisms that deliver these properties under increasingly demanding conditions, and of questioning whether all four properties are always necessary.
The dominant approach to isolation is serializability. A concurrent execution of transactions is serializable if its outcome is equivalent to some execution in which the transactions ran one after another, with no interleaving. Serializability gives users a simple mental model: they can write each transaction as if it were the only activity in the system, and the database will produce a result indistinguishable from some sequential order.
The canonical mechanism for enforcing serializability is two-phase locking (2PL). Each transaction acquires locks on data items before accessing them and releases locks only after it has entered its "shrinking phase," which begins when it releases its first lock. The two-phase rule—all lock acquisitions precede all releases—is what guarantees serializability. A transaction that reads an item holds a shared lock, preventing writers; a transaction that writes holds an exclusive lock, preventing both readers and writers. If two transactions request conflicting locks, one waits.
Two-phase locking is simple and correct, but it has a well-known cost: it can deadlock. Transaction A holds a lock on item X and wants item Y, while transaction B holds Y and wants X. Neither can proceed. Systems handle deadlock by detecting cycles in a wait-for graph and aborting one victim transaction, or by timing out waiting transactions. Aborting a transaction is acceptable because atomicity guarantees that its partial effects can be undone.
A second classical mechanism is timestamp ordering. Each transaction receives a timestamp, and the system orders conflicting operations by timestamp, effectively forcing a serial order. If a transaction attempts an operation that would violate this order, it is aborted and restarted. Timestamp ordering avoids deadlocks but can cause more aborts than locking, especially under contention. Both approaches enforce serializability, but they represent different trade-offs between waiting and restarting.
Atomicity and durability require a recovery subsystem. The standard architecture is write-ahead logging: before a transaction modifies a data item, the system records the change in a log on stable storage. The log contains enough information to undo the change (the old value) and to redo it (the new value). If the system crashes, recovery examines the log and decides, for each transaction, whether it committed before the crash. Committed transactions have their effects redone if they were not yet written to the database; uncommitted transactions have their effects undone.
This design separates concerns cleanly. Concurrency control determines which operations may interleave; recovery determines what to do after a crash. The two interact in subtle ways, however. A recovery algorithm must know which locks were held by which transactions at crash time, and it must ensure that no uncommitted transaction's effects are visible to committed ones. The standard solution is to record lock information in the log and to reacquire locks during recovery before undoing or redoing work.
A further refinement is the distinction between physical and logical logging. Physical logging records byte-level changes to pages; it is simple but can be inefficient. Logical logging records higher-level operations, such as "insert key K into index I," which is more compact but harder to undo correctly. Modern systems often use a hybrid, logging physical changes for most operations and logical changes for index structures.
Serializability is a strong guarantee, but it is not always necessary, and enforcing it can be expensive. Applications that tolerate occasional anomalies can run faster under weaker isolation levels. The SQL standard defines several: read uncommitted, read committed, repeatable read, and serializable. Each permits a different set of anomalies.
Read committed, the default in many systems, prevents dirty reads (reading uncommitted data) but allows non-repeatable reads: a transaction may read the same row twice and see different values because another transaction committed a change in between. Repeatable read prevents that anomaly but allows phantoms: a transaction may execute the same query twice and see different sets of rows because another transaction inserted or deleted rows matching the predicate. Serializability prevents all of these.
The practical significance of weaker levels is that they allow more concurrency. A transaction that only reads can proceed without blocking writers, and a transaction that writes a single row need not block readers of other rows. Many applications, such as reporting queries that tolerate slightly stale data, are well served by read committed. The field's contribution here is a precise taxonomy of anomalies and a clear statement of what each level guarantees, so that developers can make an informed trade-off.
A related development is snapshot isolation, implemented in many modern systems. Under snapshot isolation, each transaction reads from a consistent snapshot taken at its start, and writes are validated at commit time: if two transactions wrote the same item, one is aborted. Snapshot isolation prevents most common anomalies and offers excellent concurrency because readers never block writers. However, it does not guarantee serializability in all cases. The classic counterexample is write skew: two transactions each read overlapping data and write disjoint data, producing a result that no serial execution could produce. Systems that offer snapshot isolation often add serializability as an optional layer, using techniques such as serializable snapshot isolation, which detects and aborts dangerous interleavings.
When data is spread across multiple machines, transaction processing becomes substantially harder. The central problem is atomic commitment: ensuring that all participants in a transaction agree on its outcome, even if some fail or the network partitions. The classical solution is the two-phase commit protocol. A coordinator asks each participant to prepare—to write its log records and promise that it can commit. If all participants reply "prepared," the coordinator tells them all to commit; if any replies "abort" or fails to reply, the coordinator tells them all to abort.
Two-phase commit is correct but has a notorious weakness: if the coordinator fails after sending "prepare" but before sending the final decision, participants are blocked, unable to determine whether to commit or abort. They must wait for the coordinator to recover. This blocking problem is inherent to any protocol that must guarantee atomicity in the presence of arbitrary failures, a result known as the FLP impossibility theorem in a related but distinct setting. Three-phase commit can avoid blocking under certain failure assumptions but requires that the network not partition, which is not a realistic assumption in practice.
The practical response to this difficulty has been to avoid distributed transactions where possible. Many modern systems use a single leader per partition and route transactions that touch multiple partitions to a coordinator that performs two-phase commit, accepting the blocking risk. Others adopt a different model entirely: eventual consistency, in which replicas may temporarily diverge and converge later, with no atomicity guarantee across machines. This is a deliberate trade-off, accepting weaker consistency for higher availability and lower latency.
A more recent approach is the use of deterministic transactions. If the system can determine, before execution, the order in which transactions will run, it can replicate that order to all machines and execute each transaction deterministically, avoiding the need for two-phase commit. This idea, explored in research prototypes, trades the flexibility of dynamic concurrency control for a simpler replication protocol.
Transaction processing emerged from the operational needs of large-scale data processing in the 1960s and 1970s. Early systems, such as IBM's IMS, used hierarchical data models and provided limited recovery and concurrency controls. The relational model, proposed by Edgar Codd in 1970, created a cleaner foundation, and the subsequent development of SQL and commercial relational databases in the 1980s made transactions a standard feature.
The theoretical foundations were laid in the late 1970s and early 1980s. The concept of serializability was formalized, and the equivalence between two-phase locking and serializability was established. The ACID properties were articulated as a unifying framework. This period also saw the development of the write-ahead logging protocol and the ARIES recovery algorithm, which became the de facto standard for database recovery.
The 1990s and 2000s saw the rise of the web and e-commerce, which dramatically increased the scale and availability demands on transaction processing. This period also saw the emergence of the NoSQL movement, which rejected the relational model and ACID transactions in favor of simpler data models and weaker consistency guarantees. The debate between ACID and BASE (Basically Available, Soft state, Eventually consistent) was often framed as a stark opposition, but in practice it led to a more nuanced understanding of when each approach is appropriate.
The modern landscape is characterized by a spectrum of guarantees. At one end are traditional relational databases offering full ACID transactions with serializable isolation. At the other are key-value stores and document databases offering eventual consistency. In between are systems that offer snapshot isolation, read committed, or other intermediate levels. The field's enduring contribution is not a single protocol but a framework for reasoning about what guarantees are needed and what they cost.
Contemporary transaction processing research and practice focus on several fronts. One is performance at scale: how to process millions of transactions per second across many machines while maintaining strong guarantees. This has led to innovations such as in-memory databases, which keep the entire dataset in RAM and use logging primarily for durability rather than for recovery of the working set. In-memory systems can use optimistic concurrency control, which validates transactions at commit time rather than locking during execution, often achieving higher throughput under low contention.
Another direction is the integration of transactions with modern hardware. Non-volatile memory, which is nearly as fast as RAM but persists across power failures, promises to simplify recovery by allowing the database to keep its state in persistent memory. The challenge is to design logging and recovery protocols that exploit this hardware without assuming it behaves exactly like disk.
A third direction is the reconciliation of transactions with distributed data processing frameworks. Systems such as Apache Spark and Flink process large datasets in parallel across clusters, but they traditionally provide only weak consistency guarantees. Recent work has explored how to add transactional semantics to such systems, or how to express applications in terms of operations that can be executed deterministically without requiring distributed coordination.
A persistent open question is the relationship between serializability and performance. Some researchers argue that serializable transactions are achievable at acceptable cost with modern techniques, such as serializable snapshot isolation or deterministic execution. Others maintain that the overhead is fundamental and that applications must learn to tolerate weaker guarantees. The field has not resolved this debate, and it may be unresolvable in general, since the right answer depends on the application's requirements and the system's workload.
A related question concerns the semantics of transactions in systems that span multiple data models. A modern application might use a relational database for structured data, a document store for semi-structured data, and a search index for full-text queries. Coordinating a transaction across these heterogeneous systems is difficult because each has its own concurrency control and recovery mechanisms. The field has not produced a widely adopted standard for cross-system transactions, and it is an active area of research.
Transaction processing is best understood as a set of interlocking mechanisms—concurrency control, recovery, replication, and commitment—each addressing a distinct threat to correctness, and each offering a range of designs with different trade-offs. The field's intellectual core is the precise specification of correctness conditions and the proof that mechanisms enforce them. Its practical core is the engineering of systems that deliver these guarantees at acceptable performance.
The field is not organized around rival schools in the way that, say, theoretical physics has competing interpretations. Rather, it is organized around a shared problem and a toolkit of solutions that are combined and adapted. The major approaches—locking, timestamp ordering, optimistic concurrency, snapshot isolation, two-phase commit, write-ahead logging—are not competitors in the sense that one must win. They are options in a design space, and the field's progress consists of understanding the space better: characterizing the guarantees each option provides, the conditions under which each performs well, and the ways they can be composed.
What remains durable is the framework of ACID and serializability as the reference point. Even systems that deliberately weaken these guarantees define themselves in relation to them. A developer choosing eventual consistency is making a statement about what they are willing to give up, and that statement is meaningful only because the stronger guarantee is well understood. Transaction processing thus provides a common language for discussing correctness in data systems, and that language is likely to remain central as long as data systems exist.