- Atlas
- Computer Science
- Software Testing
Software Testing
Software testing is the work of finding out whether a program does what it is supposed to do—and, just as importantly, failing to find out is normal, because exhaustively checking a real system is impossible.
16 frameworks, from 1945 to today · Software Engineering · Computer Science
- Debugging-oriented testing 1945-1968
- Formal verification 1969-Present
- Black-box testing 1970-Present
- Defect-finding testing 1970-Present
- White-box testing 1970-Present
- Formal code inspection 1972-1995
- Regression testing 1975-Present
- V-model 1980-2005
- Fuzzing 1988-Present
- Exploratory testing 1990-Present
- Load and performance testing 1990-Present
- Security testing 1990-Present
- Test-driven development 1999-Present
- Property-based testing 2000-Present
- Whole-team testing in Agile and continuous integration 2001-Present
- Chaos engineering 2011-Present
Overview
Software testing is the work of finding out whether a program does what it is supposed to do—and, just as importantly, failing to find out is normal, because exhaustively checking a real system is impossible1. A program with a dozen inputs that each take a thousand values has more paths through it than there are atoms in the observable universe; the tester's craft is choosing the few checks that will expose the most likely faults1. The history of the field is a sequence of answers to that problem: how to sample intelligently, who should do the sampling, when in the build to do it, and what counts as evidence that software is good enough. What began as debugging by the programmer who wrote the code became a separate discipline with its own levels, its own mathematics, its own automation, and eventually its own culture of deliberate disruption.
Debugging by the maker: testing as an afterthought
In the earliest commercial computing, through the 1950s, "testing" meant what a programmer did after writing code: run it, watch it fail, and fix it. The word "bug" was already in engineering use, and the story of a moth taped into a 1947 Harvard Mark II logbook—now in the Smithsonian—captures the era's spirit: faults were physical and local. There was no test plan, no separate tester, and no distinction between checking that the program met its specification and repairing it when it did not.
The 1950s and 1960s professionalized the surrounding work. The 1956 and 1958 conferences on the "preparation of programs" for computers (the second organized under the auspices of the National Physical Laboratory in Britain) gathered practitioners to discuss how to build reliable software, and by the mid-1960s the idea that a program should be checked against a written specification, by someone other than its author, began to appear. Still, the dominant view remained that testing was a phase of "making it work."
The 1970s: testing becomes a discipline with a purpose
The turning point was a change of purpose. Instead of demonstrating that software works, testing came to be understood as the process of finding defects1. Glenford Myers's The Art of Software Testing (1979) made the case bluntly: a successful test is one that finds a fault. That reframing gave the field its economics—if you cannot test everything, aim your tests where failures are likely.
Two complementary ways of choosing tests took shape in this decade. Black-box testing treats the program as opaque and derives tests from its specification: give inputs, compare outputs, never look inside12. The classic techniques—equivalence partitioning (group inputs that should be handled alike and test one from each group) and boundary-value analysis (test the edges, where faults cluster)—were systematized in this period. White-box testing does the opposite, using the code's structure: statement coverage, branch coverage, and later path coverage measure how much of the program a test suite actually exercises. The two approaches became the field's basic coordinates, and they remain so.
Formal inspection arrived as a third, complementary route. In the early 1970s Michael Fagan, working at IBM, developed a structured, meeting-based process for reading code and design documents line by line against checklists—formal code inspection, sometimes called Fagan inspection. It was striking because it found defects before anything ran; studies at IBM and elsewhere through the 1970s and 1980s reported that inspections caught large shares of defects far more cheaply than execution testing. This established a durable split in the field between static techniques (reading and analyzing without running) and dynamic ones (executing).
Structured levels, the waterfall, and the V
As large projects in the 1960s and 1970s overran budgets—the famous "software crisis"—process models tried to impose order. The waterfall model, usually credited to Winston Royce's 1970 paper (though Royce himself advocated iterating), treated testing as a late phase following analysis, design, and coding. The consequence was costly: defects introduced early, in requirements, were found only at the end.
The V-model, developed in the 1980s in the context of German and other European defense and public-sector projects, answered this by pairing each development stage with a matching test level. Requirements analysis corresponds to acceptance testing (does the system satisfy the customer?), system design to system testing (does the whole work together?), detailed design to integration testing (do the modules talk to each other correctly?), and coding to unit testing (does each piece behave?). The point was not just symmetry: it was that test planning should begin while requirements are being written, so that each stage's tests are designed against the same specification the stage was built from. This vocabulary—unit, integration, system, acceptance—became the standard ladder of testing levels, still taught and still used, even where teams no longer follow the V literally.
Formal methods and the limits of demonstration
A parallel tradition tried to prove programs correct rather than test them. Formal verification uses mathematical logic to show that a program satisfies a specification for all inputs. The landmark result is Tony Hoare's 1969 paper on an axiomatic basis for computer programming, which introduced the idea that a program's meaning could be captured by assertions about its state, and that these could be reasoned about compositionally (what later became known as Hoare logic). Around it grew a research program in which specifications were written in formal languages and programs checked against them, with early systems for program proving developed at several centers in the 1970s.
Formal verification's practical reach expanded with model checking, developed in the early 1980s by Edmund Clarke and E. Allen Emerson in the United States and independently by Joseph Sifakis in France, which automatically explores all the states of a finite system. In the 1990s and after, these techniques moved from research into industry for hardware and safety-critical software, and into tools that check software models. They force precise specifications and can be conclusive, but they are costly and hard to scale to whole systems—which is why testing remains the everyday instrument even where formal methods are used.
Regression, automation, and the cost of change
As software became long-lived, a new problem appeared: a change that fixes one thing breaks another. Regression testing—re-running tests to confirm that new work has not damaged old behavior—became central. Doing it by hand does not scale, so from the 1970s onward the field built test automation: scripts that drive the software and compare results with expected ones. Early tools ran batch jobs; by the 1990s commercial "capture and replay" tools recorded keystrokes and mouse actions against graphical interfaces and replayed them. The persistent difficulty was maintenance: an automated test suite is itself software, and it breaks when the system under test changes.
Security and performance: testing beyond correctness
Two important offshoots tested properties other than functional correctness. Security testing grew with networked systems in the 1980s and 1990s: it asks whether the software resists attack, covering input handling, authentication, access control, and injection of malicious data, and it broadened from manual review into a toolkit of static analyzers, dynamic scanners, and penetration testers. Load and performance testing asks whether the system stays fast and stable under real traffic; practitioners simulate concurrent users and heavy data and measure response times and resource use, a practice that intensified in the web era.
A distinctive descendant is fuzzing. The idea is to throw vast quantities of semi-random or malformed input at a program and watch for crashes3. It is often traced to early 1980s work at the University of Wisconsin by Barton Miller and colleagues, who found that ordinary Unix utilities crashed readily on random input. Fuzzing matured in the 2000s and 2010s into a standard security and reliability tool, notably with coverage-guided fuzzers such as American Fuzzy Lop (AFL), released by Michał Zalewski in 2013, which use feedback about which code paths the input reached to steer the next inputs.
The 1990s: thinking about tests and testing by exploration
Two ideas from this decade reshaped everyday practice. First, testing theory matured around test design: the recognition that the choice of test cases is an engineering decision with measurable returns, and that exploratory testing—testing unscripted, learning about the system while designing and running tests in quick cycles—is a skilled activity rather than a sloppy one4. Cem Kaner popularized the term in the 1980s and 1990s in the context of commercial software, and it gained a formal framework in the work of Kaner, James Bach, and Elisabeth Hendrickson. Exploratory testing claimed that rigid scripts miss discovery, and that the tester's expertise in noticing anomalies is the real instrument.
Second, the language-level movement for test-driven development began5. Kent Beck, working with the Smalltalk testing framework SUnit in the 1990s, generalized it into the xUnit family of test frameworks, and in 2002 articulated TDD in Test-Driven Development: By Example. The discipline inverts the order of work: write a failing test, write the minimal code that passes it, then refactor5. TDD's supporters argued it made design simpler and caught errors early; critics pointed to maintenance costs and inconclusive evidence about defect rates. Whatever the verdict, TDD established automated unit tests as a normal part of a programmer's work.
Agile and continuous: testing every day
With the rise of Agile methods after the 2001 "Manifesto for Agile Software Development," testing moved from a phase to a continuous activity. Extreme Programming (late 1990s) pushed TDD and on-site customer involvement; the broader whole-team testing view held that quality was the team's responsibility, not a separate department's—testers help define acceptance criteria and evolve tests continuously56. Two enabling practices made this workable. Continuous integration, popularized in the early 2000s (with influential writing by Martin Fowler), has developers merge their changes frequently, each merge triggering an automated build and test run, so breakage is found within hours rather than weeks. And test frameworks grew into ecosystems (JUnit, later xUnit variants in nearly every language) with advanced libraries like Mockito (2008) for replacing dependencies in unit tests.
Property-based testing arrived in this current. Instead of specifying exact input-output examples, the tester states properties that must hold for all input—for instance, that reversing a list twice returns the original—and the tool generates many random inputs to try to falsify them1. QuickCheck, released as a Haskell library by Koen Claessen and John Hughes in 2000, is the origin; property-based testing is now widely available in other languages. It extends the older idea of generating test data to the level of assertions.
Chaos engineering: testing the system in production
As systems became distributed, testing individual components was no longer enough: failures emerge in the interactions of many machines, networks, and services. Chaos engineering was the response, articulated publicly in the mid-2010s by engineers at Netflix, who built the tool Chaos Monkey (released 2011–2012) to randomly terminate instances in their production cloud and confirm that the system kept serving. The practice deliberately injects failures in controlled experiments to discover weaknesses before they cause outages. It extends the field's oldest idea—find failure on purpose—from a single program to an entire running system.
Where the field stands
Today's practice descends directly from these layers. Teams run unit tests in continuous integration, design integration and system tests with black-box and white-box techniques, watch security and performance continuously, use fuzzing and chaos experiments against hostile and failure-prone environments, and draw on exploratory testing to find what scripts miss1. Automation frameworks, cloud test farms, and static analyzers are standard. The open questions are the ones the history keeps returning to: how much testing is enough, how to test systems whose behavior depends on machine learning or on combinations too numerous to enumerate, how to keep automated suites maintainable, and what evidentiary weight tests should carry when correctness must be argued rather than demonstrated.
References
- Software testing (introduction, Code coverage, History, Black-box testing, Property testing), Wikipedia
- Black-box testing, Wikipedia
- Fuzzing, Wikipedia
- Exploratory testing, Wikipedia
- Test-driven development, Wikipedia
- Agile testing, Wikipedia
The frameworks of Software Testing
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1945-1968
Debugging-oriented testing
How do I find and fix the fault when the program fails?
- 1969-Present
- 1970-Present
- 1970-Present
Defect-finding testing
How do we choose tests that expose the most faults?
- 1970-Present
- 1972-1995
Formal code inspection
How do we find defects by reading code before running it?
- 1975-Present
- 1980-2005
- 1988-Present
- 1990-Present
- 1990-Present
Load and performance testing
Does the system stay fast and stable under real traffic?
- 1990-Present
- 1999-Present
- 2000-Present
Property-based testing
What properties should hold for every generated input?
- 2001-Present
Whole-team testing in Agile and continuous integration
How does the whole team keep the suite green as they integrate?
- 2011-Present
Chaos engineering
What happens when we deliberately break the running system?
How the ideas connect
Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.
Defect-finding testing replaced Debugging-oriented testing
The debugging view treated testing as making the program work; Myers's 1979 book reframed a successful test as one that finds a defect, replacing the earlier purpose. This gave the field its economics of aiming tests where failures are likely.
Carried by Glenford Myers; The Art of Software Testing (1979).
Formal verification reacted against Debugging-oriented testing
Formal verification arose as a mathematical alternative to the run-and-fix debugging practice of the 1950s and 1960s, aiming to prove correctness rather than expose faults by execution. Hoare's 1969 axiomatic basis framed program meaning as assertions, defining itself against the trial-and-error approach.
Carried by C. A. R. Hoare; An Axiomatic Basis for Computer Programming (1969).
Security testing grew out of Black-box testing
Security testing extended black-box input testing to malicious data, asking whether the software resists attack through input handling, authentication and access control. It broadened from manual review into static analyzers, dynamic scanners and penetration testers.
Exploratory testing grew out of Defect-finding testing
Exploratory testing kept the defect-finding purpose but made test design and execution simultaneous, learning about the system while testing in quick cycles. Kaner popularized the term as a skilled activity rather than a sloppy one.
Carried by Cem Kaner.
Test-driven development grew out of Defect-finding testing
Test-driven development kept the defect-finding aim but moved it into design: each new behavior starts with a failing test that defines what the code must do. It grew out of the practice of aiming tests where faults are likely.
Carried by Kent Beck.
Whole-team testing in Agile and continuous integration influenced by Regression testing
Continuous integration depends on regression testing: every change re-runs the automated suite to confirm old behavior is intact. The practice of re-running tests after change became the backbone of the whole-team testing culture.
Property-based testing influenced by Fuzzing
Property-based testing generalized the fuzzing idea of generating many inputs automatically, but checked them against stated properties rather than watching for crashes. It carried the insight that random generation can expose faults a hand-written case misses.
Whole-team testing in Agile and continuous integration grew out of Test-driven development
Whole-team testing in Agile and continuous integration grew out of test-driven development's practice of writing tests first and running them constantly. The automated suite became the team's shared safety net in continuous integration.
V-model influenced by Black-box testing
The V-model paired each development stage with a matching test level, including acceptance and system testing derived from the specification. This carried the black-box idea of testing against a written specification into the structure of the process.
V-model influenced by White-box testing
The V-model's unit and integration levels, testing each piece and how modules talk, drew on the white-box concern with code structure. Test planning began while requirements were written, pairing stages with levels.
Beyond the field
Where Software Testing's ideas came from in other fields, and what they led to there: the strongest 12 of 54 links.
Formal verification led to Deductive Software Verification Tools in Formal Methods
Deductive software verification tools are the mechanised continuation of the formal verification tradition, turning program logics and proof rules into working tool support.
Formal verification influenced Formal Verification and Evaluated Assurance in Security Theory
Formal verification techniques, program proving and specification methods developed from the late 1960s, were adopted as the highest assurance evidence in the security evaluation criteria.
Carried by John Rushby.
Formal verification led to SMT-Based Deductive Verification in Formal Methods
SMT-based deductive verification is a continuation of formal verification, replacing or supplementing interactive proof with satisfiability-modulo-theories solvers.
Fuzzing led to Coverage-Guided Fuzzing and Sanitizers in Software Security
Coverage-guided fuzzing is a refinement of earlier fuzzing: it keeps the same input-generation loop but adds feedback from coverage instrumentation to steer toward new paths. The technique grew directly out of the earlier fuzzing tradition as its coverage-driven branch.
Test-driven development led to Specification by Example in Requirements Engineering
Specification by Example grew directly out of test-driven development and its acceptance-test variant; Gojko Adzic's 2011 book synthesized practices from teams using executable acceptance criteria in the TDD tradition, notably the Cucumber/FitNesse lineage.
Carried by Gojko Adzic; Specification by Example (2011).
Debugging-oriented testing influenced Structured test programme verification in Systems Verification
Debugging-oriented testing, with its test cases aimed at exposing faults, is the immediate predecessor of the structured test programme. Structured verification organised such tests into a planned, documented programme.
Debugging-oriented testing influenced Integration testing in Systems Integration
Early debugging-oriented testing established the practice of exercising assembled code to expose faults at component boundaries. Integration testing took up that fault-finding aim and applied it to the interfaces between combined modules.
Debugging-oriented testing influenced Software Testing in Software Engineering
Software testing retained debugging-oriented testing's practice of constructing test cases to reveal defects, while separating the act of testing from the act of debugging that follows it.
Carried by Glenford Myers.
Debugging-oriented testing influenced Software Testing and Verification in Software Engineering
Software testing from 1970 onward grew out of debugging-oriented testing: the earlier practice of writing tests to locate defects was retained but reframed as testing to demonstrate correctness, with debugging separated from testing.
Carried by Glenford Myers.
Formal verification influenced Standards-based safety certification in Systems Verification
Formal verification techniques became accepted certification evidence for high-integrity software, as in standards that permit formal methods in place of exhaustive testing. Certification schemes incorporated proof obligations from the formal verification tradition.
Formal verification reacted against Test-and-fix verification in Systems Verification
Formal verification defined itself against test-and-fix practice, arguing that testing can show the presence of errors but never their absence. The field's founding rhetoric was a rejection of debugging-by-testing as a basis for confidence.
Carried by Edsger W. Dijkstra.
Formal verification rival of Software Testing in Software Engineering
Software testing and formal verification competed from the late 1960s onward over how to assure correctness; testing advocates argued that verification of real programs was impractical, while formalists argued testing could never prove absence of defects.
Carried by Edsger W. Dijkstra; Glenford Myers.

Software Testing is a planet in Nousfarer
Its city has a tower for each of these 16 frameworks. Land, walk in, and climb one concept at a time.
Land on Software TestingRelated fields
Fields its ideas are linked to
- Software Security 12 links
- Software Engineering 10 links
- Systems Verification 7 links
- Software Evolution 4 links
- Program Analysis 3 links
- Quality Engineering 2 links
- Systems Simulation 2 links
- Security Theory 2 links
- Formal Methods 2 links
- Reliability Engineering 1 link
- Quality Management 1 link
- Modal Logic 1 link
More in Software Engineering
- Computing Education
- Design Patterns
- Devops And Sre
- Formal Methods
- Open Source Software
- Requirements Engineering
- Software Architecture
- Software Engineering
- Software Evolution
- Software Process Models
See where Software Testing sits on the Map of Human Knowledge