Software Engineering

Software engineering is the discipline of building software that is large, long-lived, and made by teams rather than individuals.

14 frameworks, from 1968 to today · Software Engineering · Computer Science

still a working view ended · each leads to its entry below

Overview

Software engineering is the discipline of building software that is large, long-lived, and made by teams rather than individuals. Its central problem is not writing code but managing complexity: a program of a few hundred lines can be held in one person's head, while a system of millions of lines, built over years by people who come and go, cannot. The field's history is a sequence of attempts to make such systems reliable, affordable, and changeable—through process models, design methods, programming disciplines, testing regimes, and, increasingly, automation. What follows traces how practitioners came to think about that problem, from the "software crisis" of the 1960s to the continuous-delivery pipelines of today.

The software crisis and the birth of a discipline

In the 1950s and early 1960s, computing was dominated by hardware. Programs were written by the same people who designed the machines, often in assembly language, and were typically small and bespoke. As computers spread into business and government in the 1960s, projects grew enormous—the US Air Force's SAGE air-defense system, IBM's OS/360 operating system—and so did the failures. Projects ran over budget, shipped late, and were riddled with defects1. In 1968 the NATO Science Committee convened a conference in Garmisch, Germany, and the organizers deliberately coined the term "software engineering" to provoke the assembled programmers into treating their work as an engineering discipline rather than a craft. The phrase named a problem more than a solution: how to make software development predictable2.

The first answers came from structured programming. In 1968 the Dutch computer scientist Edsger Dijkstra published a short letter, "Go To Statement Considered Harmful," arguing that unrestricted jumps made programs hard to reason about. The constructive alternative, developed through the late 1960s and 1970s by Dijkstra, C. A. R. Hoare, and others, held that any program could be written using only three control structures—sequence, selection (if-then-else), and iteration (loops)—and that programs should be built top-down, decomposed into small procedures each with a single entry and exit. The payoff was comprehensibility: a reader could follow the flow of control without tracing jumps across the page. Structured programming became the orthodoxy of the 1970s and shaped languages such as Pascal, designed by Niklaus Wirth in the late 1960s and widely taught in universities.

Structured design and the waterfall

Structured programming addressed code in the small. Structured design, articulated most influentially by Larry Constantine and Ed Yourdon in the 1970s, addressed the shape of whole programs: decompose a system into modules that are loosely coupled (few dependencies between them) and highly cohesive (each doing one well-defined job). The guiding idea was that good modular structure made systems easier to understand, test, and change. Around the same time, the "waterfall" model gave development a sequence of phases—requirements, design, implementation, testing, maintenance—each completed before the next began3. The model is often traced to a 1970 paper by Winston Royce, though Royce himself warned that a purely linear process was risky and advocated iterating back between phases. The simplified waterfall nonetheless became the standard mental picture of disciplined development, and it dominated large projects and government procurement for decades.

Formal methods and the limits of proof

A more ambitious strand sought mathematical certainty. Formal methods use mathematical notation to specify what a system must do and to prove that an implementation satisfies the specification4. Hoare's 1969 paper on an axiomatic basis for computer programming introduced "Hoare logic," a system of rules for reasoning about what a program does to its variables. In the 1970s and 1980s, specification languages such as Z and VDM, and later the B method, were developed and applied to safety-critical systems—notably railway signaling, where the Paris Métro line 14, opened in 1998, was driven by software developed with the B method. Formal methods proved powerful but expensive and demanding of specialist skill, so they remained confined to domains where failure was catastrophic. Their lasting contribution was to make precision about requirements a recognized engineering value, and to seed ideas—invariants, contracts, model checking—that later tools absorbed.

Testing, verification, and the economics of defects

Because proof was rarely practical, testing became the main way to gain confidence in software. Early testing was often an afterthought, done by the programmers themselves. Through the 1970s and 1980s, testing matured into a discipline with its own techniques: black-box testing (deriving test cases from the specification without looking at the code), white-box or structural testing (exercising the code's internal paths), and systematic methods such as equivalence partitioning and boundary-value analysis. Glenford Myers's 1979 book The Art of Software Testing framed a durable principle: the purpose of testing is to find defects, not to demonstrate correctness. A parallel line of work studied where defects come from. The landmark 1975 study by IBM researchers on the OS/360 project found that a small fraction of modules contained most of the errors—an observation that motivated later work on code review, static analysis, and defect prediction. The economic argument that defects cost far more to fix the later they are found became a standard justification for early review and incremental testing.

Object orientation

By the 1980s, procedural decomposition was straining under systems whose behavior was dominated by many kinds of data that changed over time. Object-oriented development reorganized programs around objects—bundles of data and the operations on that data—rather than around functions. The idea had roots in the 1960s simulation language Simula, developed in Norway by Ole-Johan Dahl and Kristen Nygaard, which introduced classes and inheritance. It matured in Smalltalk, developed at Xerox PARC in the 1970s, and reached wide industrial use with C++ (Bjarne Stroustrup, early 1980s). The 1990s brought a profusion of methods—Grady Booch's, James Rumbaugh's object modeling technique, Ivar Jacobson's use-case approach—which were unified in the mid-1990s as the Unified Modeling Language (UML), a standard notation for describing object designs. Object orientation promised that software could mirror the concepts of its problem domain, making systems easier to extend; it became the dominant paradigm of enterprise software by the late 1990s.

Process maturity and the spiral

As projects kept failing, attention turned to the management of the development process itself. The Capability Maturity Model (CMM), developed at the Software Engineering Institute at Carnegie Mellon University from the late 1980s, rated an organization's processes on five levels, from "initial" (ad hoc) to "optimizing" (continuously improved), on the theory that predictable processes produce predictable products. It was widely adopted, especially in defense contracting, and gave organizations a vocabulary for process improvement, though critics noted that a high maturity rating did not guarantee good software. Meanwhile Barry Boehm's spiral model (1988) offered an alternative to the waterfall: development proceeds in repeated loops, each beginning with risk analysis and ending with a prototype or increment that stakeholders evaluate. The spiral model made explicit what waterfall hid—that requirements are uncertain and risk must be managed iteratively—and it anticipated the incremental, feedback-driven approaches that followed.

Components and models

Two further attempts to raise the level of reuse appeared in the 1990s and 2000s. Component-based development proposed assembling systems from independently developed, replaceable software components with well-defined interfaces, in the spirit of hardware parts. It produced standards such as Microsoft's COM and the CORBA and Enterprise JavaBeans platforms, and it shaped later service-oriented and microservice architectures, though truly plug-and-play reuse proved harder than hoped. Model-driven engineering pushed abstraction further: rather than writing code, developers would build precise models of a system and generate code from them automatically. The Object Management Group's Model Driven Architecture (early 2000s) was its most visible expression5. The approach delivered real benefits in narrow domains—code generation for embedded and telecommunications systems—but general-purpose model-driven development struggled with the gap between models and the messy realities of running systems.

Agile and continuous delivery

The 1990s also produced a reaction against heavy process. "Lightweight" methods—Extreme Programming (Kent Beck, late 1990s), Scrum, and Dynamic Systems Development Method—emphasized short iterations, close collaboration with customers, working software over documentation, and the ability to respond to change. In 2001 seventeen practitioners met at a ski resort in Snowbird, Utah, and published the Manifesto for Agile Software Development, which set these values against the plan-driven orthodoxy6. Agile spread rapidly, first in small teams and startups, then across large organizations, and reshaped how requirements, planning, and testing were done—test-driven development, in which tests are written before the code, became a signature practice.

Agile's emphasis on frequent, small releases fed directly into continuous delivery and DevOps. Continuous integration—automatically building and testing every change—and continuous delivery—keeping software always in a releasable state through automated pipelines—turned the ideal of frequent release into an engineering practice, described influentially in the 2010 book Continuous Delivery by Jez Humble and David Farley. DevOps extended this by dissolving the wall between development and operations, so that the people who build software also run it, supported by automation for deployment, monitoring, and infrastructure7. These movements are treated in depth in the neighboring DevOps and site reliability engineering subfield; within software engineering they mark the point where process, tooling, and organizational structure became inseparable8.

The present

Today's software engineering draws on all of these strands at once. Agile iteration and continuous delivery are the default rhythm of much professional work; automated testing and static analysis carry forward the testing tradition; version control, code review, and continuous integration embody the modular, review-heavy discipline that structured design began. Formal methods survive where correctness is paramount, and lightweight formal tools—type systems, contracts, property-based testing—have quietly spread into ordinary practice. Open-source development and large-scale collaboration platforms changed how teams are organized, and machine learning is now being applied to code completion, defect prediction, and test generation. The open questions are recognizably the old ones in new dress: how to keep large systems understandable as they evolve, how much process is enough, how to verify systems whose behavior emerges from many interacting parts, and how to build software whose correctness and security can be trusted at the scale on which society now depends on it.

References

  1. Software engineering (History), Wikipedia
  2. Software crisis, Wikipedia
  3. Waterfall model, Wikipedia
  4. Formal methods, Wikipedia
  5. Model-driven architecture, Wikipedia
  6. Agile software development (introduction, History), Wikipedia
  7. DevOps, Wikipedia
  8. Site reliability engineering, Wikipedia

The frameworks of Software Engineering

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1968-1985

    Structured Programming

    How should control flow in a program be organized?

    Fly to its tower in Nousfarer

  2. 1969-Present

    Formal Methods

    Can we prove a program meets its specification?

    Fly to its tower in Nousfarer

  3. 1970-Present
  4. 1970-1990

    Structured Design

    How should a program be divided into modules?

    Fly to its tower in Nousfarer

  5. 1970-1995

    Waterfall Model

    In what order should development phases proceed?

    Fly to its tower in Nousfarer

  6. 1975-Present

    Software Testing

    How do we design effective test cases?

    Fly to its tower in Nousfarer

  7. 1980-Present

    Object-oriented programming

    How should software be structured around data and behavior?

    Fly to its tower in Nousfarer

  8. 1987-2010

    Capability Maturity Model

    How can an organization's process be made predictable?

    Fly to its tower in Nousfarer

  9. 1988-2005

    Spiral Model

    How can risk be managed across repeated development cycles?

    Fly to its tower in Nousfarer

  10. 1995-2015

    Component-based software engineering

    How can software be assembled from reusable parts?

    Fly to its tower in Nousfarer

  11. 1999-Present

    Agile Software Development

    How can teams deliver software in short feedback loops?

    Fly to its tower in Nousfarer

  12. 2001-2015

    Model-Driven Engineering

    Can code be generated automatically from models?

    Fly to its tower in Nousfarer

  13. 2009-Present

    DevOps and Continuous Delivery

    How can development and operations work as one?

    Fly to its tower in Nousfarer

  14. 2010-Present

    Continuous Delivery

    How can software always be ready to release?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Structured ProgrammingFormal MethodsSoftware Testing and VerificationStructured DesignWaterfall ModelSoftware TestingObject-oriented programmingCapability Maturity ModelSpiral ModelComponent-based software engineeringAgile Software DevelopmentModel-Driven EngineeringDevOps and Continuous DeliveryContinuous Delivery

Beyond the field

Where Software Engineering's ideas came from in other fields, and what they led to there: the strongest 12 of 101 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Software Engineering is a planet in Nousfarer

Its city has a tower for each of these 14 frameworks. Land, walk in, and climb one concept at a time.

Land on Software Engineering

Related fields

Fields its ideas are linked to

More in Software Engineering

See where Software Engineering sits on the Map of Human Knowledge