Software Security

Software security is the branch of cybersecurity concerned with whether a program does what it is supposed to do and nothing more—especially when someone is deliberately trying to make it misbehave.

10 frameworks, from 1965 to today · Cybersecurity · Computer Science

still a working view ended · each leads to its entry below

Overview

Software security is the branch of cybersecurity concerned with whether a program does what it is supposed to do and nothing more—especially when someone is deliberately trying to make it misbehave. Its central assumption is that the adversary has the code1. Unlike a locked door, software can be read, copied, and probed at leisure, so defenses cannot rest on secrecy of the program itself1. What is at stake is everything a program touches: memory, files, money, identities, and the physical world through the devices software now runs. The history of the field is a long argument between two instincts—build the program so it cannot be broken, or find the breaks before an attacker does—and the gradual realization that both are needed.

The Multics Inheritance: Reference Monitors and Secure Design

The field's intellectual foundations were laid in the 1960s and 1970s, when time-sharing systems forced the question of how one user's program could be prevented from reading another's data. The answer that crystallized was the reference monitor: a small, tamper-proof piece of the system that checks every access by every program to every protected object, and that must be complete (it mediates all accesses), isolated (it cannot be modified by the code it polices), and verifiable (small enough to be proven correct)2. The collection of hardware and software the monitor depends on became known as the trusted computing base, or TCB—the part of the system you must trust for security to hold3.

The Multics system, developed at MIT with Bell Labs and General Electric from the mid-1960s, was the great proving ground. Its ring-structured hardware and segmented, access-controlled file system made the reference monitor idea concrete. The U.S. Department of Defense's Trusted Computer System Evaluation Criteria, published in 1983 and known as the Orange Book, turned these ideas into a grading scheme: systems were rated from D (minimal protection) up through C and B levels to A1, which required a formal proof that the design matched a mathematical model of security. The Orange Book's insistence on formal top-level specifications and covert-channel analysis set the tone for a generation of assurance work, even as critics noted that few real systems ever reached the top grades.

Alongside the monitor came a body of secure design principles distilled from experience—most influentially by Jerome Saltzer and Michael Schroeder in their 1975 paper "The Protection of Information in Computer Systems."4 Their list included least privilege (give each part of the system only the authority it needs), economy of mechanism (keep the security-relevant code small and simple), complete mediation, open design (security must not depend on the attacker's ignorance), separation of privilege, and fail-safe defaults. These principles were not algorithms but habits of thought, and they proved durable: they reappear, often under the same names, in every later era of the field.

Formal Assurance and the Evaluation Criteria Era

The 1980s pushed formal methods hard. The Orange Book's A1 class demanded a formal model of the security policy and a proof that the implementation conformed to it. Projects such as the Kernelized Secure Operating System (KSOS) and the Honeywell SCOMP attempted to deliver such assurance, and the Gypsy Verification Environment and related tools were built to support proofs about programs. The results were sobering: formal verification of a full operating system was enormously expensive, and the gap between a verified design and a verified implementation remained wide. The evaluation criteria themselves spread internationally—the European Information Technology Security Evaluation Criteria and the Canadian criteria preceded the Common Criteria, first published in 1996 and still the basis for most government and commercial security evaluations today. The lesson the field drew was that formal assurance is valuable for small, critical components (cryptographic modules, separation kernels) but scales poorly to whole applications.

The Buffer Overflow and the Rise of Vulnerability Research

The 1988 Morris worm changed the field's center of gravity5. It exploited, among other things, a buffer overflow in the finger daemon—a program that wrote more data into a fixed-size memory buffer than it could hold, letting the excess overwrite adjacent memory and redirect execution65. The worm's author later acknowledged the bug was well known in the small community that read security advisories. What the worm demonstrated publicly was that ordinary programs, written by competent people, routinely contained exploitable flaws.

Through the 1990s, vulnerability discovery and exploit research became a discipline of its own. The buffer overflow became the canonical example: a program trusts input, the input exceeds a buffer, and the attacker gains control of the instruction pointer5. Aleph One's 1996 article "Smashing the Stack for Fun and Profit" in Phrack magazine taught a generation how stack overflows worked and how to write exploits for them5. The same era saw format-string vulnerabilities, integer overflows, and race conditions catalogued and weaponized. Mailing lists such as Bugtraq, founded in 1993, created a public record of flaws and a norm of disclosure. The defensive response was slow and partial: compiler stack canaries (StackGuard, 1997), non-executable memory, and address space layout randomization each raised the cost of an exploit without eliminating the underlying class of bug.

The Secure Development Lifecycle

The realization that vulnerabilities were being manufactured faster than they could be patched led, in the early 2000s, to the idea that security had to be built into how software is made. Microsoft, stung by a series of high-profile worms against its products, introduced its Trustworthy Computing initiative in 2002 and, from it, the Security Development Lifecycle—a set of mandatory practices: threat modeling during design, static analysis and code review during implementation, penetration testing before release, and a formal incident-response process after. The model spread: other vendors adopted variants, and the Building Security In Maturity Model (BSIMM), first published in 2009, measured how organizations actually practiced these activities. The SDL's contribution was organizational rather than technical—it made security a gate in the development process rather than an afterthought.

Static Analysis and Program Verification

Running alongside was a quieter tradition: analyzing programs without running them7. Early tools such as lint (1978) caught suspicious constructs; the security-oriented descendants were far more ambitious. Static analysis for security looks for patterns that indicate taint—data from an untrusted source flowing into a sensitive operation without sanitization. The insight, formalized in the late 1990s and 2000s, was that many vulnerability classes could be described as taint-flow problems and detected automatically. Tools such as Coverity (founded 2002) brought this to industrial codebases, and academic systems pushed toward soundness: abstract interpretation, model checking, and separation logic allowed proofs that a program never dereferences a null pointer or never leaks a file handle. The practical tension was and remains false positives: a tool that reports every possible flaw drowns its users, while a tool tuned to report only certain flaws misses real ones8.

Memory-Safe Language Design

A different answer to the buffer overflow was to remove the gun. Memory-safe languages—those that check bounds and lifetimes at runtime or compile time—had existed since the 1960s (Lisp, later Java, C#, and others), but C and C++ dominated systems programming because they gave programmers direct control of memory. The security cost of that control became impossible to ignore: studies through the 2010s attributed the majority of serious vulnerabilities in large codebases to memory-safety errors. The response gathered force in the 2010s and 2020s: Mozilla's Rust, first released in 2010, offered memory safety without garbage collection through its ownership and borrowing rules, and was adopted inside Firefox and then in operating-system kernels. Government agencies began recommending memory-safe languages for new systems work. The debate is not settled—safe languages carry performance and interoperability costs, and enormous quantities of C and C++ remain in service—but the direction of travel is clear.

Fuzzing and Sanitizers

The most consequential practical advance of the 2010s was automated bug-finding by feeding programs unexpected input9. Fuzzing—running a program on random or mutated inputs and watching for crashes—dates to the late 1980s, but it was blind and inefficient9. Coverage-guided fuzzing, introduced by the AFL tool in 2013, changed that: the fuzzer instruments the program to record which code paths each input reaches, then keeps inputs that explore new paths and mutates them further. The result was a feedback loop that found bugs at a rate earlier tools could not approach. Google's OSS-Fuzz, launched in 2016, applied this continuously to open-source projects and has since reported tens of thousands of vulnerabilities.

Fuzzing was amplified by sanitizers—compiler instrumentation that turns silent memory corruption into an immediate, diagnosable crash10. AddressSanitizer, released in 2011, detects out-of-bounds and use-after-free errors; UndefinedBehaviorSanitizer catches other classes. Together, coverage-guided fuzzing and sanitizers made it routine to find, in hours, bugs that had survived years of code review.

The Supply Chain

The final strand is the recognition that most software is assembled rather than written. A modern application pulls in hundreds of open-source libraries, each with its own dependencies. The 2014 Heartbleed bug in OpenSSL showed how a single flaw in a ubiquitous library could expose a large fraction of the internet. Later incidents—the 2021 Log4Shell vulnerability in a widely used Java logging library, and the 2020 SolarWinds compromise in which attackers inserted malicious code into a vendor's build process—made software supply chain security a first-order concern. The responses are still maturing: software bills of materials that list a product's components, signed builds and provenance attestations, and dependency scanning in continuous integration11. The field has learned that trust in software is transitive, and that the weakest link is often a package nobody reviewed.

Where the Field Stands

Today the approaches coexist and interlock. Secure development lifecycles embed threat modeling and review into engineering process; static analysis and fuzzing run continuously in build pipelines; memory-safe languages are displacing C and C++ in new systems code; formal methods are reserved for small, critical components where proof is tractable; and supply chain tooling tries to make the provenance of every dependency visible. The open questions are the old ones in new dress: how to verify that a large system is secure when no one can read all its code, how to make security usable enough that developers actually follow the practices, and how to extend assurance across a supply chain of components no single organization controls. The reference monitor's demand—small, complete, verifiable—remains the ideal against which every new technique is measured.

References

  1. Kerckhoffs's principle, Wikipedia
  2. Reference monitor, Wikipedia
  3. Trusted computing base, Wikipedia
  4. The Protection of Information in Computer Systems, Wikipedia
  5. Buffer overflow (introduction, History), Wikipedia
  6. Memory safety (History), Wikipedia
  7. Static program analysis, Wikipedia
  8. Application security (Tooling for security testing), Wikipedia
  9. Fuzzing (introduction, History), Wikipedia
  10. Code sanitizer, Wikipedia
  11. Software supply chain, Wikipedia

The frameworks of Software Security

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1965-Present

    Reference Monitor and Trusted Computing Base

    How can a system check every access to protected data?

    Fly to its tower in Nousfarer

  2. 1975-Present

    Secure Design Principles

    What habits keep a system's design secure?

    Fly to its tower in Nousfarer

  3. 1978-Present

    Static program analysis

    Can a program's flaws be found without running it?

    Fly to its tower in Nousfarer

  4. 1983-Present

    Formal Assurance and Evaluation Criteria

    How can we prove a system meets its security policy?

    Fly to its tower in Nousfarer

  5. 1988-Present
  6. 2002-Present

    Secure Development Lifecycle

    How should security be built into how software is made?

    Fly to its tower in Nousfarer

  7. 2010-Present

    Memory-Safe Language Design

    Can a language prevent memory errors by design?

    Fly to its tower in Nousfarer

  8. 2011-Present

    Fuzzing

    Can random inputs find a program's crashes?

    Fly to its tower in Nousfarer

  9. 2013-Present

    Coverage-Guided Fuzzing and Sanitizers

    How can a fuzzer learn which inputs reach new code?

    Fly to its tower in Nousfarer

  10. 2014-Present

    Software Supply Chain Security

    How can we trust the code we did not write?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Reference Monitor and Trusted Computi…Secure Design PrinciplesStatic program analysisFormal Assurance and Evaluation Crite…Vulnerability Discovery and Exploit R…Secure Development LifecycleMemory-Safe Language DesignFuzzingCoverage-Guided Fuzzing and SanitizersSoftware Supply Chain Security

Beyond the field

Where Software Security's ideas came from in other fields, and what they led to there: the strongest 12 of 68 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Software Security is a planet in Nousfarer

Its city has a tower for each of these 10 frameworks. Land, walk in, and climb one concept at a time.

Land on Software Security

Related fields

Fields its ideas are linked to

More in Cybersecurity

See where Software Security sits on the Map of Human Knowledge