- Atlas
- Computer Science
- Software Security
Software Security
Software security is the branch of cybersecurity concerned with whether a program does what it is supposed to do and nothing more—especially when someone is deliberately trying to make it misbehave.
10 frameworks, from 1965 to today · Cybersecurity · Computer Science
- Reference Monitor and Trusted Computing Base 1965-Present
- Secure Design Principles 1975-Present
- Static program analysis 1978-Present
- Formal Assurance and Evaluation Criteria 1983-Present
- Vulnerability Discovery and Exploit Research 1988-Present
- Secure Development Lifecycle 2002-Present
- Memory-Safe Language Design 2010-Present
- Fuzzing 2011-Present
- Coverage-Guided Fuzzing and Sanitizers 2013-Present
- Software Supply Chain Security 2014-Present
Overview
Software security is the branch of cybersecurity concerned with whether a program does what it is supposed to do and nothing more—especially when someone is deliberately trying to make it misbehave. Its central assumption is that the adversary has the code1. Unlike a locked door, software can be read, copied, and probed at leisure, so defenses cannot rest on secrecy of the program itself1. What is at stake is everything a program touches: memory, files, money, identities, and the physical world through the devices software now runs. The history of the field is a long argument between two instincts—build the program so it cannot be broken, or find the breaks before an attacker does—and the gradual realization that both are needed.
The Multics Inheritance: Reference Monitors and Secure Design
The field's intellectual foundations were laid in the 1960s and 1970s, when time-sharing systems forced the question of how one user's program could be prevented from reading another's data. The answer that crystallized was the reference monitor: a small, tamper-proof piece of the system that checks every access by every program to every protected object, and that must be complete (it mediates all accesses), isolated (it cannot be modified by the code it polices), and verifiable (small enough to be proven correct)2. The collection of hardware and software the monitor depends on became known as the trusted computing base, or TCB—the part of the system you must trust for security to hold3.
The Multics system, developed at MIT with Bell Labs and General Electric from the mid-1960s, was the great proving ground. Its ring-structured hardware and segmented, access-controlled file system made the reference monitor idea concrete. The U.S. Department of Defense's Trusted Computer System Evaluation Criteria, published in 1983 and known as the Orange Book, turned these ideas into a grading scheme: systems were rated from D (minimal protection) up through C and B levels to A1, which required a formal proof that the design matched a mathematical model of security. The Orange Book's insistence on formal top-level specifications and covert-channel analysis set the tone for a generation of assurance work, even as critics noted that few real systems ever reached the top grades.
Alongside the monitor came a body of secure design principles distilled from experience—most influentially by Jerome Saltzer and Michael Schroeder in their 1975 paper "The Protection of Information in Computer Systems."4 Their list included least privilege (give each part of the system only the authority it needs), economy of mechanism (keep the security-relevant code small and simple), complete mediation, open design (security must not depend on the attacker's ignorance), separation of privilege, and fail-safe defaults. These principles were not algorithms but habits of thought, and they proved durable: they reappear, often under the same names, in every later era of the field.
Formal Assurance and the Evaluation Criteria Era
The 1980s pushed formal methods hard. The Orange Book's A1 class demanded a formal model of the security policy and a proof that the implementation conformed to it. Projects such as the Kernelized Secure Operating System (KSOS) and the Honeywell SCOMP attempted to deliver such assurance, and the Gypsy Verification Environment and related tools were built to support proofs about programs. The results were sobering: formal verification of a full operating system was enormously expensive, and the gap between a verified design and a verified implementation remained wide. The evaluation criteria themselves spread internationally—the European Information Technology Security Evaluation Criteria and the Canadian criteria preceded the Common Criteria, first published in 1996 and still the basis for most government and commercial security evaluations today. The lesson the field drew was that formal assurance is valuable for small, critical components (cryptographic modules, separation kernels) but scales poorly to whole applications.
The Buffer Overflow and the Rise of Vulnerability Research
The 1988 Morris worm changed the field's center of gravity5. It exploited, among other things, a buffer overflow in the finger daemon—a program that wrote more data into a fixed-size memory buffer than it could hold, letting the excess overwrite adjacent memory and redirect execution65. The worm's author later acknowledged the bug was well known in the small community that read security advisories. What the worm demonstrated publicly was that ordinary programs, written by competent people, routinely contained exploitable flaws.
Through the 1990s, vulnerability discovery and exploit research became a discipline of its own. The buffer overflow became the canonical example: a program trusts input, the input exceeds a buffer, and the attacker gains control of the instruction pointer5. Aleph One's 1996 article "Smashing the Stack for Fun and Profit" in Phrack magazine taught a generation how stack overflows worked and how to write exploits for them5. The same era saw format-string vulnerabilities, integer overflows, and race conditions catalogued and weaponized. Mailing lists such as Bugtraq, founded in 1993, created a public record of flaws and a norm of disclosure. The defensive response was slow and partial: compiler stack canaries (StackGuard, 1997), non-executable memory, and address space layout randomization each raised the cost of an exploit without eliminating the underlying class of bug.
The Secure Development Lifecycle
The realization that vulnerabilities were being manufactured faster than they could be patched led, in the early 2000s, to the idea that security had to be built into how software is made. Microsoft, stung by a series of high-profile worms against its products, introduced its Trustworthy Computing initiative in 2002 and, from it, the Security Development Lifecycle—a set of mandatory practices: threat modeling during design, static analysis and code review during implementation, penetration testing before release, and a formal incident-response process after. The model spread: other vendors adopted variants, and the Building Security In Maturity Model (BSIMM), first published in 2009, measured how organizations actually practiced these activities. The SDL's contribution was organizational rather than technical—it made security a gate in the development process rather than an afterthought.
Static Analysis and Program Verification
Running alongside was a quieter tradition: analyzing programs without running them7. Early tools such as lint (1978) caught suspicious constructs; the security-oriented descendants were far more ambitious. Static analysis for security looks for patterns that indicate taint—data from an untrusted source flowing into a sensitive operation without sanitization. The insight, formalized in the late 1990s and 2000s, was that many vulnerability classes could be described as taint-flow problems and detected automatically. Tools such as Coverity (founded 2002) brought this to industrial codebases, and academic systems pushed toward soundness: abstract interpretation, model checking, and separation logic allowed proofs that a program never dereferences a null pointer or never leaks a file handle. The practical tension was and remains false positives: a tool that reports every possible flaw drowns its users, while a tool tuned to report only certain flaws misses real ones8.
Memory-Safe Language Design
A different answer to the buffer overflow was to remove the gun. Memory-safe languages—those that check bounds and lifetimes at runtime or compile time—had existed since the 1960s (Lisp, later Java, C#, and others), but C and C++ dominated systems programming because they gave programmers direct control of memory. The security cost of that control became impossible to ignore: studies through the 2010s attributed the majority of serious vulnerabilities in large codebases to memory-safety errors. The response gathered force in the 2010s and 2020s: Mozilla's Rust, first released in 2010, offered memory safety without garbage collection through its ownership and borrowing rules, and was adopted inside Firefox and then in operating-system kernels. Government agencies began recommending memory-safe languages for new systems work. The debate is not settled—safe languages carry performance and interoperability costs, and enormous quantities of C and C++ remain in service—but the direction of travel is clear.
Fuzzing and Sanitizers
The most consequential practical advance of the 2010s was automated bug-finding by feeding programs unexpected input9. Fuzzing—running a program on random or mutated inputs and watching for crashes—dates to the late 1980s, but it was blind and inefficient9. Coverage-guided fuzzing, introduced by the AFL tool in 2013, changed that: the fuzzer instruments the program to record which code paths each input reaches, then keeps inputs that explore new paths and mutates them further. The result was a feedback loop that found bugs at a rate earlier tools could not approach. Google's OSS-Fuzz, launched in 2016, applied this continuously to open-source projects and has since reported tens of thousands of vulnerabilities.
Fuzzing was amplified by sanitizers—compiler instrumentation that turns silent memory corruption into an immediate, diagnosable crash10. AddressSanitizer, released in 2011, detects out-of-bounds and use-after-free errors; UndefinedBehaviorSanitizer catches other classes. Together, coverage-guided fuzzing and sanitizers made it routine to find, in hours, bugs that had survived years of code review.
The Supply Chain
The final strand is the recognition that most software is assembled rather than written. A modern application pulls in hundreds of open-source libraries, each with its own dependencies. The 2014 Heartbleed bug in OpenSSL showed how a single flaw in a ubiquitous library could expose a large fraction of the internet. Later incidents—the 2021 Log4Shell vulnerability in a widely used Java logging library, and the 2020 SolarWinds compromise in which attackers inserted malicious code into a vendor's build process—made software supply chain security a first-order concern. The responses are still maturing: software bills of materials that list a product's components, signed builds and provenance attestations, and dependency scanning in continuous integration11. The field has learned that trust in software is transitive, and that the weakest link is often a package nobody reviewed.
Where the Field Stands
Today the approaches coexist and interlock. Secure development lifecycles embed threat modeling and review into engineering process; static analysis and fuzzing run continuously in build pipelines; memory-safe languages are displacing C and C++ in new systems code; formal methods are reserved for small, critical components where proof is tractable; and supply chain tooling tries to make the provenance of every dependency visible. The open questions are the old ones in new dress: how to verify that a large system is secure when no one can read all its code, how to make security usable enough that developers actually follow the practices, and how to extend assurance across a supply chain of components no single organization controls. The reference monitor's demand—small, complete, verifiable—remains the ideal against which every new technique is measured.
References
- Kerckhoffs's principle, Wikipedia
- Reference monitor, Wikipedia
- Trusted computing base, Wikipedia
- The Protection of Information in Computer Systems, Wikipedia
- Buffer overflow (introduction, History), Wikipedia
- Memory safety (History), Wikipedia
- Static program analysis, Wikipedia
- Application security (Tooling for security testing), Wikipedia
- Fuzzing (introduction, History), Wikipedia
- Code sanitizer, Wikipedia
- Software supply chain, Wikipedia
The frameworks of Software Security
Each framework is a way of working the field has used: when it held, and the question it answers.
- 1965-Present
Reference Monitor and Trusted Computing Base
How can a system check every access to protected data?
- 1975-Present
- 1978-Present
Static program analysis
Can a program's flaws be found without running it?
- 1983-Present
Formal Assurance and Evaluation Criteria
How can we prove a system meets its security policy?
- 1988-Present
Vulnerability Discovery and Exploit Research
How do attackers break into ordinary programs?
- 2002-Present
Secure Development Lifecycle
How should security be built into how software is made?
- 2010-Present
Memory-Safe Language Design
Can a language prevent memory errors by design?
- 2011-Present
- 2013-Present
Coverage-Guided Fuzzing and Sanitizers
How can a fuzzer learn which inputs reach new code?
- 2014-Present
Software Supply Chain Security
How can we trust the code we did not write?
How the ideas connect
Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.
Secure Design Principles influenced by Reference Monitor and Trusted Computing Base
Saltzer and Schroeder's 1975 secure design principles were distilled from the experience of building Multics and its reference monitor, and they formalized the monitor's requirements (complete mediation, economy of mechanism, open design) as general design habits.
Carried by Jerome Saltzer; Michael Schroeder; The Protection of Information in Computer Systems (1975).
Formal Assurance and Evaluation Criteria grew out of Reference Monitor and Trusted Computing Base
The Orange Book's A1 class turned the reference monitor's verifiability requirement into a formal evaluation criterion, demanding a formal top-level specification and a proof that the implementation matched a mathematical security model.
Carried by Trusted Computer System Evaluation Criteria (1983).
Coverage-Guided Fuzzing and Sanitizers grew out of Fuzzing
Coverage-guided fuzzing, introduced by AFL in 2013, refined blind fuzzing by instrumenting the program to record which paths each input reached and mutating inputs to maximize new coverage; sanitizers were added to turn silent memory errors into detectable crashes.
Carried by AFL (2013).
Secure Development Lifecycle influenced by Secure Design Principles
Microsoft's Security Development Lifecycle embedded Saltzer and Schroeder's principles into process gates: threat modeling operationalized least privilege and complete mediation, and mandatory code review enforced economy of mechanism.
Carried by Microsoft Trustworthy Computing (2002).
Secure Development Lifecycle influenced by Static program analysis
The Security Development Lifecycle made static analysis a mandatory implementation-phase activity, adopting the taint-flow and pattern-checking tools that had been developed in the static analysis tradition.
Carried by Microsoft Security Development Lifecycle.
Coverage-Guided Fuzzing and Sanitizers influenced by Static program analysis
Sanitizers such as AddressSanitizer borrowed instrumentation techniques from static and dynamic program analysis to insert runtime checks that detect out-of-bounds and use-after-free errors during fuzzing.
Carried by AddressSanitizer.
Secure Development Lifecycle influenced by Vulnerability Discovery and Exploit Research
The flood of buffer-overflow worms and public exploits documented by vulnerability research created the organizational pressure that led Microsoft to institute the Security Development Lifecycle in 2002.
Carried by Microsoft Trustworthy Computing (2002).
Memory-Safe Language Design influenced by Vulnerability Discovery and Exploit Research
Decades of exploit research showing that buffer overflows and use-after-free bugs dominated serious vulnerabilities motivated the design of Rust's ownership and borrowing rules as a compile-time defense against those exact bug classes.
Carried by Mozilla Rust (2010).
Fuzzing influenced by Vulnerability Discovery and Exploit Research
Fuzzing grew out of the vulnerability-discovery tradition as a way to find the crashes that exploit researchers had been finding by hand, automating the search for the malformed inputs that trigger memory corruption.
Software Supply Chain Security influenced by Secure Development Lifecycle
The Secure Development Lifecycle's process controls were extended beyond the organization's own code to its dependencies, adding component inventory, provenance verification, and vulnerability monitoring of third-party libraries.
Beyond the field
Where Software Security's ideas came from in other fields, and what they led to there: the strongest 12 of 68 links.
Secure Design Principles influenced Least Privilege and System Hardening in Systems Security
Least privilege is one of Saltzer and Schroeder's secure design principles, and system hardening practice is built on applying it along with complete mediation and fail-safe defaults. The later framework is a direct extension of those principles into operational hardening.
Carried by Jerome H. Saltzer; Michael D. Schroeder.
Static program analysis led to Scalable Bug-Finding Static Analysis in Program Analysis
Scalable bug-finding static analysis is a branch of static program analysis that trades soundness for scalability, using abstract interpretation and dataflow techniques on large codebases. Tools like Coverity and later industrial analyzers grew directly out of academic static analysis.
Coverage-Guided Fuzzing and Sanitizers grew out of Fuzzing in Software Testing
Coverage-guided fuzzing is a refinement of earlier fuzzing: it keeps the same input-generation loop but adds feedback from coverage instrumentation to steer toward new paths. The technique grew directly out of the earlier fuzzing tradition as its coverage-driven branch.
Coverage-Guided Fuzzing and Sanitizers grew out of Fuzzing in Program Analysis
This program-analysis entry on fuzzing is the parent tradition from which coverage-guided fuzzing developed, adding runtime coverage feedback to the earlier input-generation loop. The later technique is a branch of the same line of work.
Reference Monitor and Trusted Computing Base influenced Lattice-Based Information Flow in Security Theory
Lattice-based information flow built on the reference-monitor and kernel tradition, treating the flow lattice as the specification a trusted enforcement mechanism must check.
Carried by Dorothy E. Denning.
Reference Monitor and Trusted Computing Base influenced Multilevel security in Security Theory
The reference monitor concept, requiring that every access be mediated by a verifiable mechanism, shaped the design of multilevel security kernels and the Bell-LaPadula model's enforcement assumptions.
Carried by James P. Anderson.
Reference Monitor and Trusted Computing Base influenced Multilevel security and access control models in Cybersecurity
The reference monitor and trusted computing base formulation gave multilevel security its enforcement architecture, with the trusted computing base required to mediate all accesses to labeled objects.
Carried by James P. Anderson; David Elliott Bell; Leonard J. LaPadula.
Reference Monitor and Trusted Computing Base influenced Formal Verification and Evaluated Assurance in Security Theory
The reference monitor's requirement that the enforcement mechanism be verifiable made formal verification of the trusted computing base the central assurance target of the evaluation criteria.
Carried by James P. Anderson.
Reference Monitor and Trusted Computing Base influenced Least Privilege and System Hardening in Systems Security
The reference monitor and TCB concept defined the minimal trusted portion that hardening seeks to shrink and verify. Least-privilege hardening aims to reduce what must be trusted in the monitor and TCB.
Carried by James P. Anderson.
Secure Design Principles influenced Firewall in Systems Security
Saltzer and Schroeder's secure design principles, particularly complete mediation and least privilege, supplied the rationale for placing a single choke point that mediates all traffic. Firewalls are a direct application of complete mediation at the network boundary.
Carried by Jerome H. Saltzer; Michael D. Schroeder.
Secure Design Principles influenced Secure Development Lifecycle in Systems Security
Saltzer and Schroeder's secure design principles became the design guidance taught and checked within secure development lifecycles, particularly in threat modelling and design review stages.
Carried by Jerome H. Saltzer; Michael D. Schroeder.
Secure Design Principles influenced Usable Security in Systems Security
Usable security research took up Saltzer and Schroeder's secure design principles, especially least privilege and psychological acceptability, as the design guidance against which usable authentication and permission interfaces were assessed.
Carried by Jerome H. Saltzer; Michael D. Schroeder.

Software Security is a planet in Nousfarer
Its city has a tower for each of these 10 frameworks. Land, walk in, and climb one concept at a time.
Land on Software SecurityRelated fields
Fields its ideas are linked to
- Systems Security 14 links
- Software Testing 12 links
- Program Analysis 9 links
- Security Theory 7 links
- Cybersecurity 6 links
- Network Security 2 links
- Security Economics 2 links
- Formal Methods 2 links
- System Safety 2 links
- Systems Lifecycle 2 links
- Memory Hierarchy 1 link
- Systems Analysis And Design 1 link
More in Cybersecurity
- Applied Cryptography
- Cybersecurity
- Digital Forensics
- Digital Privacy
- Malware Analysis
- Network Security
- Security Economics
- Security Theory
- Systems Security
See where Software Security sits on the Map of Human Knowledge