Linear Algebra

Linear algebra is the study of vector spaces and the linear maps between them—the mathematics of straight lines, flat planes, and the transformations that preserve them.

11 frameworks, from 1693 to today · Algebra · Mathematics

still a working view ended · each leads to its entry below

Overview

Linear algebra is the study of vector spaces and the linear maps between them—the mathematics of straight lines, flat planes, and the transformations that preserve them1. Its central objects are vectors (lists or arrows that can be added and scaled), matrices (rectangular arrays of numbers encoding linear transformations), and the structures built from them: bases, dimensions, eigenvalues, and decompositions. What is at stake is enormous. Linear algebra is the working language of quantum mechanics, computer graphics, statistics, optimization, signal processing, and machine learning; it is also the gateway through which most students first meet abstract algebra. Its history is a long climb from practical rule-based problem solving to a fully axiomatic theory, with each stage solving problems the previous one could not.

Solving Systems by Elimination: Ancient Roots to the Eighteenth Century

The oldest strand is the solution of simultaneous linear equations1. Babylonian scribes around 1800 BCE solved two-equation problems on clay tablets by a method we would recognize as elimination—combining equations to cancel an unknown. Chinese mathematicians in the Nine Chapters on the Mathematical Art (compiled by the first century CE) went further, arranging coefficients in counting-rod arrays and performing systematic row operations to solve systems with several unknowns, including problems with more equations than unknowns. This "fang cheng" procedure is recognizably Gaussian elimination in tabular form1.

The decisive early modern step came from Gottfried Wilhelm Leibniz1. In 1693, in letters to Guillaume de l'Hôpital, Leibniz described a "resultant"—a determinant-like quantity that vanishes exactly when a system of linear equations has a nontrivial solution. He used it to test solvability, not yet as an independent object. The idea was taken up and named: Gabriel Cramer's Introduction à l'analyse des lignes courbes algébriques (1750) gave the rule expressing each unknown of a square system as a ratio of determinants. Cramer's rule was a computational convenience, but it made determinants central. Over the following decades, mathematicians including Alexandre-Théophile Vandermonde, Pierre-Simon Laplace, and Augustin-Louis Cauchy developed determinant theory systematically: Laplace's expansion by cofactors, Cauchy's 1812 memoir establishing multiplication rules and the product theorem, and Carl Gustav Jacob Jacobi's work on functional determinants (Jacobians) in the 1830s–40s. By mid-century, determinants were a mature computational tool for elimination and for geometry.

Matrices, Vectors, and the Algebra of Transformations

Determinants were numbers attached to arrays; the arrays themselves became objects of study in the nineteenth century2. Arthur Cayley's 1858 memoir "A Memoir on the Theory of Matrices" is the landmark: Cayley defined matrices as algebraic entities, gave their addition and multiplication, proved the Cayley–Hamilton theorem (a matrix satisfies its own characteristic equation) for small cases, and treated matrices as representing linear substitutions. James Joseph Sylvester, Cayley's collaborator, coined the word "matrix" in 1850 (originally for a rectangular array from which determinants are formed) and later introduced "invariant" ideas. The concept of a vector as an abstract quantity with magnitude and direction was developed in the same era by William Rowan Hamilton (quaternions, 1843) and Hermann Grassmann, whose Ausdehnungslehre (1844, expanded 1862) introduced n-dimensional vector spaces, linear independence, and exterior algebra—ideas largely ignored in his lifetime but foundational later.

The geometric payoff came with the study of quadratic forms and their reduction to sums of squares. Cauchy and Jacobi worked on orthogonal transformations; Karl Weierstrass and others studied the classification of quadratic forms. This line led directly to the eigenvalue problem: the characteristic roots of a matrix, and the question of when a matrix can be diagonalized. Cauchy (1829) and later Weierstrass and Camille Jordan (1870) developed the theory of canonical forms, culminating in the Jordan normal form—a classification of matrices up to change of basis over the complex numbers.

Axiomatization: Linear Algebra Becomes Abstract

The nineteenth-century theory was concrete: vectors were tuples of numbers, matrices were arrays. The twentieth century made it structural1. Giuseppe Peano gave an axiomatic treatment of vector spaces in 1888, but the decisive reformulation came from the Göttingen school. David Hilbert's work on integral equations (1904–1910) treated functions as vectors in infinite-dimensional spaces, and his students—Erhard Schmidt, Erhard Schmidt's orthogonality methods, and later Stefan Banach and John von Neumann—developed the geometry of such spaces. In 1920s Göttingen, Emmy Noether and others pushed the axiomatic method: a vector space is any set with addition and scalar multiplication satisfying a short list of axioms, and linear maps are structure-preserving functions between them. This abstraction, codified in textbooks such as Bartel Leendert van der Waerden's Moderne Algebra (1930–31) and later Paul Halmos's Finite-Dimensional Vector Spaces (1942), freed the subject from coordinates. The same theorems now applied to polynomials, functions, and matrices alike1.

The abstract viewpoint made possible the spectral theory of linear operators3. Hilbert and von Neumann extended eigenvalues and eigenvectors from matrices to operators on infinite-dimensional spaces; von Neumann's 1932 Mathematische Grundlagen der Quantenmechanik made self-adjoint operators and their spectral decompositions the mathematical backbone of quantum theory. The spectral theorem—that suitable operators can be decomposed into simpler pieces—became one of the most consequential results in analysis and physics.

Fields, Modules, and the Limits of the Classical Theory

Classical linear algebra assumes scalars form a field (a number system where every nonzero element has an inverse, like the rationals or reals)145. The twentieth century asked what happens over other rings. Over the integers, or over polynomial rings, not every matrix is invertible and modules replace vector spaces. This is the domain of linear algebra over arbitrary fields and rings, developed within commutative algebra and module theory: Smith normal form over principal ideal rings, structure theorems for finitely generated modules, and the theory of linear equations over rings. The abstract framework also spawned multilinear and tensor algebra: tensors as multilinear maps, exterior and symmetric powers, and the tensor product—tools that became indispensable in differential geometry, relativity, and representation theory.

Matrix Analysis and the Computational Turn

Alongside the abstract theory, a concrete tradition of matrix analysis matured: inequalities, norms, perturbation theory, positive definite matrices, and the behavior of matrices under small changes. This tradition, associated with figures such as Hermann Minkowski, Issai Schur, and later Olga Taussky-Todd and Roger Horn, asks quantitative questions—how large can an eigenvalue move when an entry changes?—that pure structure theory does not.

The mid-twentieth century brought the digital computer, and with it numerical linear algebra16. Solving large systems by hand was impossible; solving them by machine required new algorithms with controlled rounding error6. James Wilkinson's Rounding Errors in Algebraic Processes (1963) and his work at the National Physical Laboratory established backward error analysis—showing that a computed solution is the exact solution of a slightly perturbed problem. The 1950s–70s produced the core toolkit still in use: George Dantzig's simplex method (1947) for linear programming, the QR algorithm for eigenvalues (independently by John Francis and Vera Kublanovskaya, around 1961), the singular value decomposition's computational realization, and the LINPACK and EISPACK libraries (1970s) that made reliable software possible. Gene Golub and Charles Van Loan's Matrix Computations (1983) became the standard reference.

Scale forced structural innovation. Sparse matrix methods exploit the fact that most large matrices arising from physical models—finite-element meshes, networks, discretized differential equations—have mostly zero entries7. Iterative methods such as the conjugate gradient method (Magnus Hestenes and Eduard Stiefel, 1952) and preconditioning techniques made it possible to solve systems with millions of unknowns. Direct sparse factorization methods (sparse Cholesky, multifrontal methods) developed from the 1960s onward.

Random Matrices: From Nuclear Physics to Data

A surprising twentieth-century offshoot is random matrix theory: the study of matrices whose entries are random variables, and of the statistical behavior of their eigenvalues8. Eugene Wigner introduced it in the 1950s to model the energy levels of heavy atomic nuclei, where exact Hamiltonians were unknowable but statistical regularities appeared. Wigner's semicircle law describes the limiting eigenvalue distribution of large random symmetric matrices9. The field grew through the work of Freeman Dyson, Madan Lal Mehta, and others, and later found applications far beyond physics—in number theory (Montgomery's pair correlation conjecture, linked to the Riemann zeta function), in wireless communications, and in high-dimensional statistics and machine learning, where the behavior of sample covariance matrices is central.

The Present

Linear algebra today is both a mature core curriculum and an active research frontier. The abstract theory—vector spaces, linear maps, spectral decomposition—remains the organizing framework, taught from axiomatic textbooks worldwide1. Numerical linear algebra drives scientific computing, with randomized algorithms for very large matrices and structure-exploiting methods for tensors now active areas. Random matrix theory has become a major interface between probability, statistics, and physics8. Sparse and iterative methods underpin large-scale optimization and machine learning10. Matrix analysis supplies the perturbation and stability theory that makes numerical work trustworthy11. The open questions are practical and deep: efficient algorithms for tensor decompositions, the spectral theory of non-normal and random operators, and the limits of what can be computed reliably at extreme scale.

References

  1. Linear algebra (introduction, Fluid mechanics, fluid dynamics, and thermal energy systems, Vector spaces, Linear systems, History), Wikipedia
  2. Determinant (History), Wikipedia
  3. Spectral theory, Wikipedia
  4. Vector space, Wikipedia
  5. Field (mathematics), Wikipedia
  6. Numerical linear algebra, Wikipedia
  7. Sparse matrix, Wikipedia
  8. Random matrix, Wikipedia
  9. Wigner semicircle distribution, Wikipedia
  10. Conjugate gradient method, Wikipedia
  11. Condition number, Wikipedia

The frameworks of Linear Algebra

Each framework is a way of working the field has used: when it held, and the question it answers.

  1. 1693-1900

    Elimination and Determinant Theory

    How can we solve systems of linear equations?

    Fly to its tower in Nousfarer

  2. 1829-Present
  3. 1843-1930

    Matrix and Vector Algebra

    How do arrays and arrows combine algebraically?

    Fly to its tower in Nousfarer

  4. 1900-Present

    Matrix Analysis

    How do matrices behave under small changes?

    Fly to its tower in Nousfarer

  5. 1904-Present

    Spectral theory

    How can operators be decomposed into simpler parts?

    Fly to its tower in Nousfarer

  6. 1920-Present

    Module

    What replaces vector spaces when scalars form a ring?

    Fly to its tower in Nousfarer

  7. 1920-Present
  8. 1930-Present

    Multilinear algebra

    How do multilinear maps and tensors work?

    Fly to its tower in Nousfarer

  9. 1950-Present

    Numerical Linear Algebra

    How can computers solve linear systems accurately?

    Fly to its tower in Nousfarer

  10. 1952-Present

    Sparse and Iterative Methods

    How can we solve enormous systems with mostly zero entries?

    Fly to its tower in Nousfarer

  11. 1955-Present

    Random matrix

    What are the statistics of random matrices' eigenvalues?

    Fly to its tower in Nousfarer

How the ideas connect

Each arrow runs from an earlier idea to the later one it shaped. The list below says what passed between them.

Elimination and Determinant TheoryCanonical Forms and Eigenvalue TheoryMatrix and Vector AlgebraMatrix AnalysisSpectral theoryModuleVector spaceMultilinear algebraNumerical Linear AlgebraSparse and Iterative MethodsRandom matrix

Beyond the field

Where Linear Algebra's ideas came from in other fields, and what they led to there: the strongest 12 of 89 links.

A city in Nousfarer: a red-carpeted stair climbing terraces to a crowned tower on an acropolis, more towers round it, forest and snowy mountains beyond.
A city in Nousfarer, the game played in the browser.

Linear Algebra is a planet in Nousfarer

Its city has a tower for each of these 11 frameworks. Land, walk in, and climb one concept at a time.

Land on Linear Algebra

Related fields

Fields its ideas are linked to

More in Algebra

See where Linear Algebra sits on the Map of Human Knowledge