Design of Experiments

Also known as: DoE

framework · engineering · formal-scientific

Fisher's framework for systematically planning experiments to isolate causal effects.

Design of Experiments (DoE) is the foundational framework for systematically planning and analyzing experiments to isolate causal effects of input variables (factors) on output variables (responses), substantially developed by Ronald A. Fisher at Rothamsted Experimental Station (UK) from 1919 onward. Fisher's foundational works include Statistical Methods for Research Workers (1925) and The Design of Experiments (1935), which substantially established randomization, replication, and blocking as foundational principles, and substantially developed factorial experiments enabling efficient simultaneous study of multiple factors. Fisher's central contributions include: (1) randomization — random assignment of treatments to experimental units controls for confounding variables and provides basis for statistical inference; (2) replication — multiple observations under same conditions enable estimation of experimental error; (3) blocking — grouping similar experimental units to reduce variability within blocks, substantially increasing power to detect treatment effects; (4) factorial experiments — varying multiple factors simultaneously, enabling estimation of main effects and interactions far more efficiently than one-factor-at-a-time experimentation; (5) ANOVA (Analysis of Variance) — Fisher's substantial statistical methodology for partitioning observed variation into components attributable to different factors. Subsequent development substantially extended DoE: George Box and others developed response surface methodology (1951 onward); fractional factorial designs reduced experiment size for substantial-factor studies; Plackett-Burman designs enabled screening many factors with few experiments; Genichi Taguchi (separately enriched as Robust Design) developed substantial variants for product and process robustness. DoE is foundational to substantial work in agriculture (Fisher's original context), pharmaceutical clinical trials (substantial RCT tradition), industrial quality engineering, A/B testing in technology companies, and contemporary scientific research generally. The framework has substantial empirical validation as one of the most rigorous approaches to causal inference in observational science.

Originators

Ronald A. Fisher (foundational from 1919 onward at Rothamsted Experimental Station); subsequent development by George Box (response surface methodology), Frank Yates (Yates analysis), R.L. Plackett and J.P. Burman (Plackett-Burman designs), Genichi Taguchi (Taguchi methods); ongoing development in industrial and clinical trial methodology high

Year / Decade

1919 onward (Fisher foundational at Rothamsted); 1925 (Statistical Methods for Research Workers); 1935 (The Design of Experiments); ongoing development high

Primary sources

Fisher, R.A. (1925). Statistical Methods for Research Workers, Fisher, R.A. (1935). The Design of Experiments, Box, G.E.P., Hunter, J.S. & Hunter, W.G. (multiple editions). Statistics for Experimenters, Montgomery, D.C. (multiple editions). Design and Analysis of Experiments, Plackett, R.L. & Burman, J.P. (1946). 'The Design of Optimum Multifactorial Experiments' high

Core components

Three foundational principles: randomization, replication, blocking
Factorial experiments enabling efficient multi-factor study
ANOVA (Analysis of Variance) for partitioning variation
Connection to broader statistical inference
Subsequent extensions: response surface methodology (Box), fractional factorial designs, Plackett-Burman screening, Taguchi methods
Application across agriculture, clinical trials, industrial quality, A/B testing
Substantial empirical validation as rigorous causal-inference approach
Foundation for randomized controlled trials in clinical research

Primary use case

Foundational framework in scientific experimental research broadly; basis for substantial work in agriculture, pharmaceutical clinical trials, industrial quality engineering, A/B testing, contemporary scientific research; reference framework in statistical-methods education globally; foundation for substantial Six Sigma methodology and quality engineering; integration with broader statistical inference; pedagogical foundation in statistics and engineering curricula; influence on modern technology-company experimentation (A/B testing platforms at Google, Facebook, Microsoft, Netflix, etc.); foundation for substantial regulatory frameworks requiring controlled experiments (FDA drug approval, EPA pesticide registration, etc.).

Common criticisms

DoE is among the most empirically validated frameworks in scientific methodology — substantive critiques concern application boundaries and recent developments rather than foundational framework
replication crisis in biomedical and social sciences has substantially complicated some DoE applications, raising questions about whether published experimental results are reliable (substantial concerns about p-hacking, multiple-comparisons issues, publication bias)
DoE's frequentist statistical foundation has been substantially challenged by Bayesian alternatives in some contexts
the framework's experimental-control commitments don't transfer cleanly to observational research where randomization is impossible (substantial subsequent development of quasi-experimental and causal-inference methods extends DoE to observational contexts)
industrial DoE applications can produce substantial documentation that doesn't substantively drive design decisions — compliance-style adoption problem
cross-cultural application of statistical-engineering framework varies in fidelity
commercial Six Sigma training in DoE varies substantially in analytical depth
integration with online experimentation (A/B testing) requires substantial extension — substantial issues with sequential testing, peeking at results, multiple-treatment comparisons that classical DoE didn't anticipate
modern machine-learning experimentation (multi-armed bandits, contextual bandits, reinforcement learning) substantially extends DoE in directions beyond Fisher's framework
tendency for industrial DoE to focus on screening (which factors matter) rather than substantive understanding of mechanisms
recent developments in causal inference (Pearl, others — separately covered in mathematics batch territory) substantially extend DoE's classical framework
ethical concerns in clinical trials and human-subjects research require substantial frameworks beyond DoE methodology.

Lineage

Parent of: Robust Design