Frequentist Statistics
Inference based on long-run frequencies, including hypothesis testing and confidence intervals.
Frequentist Statistics is the statistical inference tradition treating probability as the long-run relative frequency of events under repeated sampling, with inferential methods designed to have well-defined performance properties under repeated application — confidence intervals containing the true parameter at the stated confidence level over many repetitions, hypothesis tests with controlled Type I error rates, etc. The tradition was substantially developed in the early 20th century through R.A. Fisher (foundational maximum-likelihood estimation, p-values, design of experiments — 1912 onward, including the foundational Statistical Methods for Research Workers 1925); Jerzy Neyman and Egon Pearson (hypothesis-testing framework with Type I and Type II errors, 1928-1933); Abraham Wald (decision-theoretic foundation, 1939-1950). Core techniques include null hypothesis significance testing (NHST), confidence intervals, maximum likelihood estimation, analysis of variance, and substantial frequentist machine-learning methods. Frequentist statistics dominated 20th-century statistics, particularly in regulated scientific contexts (clinical trials, FDA, EMA, agricultural research) where its 'objective' framing — no priors required — was institutionally appealing. The framework contrasts with Bayesian Inference on the meaning of probability and the appropriate methods for inference. The 'replication crisis' beginning in the 2010s has substantially complicated frequentist statistics' standing — widespread misuse of p-values and confidence intervals (interpreting them as posterior probabilities, p-hacking, multiple-comparisons abuse) has prompted the American Statistical Association to issue substantial guidance (2016, 2019) about appropriate p-value use.
Core components
- Probability as long-run relative frequency
- Null hypothesis significance testing (NHST)
- P-values
- Confidence intervals
- Maximum likelihood estimation
- Type I and Type II error control
- Analysis of variance (ANOVA)
- Distinction from Bayesian inference on probability interpretation
- Sampling distribution as foundation
- Application across regulated scientific contexts
Primary use case
Standard statistical methodology across most scientific disciplines through 20th century; regulated clinical trials (FDA, EMA) and pharmaceutical research; foundation for substantial scientific methodology; quality control and industrial statistics; agricultural and field-experiment research; basis for most introductory statistics curricula; integration with experimental design (Fisher, Box, Cochran).
Common criticisms
- P-value misuse is substantial and contributes to the replication crisis — widespread misinterpretation of p-values as posterior probabilities, misinterpretation of statistical significance as practical importance, p-hacking and selective reporting
- American Statistical Association's 2016 'Statement on Statistical Significance and P-Values' and 2019 special issue 'Moving to a World Beyond p < 0.05' formalize the discipline's recognition of these problems
- null hypothesis significance testing has been argued to ask the wrong question (probability of data given null vs probability we actually want — probability of hypotheses given data)
- confidence intervals have well-defined frequentist meaning but are widely misinterpreted as Bayesian credible intervals
- multiple-comparisons problems and family-wise error control are technical but practically critical
- frequentist methods can produce nonsensical results in specific cases (negative variance estimates, confidence intervals containing impossible values)
- choice of test statistic and reference distribution involves analyst decisions that frequentist 'objectivity' framing obscures
- teaching frequentist statistics has produced widespread misuse that the discipline is actively trying to correct.
Lineage
- Parent of
- Regression Analysis
- Siblings
- Bayesian Inference