Bayesian Networks

framework · mathematics · formal-scientific

Directed acyclic graphs encoding probabilistic dependencies among variables.

Bayesian Networks (also Belief Networks, Bayes Nets) are graphical models that encode probabilistic dependencies among random variables using directed acyclic graphs (DAGs), where nodes represent variables and directed edges represent direct probabilistic dependencies, with each node associated with a conditional probability distribution given its parent nodes. The framework was substantially developed by Judea Pearl and presented systematically in his 1988 Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, which was substantially influential in establishing graphical models as the dominant framework for representing uncertainty in artificial intelligence. The framework's central commitments include: the joint probability distribution factorizes according to the DAG structure (P(X1,...,Xn) = ∏ P(Xi | parents(Xi))); conditional independence relationships are read directly from the graph via d-separation; inference (computing posterior probabilities given evidence) can be performed using exact algorithms (variable elimination, junction tree) or approximate algorithms (loopy belief propagation, variational methods, MCMC); learning Bayesian networks from data involves both structure learning (which DAG?) and parameter learning (what conditional distributions?). Bayesian networks were Pearl's principal contribution before he turned to causal inference (separately enriched), where causal Bayesian networks add the substantive interpretation that directed edges represent causal influences. Bayesian networks have been substantially applied to medical diagnosis (Pathfinder for lymph node pathology), genetics (gene regulatory networks), natural language processing, fault diagnosis, and substantial commercial applications.

Originators

Judea Pearl (foundational); intellectual antecedents in probabilistic reasoning literature; subsequent figures including Daphne Koller, Nir Friedman, Stuart Russell high

Year / Decade

Mid-1980s development; 1988 (Pearl's Probabilistic Reasoning); ongoing development high

Primary sources

Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Koller, D. & Friedman, N. (2009). Probabilistic Graphical Models: Principles and Techniques, Russell, S. & Norvig, P. (multiple editions). Artificial Intelligence: A Modern Approach high

Core components

Directed acyclic graph (DAG) representation
Nodes as random variables
Edges as direct probabilistic dependencies
Conditional probability distributions at each node given parents
Joint distribution factorization
Conditional independence via d-separation
Inference algorithms (variable elimination, junction tree, loopy belief propagation, variational methods, MCMC)
Structure learning and parameter learning from data
Causal Bayesian networks (causal interpretation of edges)
Connection to broader probabilistic graphical models (including undirected Markov random fields)

Primary use case

Probabilistic reasoning under uncertainty in AI; medical diagnosis (Pathfinder, Quick Medical Reference); genetic and biological network analysis; natural language processing (parts-of-speech tagging, named entity recognition); fault diagnosis in engineering; foundation for causal inference (Pearl's subsequent work); substantial commercial applications including Microsoft's Office Assistant; pedagogical reference in AI and machine-learning curricula; foundation for many modern probabilistic programming frameworks.

Common criticisms

Structure learning is computationally hard — finding the optimal Bayesian network from data is NP-hard, requiring heuristics or restrictions
specifying conditional probability tables manually requires substantial expert knowledge that may not be available or reliable
high-dimensional problems face curse-of-dimensionality concerns in conditional probability estimation
the strict conditional-independence assumptions encoded by the DAG may not hold in real systems
modern deep learning methods often outperform classical Bayesian networks on many tasks where flexible function approximation matters more than explicit probabilistic reasoning
commercial Bayesian network software has produced compliance-style adoption in some industries
tension between Bayesian networks as interpretable models and contemporary preference for deep-learning representations
integration with continuous variables (vs originally-discrete focus) involves additional complexity
the framework's substantial early-AI prominence has been somewhat eclipsed by deep learning, though Bayesian network methods remain important in causal inference and specific domains.

Lineage

Parent of: Causal Inference
Child of: Bayesian Inference
Siblings: Causal Inference, Bayesian Inference
Derived from: Bayesian Inference