Bayesian Inference

Bayesian Inference is a method of Statistical Inference that uses Bayes' Theorem to update the probability of a hypothesis as evidence becomes available. In Machine Learning, it provides a framework for reasoning under uncertainty by combining a Prior Probability Distribution, representing beliefs before observing the data, with a Likelihood Function based on observed data to obtain a Posterior Probability Distribution. This approach contrasts with Frequentist Inference, which conventionally treats model parameters as fixed but unknown quantities: Bayesian models represent Epistemic Uncertainty about parameters using Probability Distributions, without necessarily asserting that the underlying parameters are intrinsically random. It underpins inference in Bayesian Networks and Gaussian Processes and is supported by Probabilistic Programming tools such as PyMC and TensorFlow Probability. Because integration over high-dimensional spaces can make exact inference computationally demanding, practical applications often rely on Markov Chain Monte Carlo (MCMC) methods to sample from the posterior or Variational Inference to construct an approximation to it.