Abstract: Gaussian processes are a well-studied stochastic machine learning method that has
proven to be useful in many areas of application. They are flexible due to their non-
parametric approach and are able to quantify the uncertainty of their predictions. This
work describes the basic concept behind Gaussian process regression models and ex-
plains how Bayesian inference is used to incorporate both noise-free and noisy data. It
describes the optimization of hyperparameters using the maximum marginal likelihood
and explores how this method can be used to learn the parameters of linear ordinary
differential equations. Every step of the process is illustrated by comprehensive plots
with the goal of building an intuitive understanding of Gaussian processes and their
capabilities.
Keywords: Gaussian processes, probabilistic machine learning, parameter estimation,
differential equations
