Skip to main content
Back to top
Ctrl
+
K
Introduction to Scientific Machine Learning (Lecture Book)
Preface
Introduction
Lecture 1 - Introduction to Predictive Modeling
The Basics
The Uncertainty Propagation Problem
The Model Calibration Problem
Review of Probability
Lecture 2 - Basics of Probability Theory
Basics of Probability Theory
Experiment with “Randomness”
Lecture 3 - Discrete Random Variables
Discrete Random Variables
Discrete Random Variables in Python
Lecture 4 - Continuous Random Variables
Continuous Random Variables
The Uniform Distribution
The Gaussian Distribution
Lecture 5 - Collections of Random Variables
Collections of Random Variables: Theory
Practicing with Joint Probability Mass Functions
Lecture 6 - Random Vectors
Random Vectors
The Multivariate Normal - Diagonal Covariance Case
The Multivariate Normal - Full Covariance Case
The Multivariate Normal - Marginalization
The Multivariate Normal - Conditioning
Uncertainty Propagation
Lecture 7 - Basic Sampling
Pseudo-random number generators
Sampling the uniform distribution
Sampling the categorical
Sampling from continuous distributions - Inverse sampling
Lecture 8 - The Monte Carlo Method for Estimating Expectations
The Uncertainty Propagation Problem
The Monte Carlo Method for Estimating Expectations
Sampling Estimates of Variance
Lecture 9 - Monte Carlo Estimates of Various Statistics
Sampling Estimates of the Cumulative Distribution Function
Sampling Estimates of the Probability Density via Histograms
Estimating Predictive Quantiles
Uncertainty propagation through an ordinary differential equation
Lecture 10 - Quantify Uncertainty in Monte Carlo Estimates
Visualizing Monte Carlo Uncertainty
The Central Limit Theorem
Quantifying Epistemic Uncertainty in Monte Carlo Estimates
Uncertainty Propagation Through a Boundary Value Problem
Principles of Bayesian Inference
Lecture 11 - Selecting Prior Information
Selecting Prior Information
Information Entropy
The Principle of Maximum Entropy for Discrete Random Variables
The Principle of Maximum Entropy for Continuous Random Variables
Lecture 12 - Analytical Examples of Bayesian Inference
Bayesian inference
Example: Inferring the probability of a coin toss from data
Credible Intervals
Decision Making
Posterior Predictive Checking
Supervised Learning
Lecture 13 - Linear Regression via Least Squares
Linear Regression via Least Squares
Linear regression with a single variable
Polynomial Regression
The Generalized Linear Model
Measures of Predictive Accuracy
Lecture 14 - Bayesian Linear Regression
Bayesian Linear Regression
Probabilistic Interpretation of Least Squares - Estimating the Measurement Noise
Maximum a Posteriori Estimate - Avoiding Overfitting
Bayesian Linear Regression
The point-predictive Distribution - Separating Epistemic and Aleatory Uncertainty
Lecture 15 - Advanced Topics in Bayesian Linear Regression
Advanced Topics in Bayesian Linear Regression
Evidence approximation
Automatic Relevance Determination
Diagnostics for Posterior Predictive
Lecture 16 - Classification
Theoretical Background on Classification
Logistic regression with one variable (High melting explosives)
Logistic Regression with Many Features
Decision making
Diagnostics for Classifications
Multi-class Logistic Regression
Unsupervised Learning
Lecture 17 - Clustering and Density Estimation
Unsupervised Learning
Clustering using k-means
Density Estimation via Gaussian mixtures
Lecture 18 - Dimensionality Reduction
Dimensionality Reduction
Dimensionality Reduction Examples
Clustering High-dimensional Data
Density Estimation with High-dimensional Data
State Space Models
Lecture 19 - State Space Models - Filtering Basics
State Space Models - Filtering Basics
Object Tracking Example
Lecture 20 - State Space Models - Kalman Filters
State Space Models - Kalman Filters
Kalman Filter for the Object Tracking Example
Gaussian Process Regression
Lecture 21 - Gaussian Process Regression: Priors on Function Spaces
Gaussian Process Theory
Example: Priors on function spaces
Lecture 22 - Gaussian Process Regression: Conditioning on Data
Gaussian Process Regression - Theory
Gaussian Process Regression Without Noise
Gaussian Process Regression with Noise
Tuning the Hyperparameters
Multivariate Gaussian Process Regression
Lecture 23 - Bayesian Global Optimization
Bayesian Global Optimization
Maximum Mean - A Bad Information Acquisition Function
Maximum Upper Interval
Probability of Improvement
Expected Improvement
Expected Improvement - With Observation Noise
Quantifying Epistemic Uncertainty about the Solution of the Optimization problem
Neural Networks
Lecture 24 - Deep Neural Networks
Deep Neural Networks
Regression with Deep Neural Networks
Lecture 25 - Deep Neural Networks Continued
Deep Neural Networks Continued
Classification with Deep Neural Networks
Lecture 26 - Physics-informed Deep Neural Networks
Physics-informed Deep Neural Networks
Physics-informed regularization: Solving ODEs
Physics-informed regularization: Solving PDEs
Advanced Methods for Characterizing Posteriors
Lecture 27 - Sampling Methods
Sampling Methods
Probabilistic numerics using
pyro
Sampling From the Distributions With Random Walk Metropolis
The Metropolis-Hastings Algorithm
Hierarchical Bayesian Models
Lecture 28 - Variational Inference
Variational Inference
Variational Inference Examples
Homework
Homework 1
Homework 2
Homework 3
Homework 4
Homework 5
Homework 6
Homework 7
Homework 8
Bibliography
Index