Skip to main content
Back to top
Ctrl
+
K
Advanced Scientific Machine Learning
Modern Machine Learning Software
Functional Programming
A Primer on Functional Programming
Just in Time Compilation
Vectorization
Pseudo Random Numbers without Side Effects
Type Systems, Pytrees, and Models
Type Systems and why We Care
Python Type Annotations
Haskell Type System
Pytrees to represent model parameters
Differentiable Programming
Numerical Differentiation
Symbolic Differentiation
Automatic Differentiation
Autograd with JAX
Optimization for Scientific Machine Learning
Basics of Optimization Problems
Gradient Descent
Gradient Descent with Momentum
Optax
- Optimizers in JAX
Stochastic Gradient Descent
Optimization Algorithms with Adaptive Learning Rates
Second-order Methods for Optimization
Initialization of Neural Network Parameters
Training a neural network on the GPU
Uncertainty Propagation through Scientific Models
Sensitivity Analysis of ODEs and PDEs
Local Sensitivity Analysis for Ordinary Differential Equations
Differentiating the Solution of Ordinary Differential Equations
Example: The Duffing Oscillator
Example: Lorenz System
Beyond Local Sensitivity Analysis: The Fokker-Planck Equation
Latin Hypercube Designs
Sobol’s Sequence
Global Sensitivity Analysis
Uncertainty Propagation using Polynomial Chaos
Required Functional Analysis
Symbolic Construction of Polynomial Chaos for Uniform Random Variables
Symbolic Construction of Polynomial Chaos for Gaussian Random Variables
Numerical Estimation of Orthogonal Polynomials
Using Polynomial Chaos to Propagate Uncertainty through an ODE
Polynomial Chaos in Many Dimensions
Uncertainty Propagation in Dynamical Systems
Limitations of Polynomial Chaos
Surrogates Models
Basic Elements of Surrogate Modeling
Example of a Neural Network Surrogate
Example of Gaussian Process Surrogate
Example – Gaussian Process Regression with Large Datasets
Multi-fidelity Surrogates
Multifidelity modeling
Multifidelity Gaussian process surrogates
Active Learning
Active Learning Basics
Uncertainty Sampling Example
Embedding Symmetries in Surrogate Models
Enforcing Symmetries in Neural Networks
Euclidean Neural Networks
High-dimensional Uncertainty Propagation
Functional Inputs to Scientific Models
Functional Inputs to Scientific Models
Singular Value Decomposition
Connection Between SVD and Principal Component Analysis (PCA)
The Karhunen-Loève Expansion of a Gaussian Process
Example – Surrogate for stochastic heat equation
Example – Surrogate for stochastic heat equation with principal component analysis
Operator Learning
Operator Learning
Example – DeepONets
Example - Fourier Neural Operators
Inverse Problems in Deterministic Scientific Models
Basics of Inverse Problems
The Classical Formulation of Inverse Problems
Example – The catalysis Problem using a Classical Approach
Bayesian formulation to inverse problems
The Laplace approximation
Example - The Catalysis Problem using the Laplace Approximation
Sampling from Posteriors
Basics of MCMC
Metropolis-Hastings with Blackjax
Hamiltonian Monte Carlo with Blackjax
No-U-Turn Sampler with Blackjax
Variational Inference
Basics of Variational Inference
Example – The catalysis problem using variational inference
Example - 3D particle position reconstrution from images
Hierarchical Bayesian Modeling
Hierarchical Bayesian modeling
Population uncertainty
Improving posterior geometry
Amortized variational inference
Deterministic, Finite-dimensional, Dynamical Systems
Basics
Structural Identifiability of a Harmonic Oscillator
Example - A dynamical system with multiple observed trajectories
PDE-constrained inverse problems
Calibration of partial differential equations
Inferring thermal conductivity
Inferring the location of a contaminant
Data-driven Modeling of Dynamical Systems
Sparsity Promoting Regularization (L1-regularization or Lasso regression)
Sparse Identification of Nonlinear Dynamics
SINDy - Example 2: Lorenz system
Neural ODEs
Physics-informed Neural Networks (PINNs)
Basics of Physics-Informed Neural Networks
Physics-Informed Neural Networks (PINNs) - Forward Problems
Spectral Bias of Neural Networks
Energy Functionals
PINNS for Parametric Studies
Solving Parametric Problems using Physics-informed Neural Networks
Example - Physics-informed Neural Operators
PINNS for Inverse Problems
PINNs - Example of inverse problem
Example - Bayesian PINNs
Inverse Problems in Stochastic Scientific Models
Stochastic Differential Equations
Stochastic differential equations
Example - Brownian Motion
Example - Stochastic Exponential Growth
Example - Ornstein-Uhlenbeck Process
Filtering and Smoothing
Particle Filters
Example - Particle filters
Example - Particle smoother
State Estimation and Parameter Calibration
Parameter Estimation in Stochastic Differential Equations
Example - System identification with expectation maximization
Bayesian Inference in State-space Models
Example - System identification with particle MCMC
Homework Problems
Homework 1 - TEMPLATE - DO NOT DO IT YET
Homework 2 - TEMPLATE - DO NOT DO IT YET
Homework 3 - TEMPLATE - DO NOT DO IT YET
Homework 4 - TEMPLATE - DO NOT DO IT YET
Homework 5 - TEMPLATE - DO NOT DO IT YET
Homework 6 - TEMPLATE - DO NOT DO IT YET
Homework 7 - TEMPLATE - DO NOT DO IT YET
.md
.pdf
Basics of Variational Inference
Basics of Variational Inference
#
TODO: Write me.
