Hide code cell source 
import matplotlib.pyplot as plt
%matplotlib inline
import matplotlib_inline
matplotlib_inline.backend_inline.set_matplotlib_formats('png')
import seaborn as sns
sns.set_context("paper")
sns.set_style("ticks");

Homework 2#

References#

  • Lectures 4 through 8 (inclusive).

Instructions#

  • Type your name and email in the “Student details” section below.

  • Develop the code and generate the figures you need to solve the problems using this notebook.

  • For the answers that require a mathematical proof or derivation you should type them using latex. If you have never written latex before and you find it exceedingly difficult, we will likely accept handwritten solutions.

  • The total homework points are 100. Please note that the problems are not weighed equally.

Student details#

  • First Name:

  • Last Name:

  • Email:

  • Used generative AI to complete this assignment (Yes/No):

  • Which generative AI tool did you use (if applicable)?:

Problem 1 - The Pythagorean theorem on Hilbert Spaces#

Let \(H\) be a Hilbert space with inner product \(\langle \cdot, \cdot \rangle\) and norm \(\| \cdot \|\). Let \(x, y \in H\).

Part A#

Prove that if \(x\) and \(y\) are orthogonal, then the Pythagorean theorem holds, i.e.,

\[ \| x + y \|^2 = \| x \|^2 + \| y \|^2. \]

Hint: Use the fact that \(\| x + y \|^2 = \langle x + y, x + y \rangle\).

Answer:

Part B#

Prove the following generalization of the Pythagorean theorem. Let \(x_1,x_2,\dots,x_n \in H\) be pairwise orthogonal, i.e., \(\langle x_i, x_j \rangle = 0\) for all \(i \neq j\). Then,

\[ \| x_1 + x_2 + \dots + x_n \|^2 = \| x_1 \|^2 + \| x_2 \|^2 + \dots + \| x_n \|^2. \]

Hint: Use induction and the result from Part A.

Answer:

Problem 2 - All infinite dimensional Hilbert spaces are isomorphic to \(\ell^2\)#

We mentioned in the lecture that an infinite dimensional Hilbert space \(H\) are isomorphic to \(\ell^2\). In this problem we will prove this result. Intuitively, this means that we can think of vectors in \(H\) as infinite dimensional vectors in \(\ell^2\). It is as if the space \(H\) is a relabeling of the space \(\ell^2\). First, recall that

\[ \ell^2 = \left\{ a = (a_1, a_2, \dots) \mid \sum_{i=1}^\infty |a_i|^2 < \infty \right\}. \]

To show that two spaces are isomorphic, we need to show that there exists a bijective linear map between them which keeps the inner product intact. Bijection means that the map is one-to-one and onto. So, we need to find an invertible, linear map:

\[ T: H \to \ell^2. \]

To keep the inner product intact, we need to show that for all \(x, y \in H\),

\[ \langle x, y \rangle = \langle T(x), T(y) \rangle_{\ell^2}. \]

Here, on the left we have the inner product in \(H\) and on the right we have the inner product in \(\ell^2\). If the inner products are intact, orthogonality is preserved by \(T\). And also norms are preserved, since \(\| x \| = \sqrt{\langle x, x \rangle}\).

Okay, this is what you will have to do. I will give you the right \(T\) and you will have to show that it is linear, invertible, and keeps the inner product intact.

Recall that since \(H\) is separable, it has a countable orthonormal basis \(\{ e_1, e_2, \dots \}\). This means that every vector \(x \in H\) can be written as

\[ x = \sum_{i=1}^\infty \langle x, e_i \rangle e_i. \]

The idea is to use the Fourier coefficients \(\langle x, e_i \rangle\) as the entries of the vector \(T(x)\):

\[ T(x) = ( \langle x, e_1 \rangle, \langle x, e_2 \rangle, \dots ). \]

Part A#

Show that \(T(x)\) is indeed in \(\ell^2\) for all \(x \in H\). That is, show that \(\sum_{i=1}^\infty |\langle x, e_i \rangle|^2 < \infty\).

Hint: Use Parseval’s identity.

Answer:

Part B#

Show that \(T\) is a linear map, i.e., show that for all \(x, y \in H\) and \(\alpha, \beta \in \mathbb{R}\),

\[ T(\alpha x + \beta y) = \alpha T(x) + \beta T(y). \]

Answer:

Part C#

Show that \(T\) is onto.

Hint: Take a vector \(a \in \ell^2\) and show that there exists a vector \(x \in H\) such that \(T(x) = a\). Just try to write down the vector \(x\) in terms of \(a\) and the orthonormal basis \(\{ e_1, e_2, \dots \}\).

Answer:

Part D#

Show that \(T\) is one-to-one.

Hint: Take two vectors \(x, y \in H\) and show that if \(T(x) = T(y)\), then \(x = y\).

Answer:

Part E#

Show that \(T\) keeps the inner product intact. That is, show that for all \(x, y \in H\),

\[ \langle x, y \rangle = \langle T(x), T(y) \rangle_{\ell^2}. \]

Hint: Use the fact that \(T\) is linear and the definition of \(T\). The inner product of two vectors in \(\ell^2\) is defined as \(\langle a, b \rangle_{\ell^2} = \sum_{i=1}^\infty a_i b_i\).

Answer:

Problem 3 - Numerical Construction of Polynomial Chaos#

Through this problem, you are going to construct orthogonal polynomials for the exponential distribution and test a few things with them. You need to familiarize yourself with this hands-on-activity before you proceed.

Part A#

Consider the random variable:

\[ \Xi \sim \exp(1). \]

The exponential distribution has the following probability density function:

\[\begin{split} f_\Xi(\xi) = \begin{cases} e^{-\xi} & \xi \geq 0 \\ 0 & \xi < 0 \end{cases}. \end{split}\]

Use the orthojax package to construct the first 5 orthogonal polynomials for \(\Xi\). Plot them on the same figure for \(\xi \in [0, 5]\).

# your code here
# Hint: You can use the function orthojax.make_orthogonal_polynomial
# but you need to pass the argument right=jnp.inf to indicate that
# the right endpoint is infinity.

import orthojax as ojax
import jax.numpy as jnp

# Your code here

Part B#

Project the function:

\[ f(\xi) = \sin(x) \]

onto the first 5 orthogonal polynomials for \(\Xi\). Plot the function \(f\) and its projection on the same figure for \(\xi \in [0, 5]\).

Hint: Do exactly what I do in the activity. You need to extract from poly the quadrature rule so that you can do the inner product.

# Your code here

Part C#

Use the polynomial projection to calculate the mean and variance of the random variable

\[ Y = f(\Xi) = \sin(\Xi). \]

Compare to Monte Carlo estimates or the exact values.

# Your code here

Problem 4 - Uncertainty Propagation with Polynomial Chaos#

Consider the Lorenz system:

\[\begin{split} \begin{align*} \dot{x} &= \sigma(y-x),\\ \dot{y} &= x(\rho-z)-y,\\ \dot{z} &= xy-\beta z, \end{align*} \end{split}\]

with parameters \(\sigma=10\), \(\beta=8/3\), and \(\rho=28\). Take the initial conditions to be random:

\[\begin{split} \begin{align*} x(0) &\sim \mathcal{N}(0, 0.01),\\ y(0) &\sim \mathcal{N}(0, 0.01),\\ z(0) &\sim \mathcal{N}(0, 0.01). \end{align*} \end{split}\]

Part A - Build a Polynomial Chaos Surrogate#

Build a polynomial chaos surrogate. Calculate the mean and the variance as a function of time. Compare the result to Monte Carlo estimates.

# Your code here

Part B - Predictions#

Generate three random initial conditions and propagate them forward in time using the surrogate. Plot only \(x\) as a function of time for each initial condition. Compare to the ground truth.

# Your code here

Part C - Probability Density Function#

Use your surrogate to estimate the probability density function of \(x\) at \(t=1, 2, 5,\) and \(10\). Use different plots for each case. You can do this, by generating \(100,000\) initial conditions, propagating them forward through the surrogate and then plotting a histogram of the results. Compare to Monte Carlo PDFs. Use transparency in your plots.

# Your code here

Note: I may add one more homework problem here.