终其一生,我们只不过在寻找自己

0%

MSBD5004 Mathematical Methods for Data Analysis Homework 3

最近学的开始有意思了。题也没那么好做了。
MSBD5004 Mathematical Methods for Data Analysis Homework 3

Q1

Solution: All use the standard inner product$\left\langle A,B\right \rangle =\sum_{1\le j\le n}\sum_{1\le i\le n}a_{i,j}b_{i,j}$.

(a)

We can define the $A_{i=s,j=t}=1,A_{i\not=s,j\not=t}=0$, then we can get $E_{st}(X)=\langle A,X \rangle =x_{st}$.

(b)

We can define the $A_{i=j}=1,A_{i\not=j}=0$, then we can get $Tr(X)=\langle A,X \rangle =\sum_{i=1}^n x_{ii}$.

(c)

So we can define the $A=aa^T$.

Q2

(a)

Proof:
If $\boldsymbol{x}, \boldsymbol{z} \in S_{1} \cap S_{2},$ then we can get

So $(1+t) \boldsymbol{z}-t \boldsymbol{x}\in S_{1}$ and $(1+t) \boldsymbol{z}-t \boldsymbol{x}\in S_{2}$ respectively.
So $(1+t) \boldsymbol{z}-t \boldsymbol{x} \in S_{1} \cap S_{2}$ for any $t \in \mathbb{R}$.
So $S_{1} \cap S_{2}$ is a plane.

(b)

①: $\boldsymbol{z}$ is solution of $\min _{\boldsymbol{x} \in S_{1} \cap S_{2}}|\boldsymbol{x}-\boldsymbol{y}|$
②: $\langle\boldsymbol{z}-\boldsymbol{y}, \boldsymbol{z}-\boldsymbol{x}\rangle=0, \forall \boldsymbol{x} \in S_{1} \cap S_{2},\boldsymbol{z} \in S_{1} \cap S_{2}$

  • $①\rightarrow②$

As (a) result, $(1+t) \boldsymbol{z}-t \boldsymbol{x} \in S_{1} \cap S_{2}$ for any $t \in \mathbb{R}$.
Since Z is closest to y on $S_{1} \cap S_{2}$, we have

If we choose $t> 0$ , letting $t\rightarrow0_+$ gives $\langle z -y, z-x\rangle\geq 0$.
If we choose $t> 0$ , letting $t\rightarrow0_-$ gives $\langle z -y, z-x\rangle\leq 0$.
Altogether, $z$ satisfies $\langle z -y, z-x\rangle= 0,\forall \boldsymbol{x} \in S_{1} \cap S_{2}$

  • $②\rightarrow①$

Since$\langle \boldsymbol{z}-y, \boldsymbol{z}-x\rangle=0 , \forall \boldsymbol{x} \in S_{1} \cap S_{2}$

This together with $\boldsymbol{z} \in S_{1} \cap S_{2}$ implies z minimizes $|\boldsymbol{z}-y|^{2}$ in $\boldsymbol{x} \in S_{1} \cap S_{2}$.

(c)

Because $\boldsymbol{z}$ need satisfies $\langle \boldsymbol{z} -y, \boldsymbol{z}-x\rangle= 0$, so set $\boldsymbol{z}=m a_{1}+n a_{2}+y$, and $\boldsymbol{z} \in S_{1} \cap S_{2}$ so we can get this two equation.

$Simplify \downarrow$

$Simplify \downarrow$

Then we can get the explicit solution is $\boldsymbol{z}=m a_{1}+n a_{2}+y$.

(d)

Suppose we have two solutions $\boldsymbol{z_1}$ and $\boldsymbol{z_2}$. Then
$\boldsymbol{z_1}$ is a solution, $\rightarrow$ $\langle \boldsymbol{z_1} -y, \boldsymbol{z_1}-\boldsymbol{z_2}\rangle= 0$.
$\boldsymbol{z_2}$ is a solution, $\rightarrow$ $\langle \boldsymbol{z_2} -y, \boldsymbol{z_2}-\boldsymbol{z_1}\rangle= 0$.
Taking difference leads to $\langle \boldsymbol{z_1} -\boldsymbol{z_2}, \boldsymbol{z_1}-\boldsymbol{z_2}\rangle= 0$.
So $|\boldsymbol{z_1}-\boldsymbol{z_2}|^2=0,\boldsymbol{z_1}=\boldsymbol{z_2}$ . It’s unique.

Q3

(a)

For any $a \in H$, we claim that $a$ can be decomposed into a a part in training data space and a part which is orthogonal to it.

Therefore,

where $K=\left[\begin{array}{c}\boldsymbol{x}_{1} \\ \boldsymbol{x}_{2} \\ \vdots \\ \boldsymbol{x}_{N}\end{array}\right]\cdot \left[\begin{array}{c}\boldsymbol{x}_{1} , \boldsymbol{x}_{2} ,\dots ,\boldsymbol{x}_{N}\end{array}\right]$, $C=\left[\begin{array}{c}c_{1} \\ c_{2} \\ \vdots \\ c_{N}\end{array}\right] \in \mathbb{R}^{N}$
Because $|a_s|^2_2 \geq 0$, the objective function is minimized when $|a_s|^2_2=0$
Thus, the solution must in the form$a = \sum_{i=1}^Nc_i \boldsymbol{x}_i$.

(b)

From (a) we can get

The unknowns from $\boldsymbol{a}\in \mathbb{R}^{n}$ become $\boldsymbol{c}\in \mathbb{R}^{N}$ N (N<n), which has fewer unknowns than original formulation.

-------------    你的留言  是我更新的动力😊    -------------