MSBD5004 Mathematical Methods for Data Analysis Homework 4

这次作业感觉有点简单啊。🤧 最后一题凭感觉写的，不知道对不对，如果有问题请在留言板告诉我。
希望大家留言板支持一下博主，打公式真的好累。☹️ 给个例子让你们参考一下。

Q1

Solution:

$$\begin{aligned} \|A\| =\sup _{\|x\|_{V}=1}\|A x\|_{\mathbb{R}}=\sup _{\|x\|_{V}=1}\|\langle a,x \rangle\|=\sup _{\|x\|_{V}=1}\sqrt{|\sum a_ix_i|} =\sqrt{\max(a_i)} \end{aligned}$$

◾

Q2

Solution:

$$\begin{aligned} &f_{1}(\boldsymbol{x})=f_{1}\left(\boldsymbol{x}^{(0)}\right)+ D f_{1}\left(\boldsymbol{x}^{(0)}\right)\left(\boldsymbol{x}-\boldsymbol{x}^{(0)}\right)+o\left(|| \boldsymbol{x}-\boldsymbol{x}^{(0)} \|\right)\\ &f_{2}(\boldsymbol{x})=f_{2}\left(\boldsymbol{x}^{(0)}\right)+D f_{2}\left(\boldsymbol{x}^{(0)}\right)\left(\boldsymbol{x}-\boldsymbol{x}^{(0)}\right)+o\left(\left\|\boldsymbol{x}-\boldsymbol{x}^{(0)}\right\|\right)\\ &\vdots\\ &f_{n}(\boldsymbol{x})=f_{i}\left(\boldsymbol{x}^{(0)}\right)+D f_{n}\left(\boldsymbol{x}^{(0)}\right)\left(\boldsymbol{x}-\boldsymbol{x}^{(0)}\right)+o\left(\left\|\boldsymbol{x}-\boldsymbol{x}^{(0)}\right\|\right).\\\\ &So, F(\boldsymbol{x})=F\left(\boldsymbol{x}^{(0)}\right)+D F\left(\boldsymbol{x}^{(0)}\right)\left(\boldsymbol{x}-\boldsymbol{x}^{(0)}\right)+o\left(\left\|\boldsymbol{x}-\boldsymbol{x}^{(0)}\right\|\right).\\ & \text{where} D F\left(\boldsymbol{x}^{(0)}\right): V \rightarrow \mathbb{R} \text{ defined by} DF\left(\boldsymbol{x}^{(0)}\right)(\boldsymbol{y})=\left[\begin{array}{l} D f_{1}\left(\boldsymbol{x}^{(0)}\right)(\boldsymbol{y}) \\ D f_{2}\left(\boldsymbol{x}^{(0)}\right)(\boldsymbol{y}) \\ \qquad \vdots\\ D f_{n}\left(\boldsymbol{x}^{(0)}\right)(\boldsymbol{y}) \\ \end{array}\right] \end{aligned}$$

In particular, if $V$ is a Hilbert space,

$$D F(\boldsymbol{x})(\boldsymbol{y})=\left[\begin{array}{c}\left\langle\nabla f_{1}(\boldsymbol{x}), \boldsymbol{y}\right\rangle \\ \left\langle\nabla f_{2}(\boldsymbol{x}), \boldsymbol{y}\right\rangle \\ \vdots \\ \left\langle\nabla f_{n}(\boldsymbol{x}), \boldsymbol{y}\right\rangle\end{array}\right], \quad \forall \boldsymbol{x} \in V$$

◾

Q3

Solution:
（a)
$\nabla f(\boldsymbol{x})=\boldsymbol{A}^T(\boldsymbol{A} \boldsymbol{x}-\boldsymbol{b})+2\lambda\boldsymbol{x}$
$\nabla^2 f(\boldsymbol{x})=\boldsymbol{A}^T\boldsymbol{A} +2\lambda$
(b)
$\nabla f(\boldsymbol{X})=\boldsymbol{b}^T\boldsymbol{X}\boldsymbol{c}=\boldsymbol{c}\boldsymbol{b}^T$
$\nabla^2 f(\boldsymbol{x})=0$
(c)
$\nabla f(\boldsymbol{x})=\boldsymbol{x}^T(\boldsymbol{A}+\boldsymbol{A}^T)$
$\nabla^2 f(\boldsymbol{x})=(\boldsymbol{A}+\boldsymbol{A}^T)$
(d)
$\nabla f(\boldsymbol{X})=\boldsymbol{b}^T\boldsymbol{X}\boldsymbol{c}+\boldsymbol{c}^T\boldsymbol{X}^T\boldsymbol{b}$
$\nabla^2 f(\boldsymbol{X})=\boldsymbol{c}\boldsymbol{b}^T+\boldsymbol{c}\boldsymbol{b}^T$
(e)
$\nabla f(\boldsymbol{X})=\boldsymbol{A}\boldsymbol{X}\boldsymbol{B}+\boldsymbol{B}\boldsymbol{X}\boldsymbol{A}$
$\nabla^2 f(\boldsymbol{X})=\boldsymbol{B}\boldsymbol{A}+\boldsymbol{A}\boldsymbol{B}$