Linear Class
If we consider the space of linear functions, then the sequential and joint estimators coincide. Without regularization the joint estimator takes the form:
\[\min _{\alpha ,\beta } \max _{\theta_1,\theta_2} \ell(\{\alpha, \beta\}, \{\theta_1,\theta_2\})\]
where
\[\ell(\{\alpha, \beta\}, \{\theta_1,\theta_2\}) := 2\theta_1^{\top} \mathbb{E}_n[c' y]-2\theta_1^{\top} \mathbb{E}_n\left[c' a^{\top}\right] \alpha + 2\theta_2^{\top} \mathbb{E}_n[c a^{\top}]\alpha-2\theta_2^{\top} \mathbb{E}_n\left[c b^{\top}\right] \beta\]
Note that the saddle point is given by the system:
\[\begin{split}\begin{aligned}
\mathbb{E}_n[(y-\langle \alpha, a\rangle)c'] &= 0 \\
\mathbb{E}_n[(\langle \alpha, a\rangle-\langle\beta, b\rangle)c] &= 0
\end{aligned}\end{split}\]
Solving first for \(\alpha\) in the first equation, and then for \(\beta\) in the second equation gives the same solution as in the sequential procedure.
Two-stage least squares estimator. |
|
|
Regularized two-stage least squares estimator using Elastic Net. |