## 2015年1月31日 星期六

### [ RubyAlg ] MIT Linear Algebra, Spring 2005 - Lec16 (Linear Regression)

Source From Here

Recall

P 矩陣用來將 b 投影到其 Column Space 上, 因此考慮一下極端案例:
* b 就在 Column Space 上, 則 Pb = b
* b 與 Column Space 垂直, 則 Pb = 0

Projections

e = b - p = (b - Pb) = (I-P)b

Linear Regression

(上面的 [1,2,3] 應該是 [1,2,2]

1. require "alg/math/LinearAlgebra"
2.
3. LA = LinearAlgebra
4.
5. A = LA.newMtx3(3,2,[1,1,1,2,1,3])
6. b = LA.newMtx3(3,1,[1,2,2])
7. printf("A:\n%s\n", A)
8. printf("A^T:\n%s\n",A.t)
9. printf("b:\n%s\n", b)
10. printf("\n")
11.
12. A2 = A.t*A
13. b2 = A.t*b
14. printf("A^T*A:\n%s\n", A2)
15. printf("A^T*b:\n%s\n", b2)

3C' + 6D' = 5
6C' + 14D' = 11

(此時的解 C'D' 與原先方程式的解 CD 是不同的!

1. x2 = A2.inv*b2
2. printf("[C',D']:\n%s\n", x2)

[C',D']:
0.6666666666666679
(C'=2/3)
0.5 (D'=1/2)

(上面的 e1, e2, e3 還需乘與一個負號

p 與 e 垂直: 很直覺, 因為 p 是在 Column Space, e 是在 Null Space of A^T 上, 這兩個分量注定垂直. (p*e=0)
e 與 Column Space 垂直: 這也是定義的觀點, 也就是說 e*[1,1,1] 與 e*[1,2,3] 皆會得到 zero!

Supplement
- If A has independent column vector(s), then A^T*A is invertable (Prove)
Suppose A^T*A*x=0, then x must be zero vector if A^T*A is invertable. Let's play a trick here. Consider:
x^T*A^T*A*x = (A*x)^T * (A*x)=0

So we can infer that A*x = 0, then we know that A has independent column which force x to be zero only!

## 關於我自己

Where there is a will, there is a way!