Prove a weaker version of Theorem 7.1: under the conditions of Theorem 7.1,
E |mn(x) − m(x)| 2 µ(dx)
≤ E |m(hˆ) nl (x) − m(x)| 2 µ(dx)+8√ 2L2 :log(2|Qn|) nt .
Hint:
|mn(x) − m(x)| 2 µ(dx) − |m(hˆ) nl (x) − m(x)| 2 µ(dx)
= E (m(H) nl (X) − Y ) 2 Dn − E (m(hˆ) nl (X) − Y ) 2 Dn
= E (m(H) nl (X) − Y ) 2 Dn − 1 nt n l+nt i=nl+1 (m(H) nl (Xi) − Yi)
+ 1 nt n l+nt i=nl+1 (m(H) nl (Xi) − Yi) 2 − 1 nt n l+nt i=nl+1 (m(hˆ) nl (Xi) − Yi) 2
+ 1 nt n l+nt i=nl+1 (m(hˆ) nl (Xi) − Yi) 2 − E{(m(hˆ) nl (X) − Y ) 2 Dn}
≤ E (m(H) nl (X) − Y ) 2 Dn − 1 nt n l+nt i=nl+1 (m(H) nl (Xi) − Yi) 2
+ 1 nt n l+nt i=nl+1 (m(hˆ) nl (Xi) − Yi) 2 − E (m(hˆ) nl (X) − Y ) 2 Dn
≤ 2 max h∈Qn 1 nt n l+nt i=nl+1 (m(h) nl (Xi) − Yi) 2 − E (m(h) nl (X) − Y ) 2 Dn
= 2 max h∈Qn 1 nt n l+nt i=nl+1 (m(h) nl (Xi) − Yi) 2 − E (m(h) nl (X) − Y ) 2 Dnl .
Use Hoeffding’s inequality (cf. Lemma A.3) to conclude
P
|mn(x) − m(x)| 2 µ(dx) − |m(hˆ) nl (x) − m(x)| 2 µ(dx) > |Dnl
≤ 2|Qn|e −nt 2 32L4 .
Compare also Problem 8.2.