Show that the univariate penalized least squares estimates is, without truncation, in general not consistent, even if X and Y are bounded. Hint: Consider random variables X with P{X = 0} = 1 2 and P{X ≤ x} = 1+x 2 for x ∈ [0, 1] and Y independent of X with P{Y = −1} = P{Y = +1} = 1 2 . Hence m(x) = 0 for all x. Now draw an i.i.d. sample (X1, Y1), ..., (Xn, Yn) from the distribution of (X, Y ). Show that if the event
A := {X1 = ··· = Xn−1 = 0; Y1, ..., Yn−1 = −1; Xn = 0; Yn = 1}
occurs, then the smoothing spline mn obtained with penalty J2 k for k ≥ 2 is the straight line through (0, −1) and (Xn, 1), mn(x) = −1+ 2x Xn . Use this to conclude that the L2 error satisfies
E |mn(x) − m(x)| 2 PX(dx) ≥ E IA · 1 2
1 0 −1 + 2x Xn 2 dx
= p 2 · E I{Xn=0} · Xn 6 −1 + 2 Xn 3 + 1
= p 4
1 0 u 6 −1 + 2 u 3 + 1 du
= ∞,
where p = P{X1 = ··· = Xn−1 = 0; Y1 = ··· = Yn−1 = −1; Yn = 1} > 0.