Suppose that x R dis a random variable with probability distribution D, i.e., x ~ D. Moreover, assume
that w ? Rdis a parameter vector and let
y = hx, wi + , where ~ N (0, s2
). (1)
Therefore, y is a linear function of x with Gaussian noise added.1. (15 points) Maximum likelihood estimation:a. Give an explicit expression for the conditional distribution of y|x, w.b. Assume that we (independently) draw n pairs (xi, yi) ? Rd ×R from the above model, i.e. we drawxi from D and then yi according to (1). What is the distribution of (y1, . . . , yn)|(x1, . . . , xn), w?Write down the associated log-likelihood.c. Derive the parameter w that maximizes the log-likelihood of the conditional model from part (b).2. (15 points) Maximum-a-posterior estimation: Assume w ~ N (0, ?2Id) where Id is the d×d identitymatrix. Note that w and are independent.a. Give an explicit expression for the conditional distribution of y|x. Note that w is not specified.b. After drawing n pairs as above, what is the distribution of (y1, . . . , yn)|(x1, . . . , xn)?c. What is the log-posterior distribution w|(x1, y1), . . . ,(xn, yn)?