ITCS 4122/5122 Visual Analytics Fall, 2018 Homework #5 due Wednesday, November 7 (end of class) In this assignment we will use a simple training dataset to learn the parameters of two different...

Please see attached. Use Anaconda/Jupter notebook for code.


ITCS 4122/5122 Visual Analytics Fall, 2018 Homework #5 due Wednesday, November 7 (end of class) In this assignment we will use a simple training dataset to learn the parameters of two different models. The Welch’s t-test will be used in conjunction with leave-one-out cross- validation and root-mean-squared-error (RMSE) to determine whether there is statistical significance in the performance of the two models. You are asked to write code to numerically solve this problem. You are required to submit this code in Canvas along with your analysis. Consider the following training dataset. i xi ti 1 0.100 0.603 2 0.200 0.986 3 0.250 0.891 4 0.350 0.834 5 0.400 0.572 6 0.450 0.353 7 0.500 -0.085 8 0.600 -0.371 9 0.750 -0.967 10 0.800 -0.989 11 0.850 -0.749 12 0.900 -0.382 The models proposed to describe this data are a linear combination of basis functions with ?0(?) = 1, so that ?0 acts as a bias. The goal is to infer model parameters w from the data. Two competing models are proposed to describe the data. The difference between the Two competing models is the order of the expansion M. Model Basis function Order M Parameters of basis functions M1 5 ?? = {0.000, 0.333, 0.667, 1.000} ? = 0.3 M2 8 ?? = {0.000, 0.167, 0.333, 0.500, 0.667, 0.833, 1.000} ? = 0.3 ITCS 4122/5122 Visual Analytics Fall, 2018 1. [10pts] Given the above x values and basis functions print the design matrix / features to be used in the linear regression for both models. 2. [10pts] Provide the standardize features / design matrix by removing the mean and scaling to unit variance all columns. Print the scaled design matrices for both models. 3. [20pts] Use leave-one-out cross-validation and L2 (ridge/quadratic) regularization with the following possible regularization coefficients (0.1, 1.0, 10.0) to learn model parameters. What is the estimated regularization and the coefficients for both models? 4. [20pts] For the estimated regularization what is the mean and standard deviation of the RMSEs. 5. [20pts] Provide two plots with the prediction of the two models as x varies between 0 and 1. On the same figure, plot also the provided training data. 6. [10pts] Use the Welch’s t-test to check whether the performance of M1 is better than performance of M2. What is the t-statistic and the p-value? 7. [10pts] Do you believe that the data is capable to discriminate between the two models and offer a clear winner? Given that you have no additional training or testing data, which model will you use in production? Please explain.