In this chapter we limited ourselves to classification problems, for which cross entropy is typically the loss function of choice. There are also problems where we want our NN to predict particular...

In this chapter we limited ourselves to classification problems, for which cross entropy is typically the loss function of choice. There are also problems where we want our NN to predict particular values. For example, undoubtedly many folks would like a program that, given the price of a particular stock today plus all sorts of other facts about the world, outputs the price of the stock tomorrow. If we were training a single-layer NN to do this we would typically use the squared-error loss:

L(X; φ) = (t 􀀀 l(X; φ))²
(1.32)

where t is the actual price that was achieved on that day and l(X; φ) is the output of the one layer NN with φ= fi,W. (This is also known as quadratic loss.) Derive the equation for the derivative of the loss with respect to bi.