4 Backpropagation Consider the following network with sigmoid activation functions in the hidden and output neurons, and binary cross entropy loss function. Assume we initialize the weights as...

Extracted text: 4 Backpropagation Consider the following network with sigmoid activation functions in the hidden and output neurons, and binary cross entropy loss function. Assume we initialize the weights as follows: W11 = 1, W12 = 0.5, W21 = 0.1, W22 = 0.2, W13 = 1, W23 = 0.5. The biases for the hidden nodes are initialized as bl = 0.1, 62 = 0.1, and for the the output node is initialized as 63 = 0.5. Backward pass: Calculate the derivative of the loss w.r.t W11. What's the value of the derivative for the initialized weights, input 1 = 0.1, input 2 = 0.2, and label = 1? How do you update W11 using gradient descent, based on the derivative you derived in the previous part and learning rate 7 = 0.1? What is the value of the loss using the updated W11? How did it change after the update?


Extracted text: Input 1 W11 Hidden 1 W13 W21 Output (3) W12 W23 Input 2 Hidden 2 W22