The sigmoid function transforms its inputs, for which values lie in the domain R, to outputs that lie on the interval (0, 1). For that reason, the sigmoid is often called a squashing function: it...

please answer correctly

Extracted text: The sigmoid function transforms its inputs, for which values lie in the domain R, to outputs that lie on the interval (0, 1). For that reason, the sigmoid is often called a squashing function: it squashes any input in the range (-inf, inf) to some value in the range (0, 1): 1 sigmoid(a) = (4.1.6) 1+ exp(-æ)" In the earliest neural networks, scientists were interested in modeling biological neurons which either fire or do not fire. Thus the pioneers of this field, going all the way back to McCulloch and Pitts, the inventors of the artificial neuron, focused on thresholding units. A thresholding activa- tion takes value 0 when its input is below some threshold and value 1 when the input exceeds the threshold. When attention shifted to gradient based learning, the sigmoid function was a natural choice be- cause it is a smooth, differentiable approximation to a thresholding unit. Sigmoids are still widely used as activation funetions on the output units, when we want to interpret the outputs as prob- abilities for binary classification problems (you can think of the sigmoid as a special case of the softmax). However, the sigmoid has mostly been replaced by the simpler and more easily train- able ReLU for most use in hidden layers. In later chapters on recurrent neural networks, we will describe architectures that leverage sigmoid units to control the flow of information across time. Below, we plot the sigmoid function. Note that when the input is close to 0, the sigmoid function approaches a linear transformation.