Logistic Regression utilizes the sigmoid activation function, which takes any numerical input and maps it to a value within the range \( [0,1] \). The mathematical representation of the sigmoid function is given below: \begin{align} \text{sigmoid}(x)=\frac{1}{1+e^{-x}} \end{align} The graph below illustrates how the sigmoid function behaves along the number line. At \(x=0\), the function outputs 0.5. For larger values of \(x\), the sigmoid function approaches 1, while for smaller values, it converges towards 0.

Why Is the Sigmoid Function Used in Logistic Regression?

In classification problems, the dependent variable has two possible values: 0 and 1. The sigmoid function takes any numerical input and maps it to a value between \( [0,1] \), allowing us to predict the class of data points based on its output. We can determine class = 1 when the sigmoid value exceeds a chosen threshold; otherwise, we assign class = 0. A common threshold choice is 0.5, meaning: \begin{align} class = \left\{ \begin{array}{cl} 1 & \text{if} \ \text{sigmoid}(x) \geq 0.5 \\ 0 & \text{if} \ \text{sigmoid}(x) < 0.5 \end{array} \right. \end{align} However, the threshold can be treated as a tunable hyperparameter. Instead of 0.5, we can set any other threshold based on specific requirements. For example, we may decide that a data point is classified as class = 1 only if the sigmoid score is greater than 0.8; otherwise, it is classified as 0.