What You Will Learn in This Section
- Derivation of Logistic Regression Cost Function using MLE
In logistic regression, we model the probability that a given input
This can be rewritten in a more compact form:
Likelihood Function
Given that the training examples are independent, the likelihood of the parameters is given by:
Substituting the probability formula:
Log-Likelihood Function
Since it is easier to work with the log of the likelihood function, we take the log-likelihood:
Negative Log-Likelihood as Cost Function
The cost function for logistic regression is chosen as the negative log-likelihood, also known as binary cross-entropy loss:
This function ensures that the predicted probabilities align well with the actual labels, and minimizing it leads to optimal parameter values for logistic regression.