Optimization

Numerical optimizers provide the means to estimate our parameters by finding the values that maximize the likelihood of generating our data. This guide helps us understand how optimization algorithms find the best estimates for our coefficients in a step-by-step process. This guide uses Gradient Descent and regularization techniques in a completely manual approach to finding parameters that most likely generated our data. Another way of thinking about gradient descent is that we are inevitably asking the algorithm the following: What parameter values will push our error to zero?