The Importance of Learning Rates in Machine Learning

The learning rate is the size of the steps that are taken when I make these adjustments to the weights. And usually it's a small value because as we say in our, the common mantra with CPMAI, the cognitive project management for AM methodology, think big, start small and iterate often. A low learning rate has small steps, so it could take longer. So actually smaller steps are usually better to get you to convergence faster. higher learning rates do result in larger steps, but risks overshooting that minimum.