LMS-Algorithm
Is the simplest and most well known Gradient Algorithm.
Given a cost [see page 14, function] this algorithm continually moves in the direction of the negative gradient until it converges towards a minimum.
The drawback with this algorithm is the step-size being difficult to calculate. It can be a value in the range from 0 to some value related to $λ\text{max}\ (the maximum eigenvalue of the auto-correlation matrix \(R_{xx}\) (which changes with each weight calculation))