,The Levenberg-Marquardt method as well as the Quasi-Newton method, is the modification of the classic Gauss-Newton method for minimization of sum of squares of functions.
The Levenberg-Marquardt method uses corrective multiplier to find direction for the next approximation. The direction is found by the formula:
,
where D is a diagonal matrix, the principal diagonal of which matches the principal diagonal of the matrix JtJ.
The corrective multiplier λ (Marquardt factor) is recalculated on each iteration of the method, besides it is decreased if the criterion function is decreased (direction is getting closer to the Gauss-Newton direction), and it is increased on insufficient decrease or inability to decrease of the criterion function (direction is getting closer to the 
).
Recalculation of length of the step αs by the method of linear search is executed not in every iteration but only on inability to decrease criterion function. The Marquardt factor also changes (increases). If five subsequent iterations do not result in decrease of criterion function, the algorithm stops. Other break conditions of the Levenberg-Marquardt algorithm coincide with break conditions of the Quasi-Newton algorithm.
See also:
Library of Methods and Models | ARIMA | ARIMA Model Coefficient Estimation