Quasi-Newton Method

Assume that there are m functions f1(β), …, fm(β) from n variables β = (β1, …, βn). It is required to minimize sum of squares:

If the initial approximation is β0, regular approximation can be found by the formula:

,

Where:

On the first iteration the Hessian matrix is approximated by the matrix, where J is a Jacobian of the S function in the β0 point, that is, Gauss-Newton direction is selected. Further, the approximation of the inverse Hessian matrix is updated using BFGS-formula (Broyden-Fletcher-Goldfarb-Shanno) provided that the condition is satisfied, and remains unchanged.

If on the next iteration the linear search procedure found that with any αs step values the criterion function is not reduced, the ps direction is changed to the Gauss-Newton (as on the first iteration) and step length is calculated again. If this does not help to find an acceptable step value, the algorithm stops.

Iteration method stops if one of the following conditions is met:

See also:

Library of Methods and Models | ARIMA | ARIMA Model Coefficient Estimation