Greedy search method for separable nonlinear models using stage Aitken gradient descent and least squares algorithms

Chen, Jing; Mao, Yawen; Gan, Min; Wang, Dongqing; Zhu, Quanmin

Abstract

Aitken gradient descent (AGD) algorithm takes some advantages over the standard gradient descent (SGD) and Newton methods: (1) can achieve at least quadratic convergence in general; (2) does not require the Hessian matrix inversion; (3) has less computational efforts. When using the AGD method for a considered model, the iterative function should be unchanging during all the iterations. This paper proposes a hierarchical AGD algorithm for separable nonlinear models based on stage greedy method. The linear parameters are estimated using the least squares algorithm, and the nonlinear parameters are updated based on the AGD algorithm. Since the iterative function is changing at each iteration, a stage AGD algorithm is introduced. The convergence properties and simulation examples show effectiveness of the proposed algorithm.

Citation

Chen, J., Mao, Y., Gan, M., Wang, D., & Zhu, Q. (2023). Greedy search method for separable nonlinear models using stage Aitken gradient descent and least squares algorithms. IEEE Transactions on Automatic Control, 68(8), 5044-5051. https://doi.org/10.1109/TAC.2022.3214474

• Journal Article Type: Article
• Acceptance Date: Oct 13, 2022
• Online Publication Date: Oct 13, 2022
• Publication Date: Aug 31, 2023
• Deposit Date: Dec 16, 2022
• Publicly Available Date: Dec 16, 2022
• Journal: IEEE Transactions on Automatic Control
• Print ISSN: 0018-9286
• Electronic ISSN: 1558-2523
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 68
• Issue: 8
• Pages: 5044-5051
• DOI: https://doi.org/10.1109/TAC.2022.3214474
• Keywords: Electrical and Electronic Engineering, Computer Science Applications, Control and Systems Engineering
• Public URL: https://uwe-repository.worktribe.com/output/10109159
• Publisher URL: https://ieeexplore.ieee.org/document/9917555