Decomposition-based recursive least squares identification methods for multivariate pseudo-linear systems using the multi-innovation

ABSTRACT This paper studies the parameter estimation algorithms of multivariate pseudo-linear autoregressive systems. A decomposition-based recursive generalised least squares algorithm is deduced for estimating the system parameters by decomposing the multivariate pseudo-linear autoregressive system into two subsystems. In order to further improve the parameter accuracy, a decomposition based multi-innovation recursive generalised least squares algorithm is developed by means of the multi-innovation theory. The simulation results confirm that these two algorithms are effective.


Introduction
System identification is the modelling theory and method for researching on static systems and dynamic systems (Anfinsen, Diagne, Aamo, & Krstic, 2017;Bravo, Suarez, Vasallo, & Alamo, 2016;Zhu, Yu, & Zhao, 2017). Mathematical model can be established by the mechanism analysis method and the statistical method (Bessaoudi, Ben Hmida, & Hsieh, 2017;Escobar & Poznyak, 2015;Zhu, Wang, Zhao, Li, & Billings, 2015). The mechanism analysis method uses equations of the law of the system motion to obtain the system model structure. Model parameters can be gained by measuring, as well as some kinds of identification methods by collecting the system input and output data (Ding, Xu, & Zhu, 2016;Ma, Ding, Alsaedi, & Hayat, 2018;. In recent years, some system identification methods have been presented by researchers all over the world. Isaksson, Sjoberg, Tornqvist, Ljung, and Kok (2015) proposed an established method for grey-box identification by using the maximum-likelihood estimation and the extended Kalman filtering. Wang and Ding (2016) presented a recursive generalised extended least squares algorithm and a generalised extended stochastic gradient algorithm for identifying the parameters of a class of nonlinear systems.  derived a filtering-based generalised stochastic gradient algorithm for multivariate pseudo-linear autoregressive systems by means of the data filtering technique.
CONTACT Feng Ding fding@jiangnan.edu.cn Some typical industrial processes are multivariate systems in essence (Milad & Javad, 2017;Ramezani, Arefi, Zargarzadeh, & Jahed-Motlagh, 2017;Tohru & Hajime, 2016). Multivariate systems have the characteristics of high dimensions, many input and output variables and lots of system parameters, and these factors result in heavy computational burden of identification algorithms (Mahnaz, 2017;Sergey & Svyatoslav, 2016;Theodorakopoulos & Rovithakis, 2016). The decomposition technique is an effective method to separate a large-scale system into several small-sized subsystems (Ding, Wang, Xu, Tasawar, & Ahmed, 2017;Ding, Wang, Xu, & Wu, 2017;. Applying the decomposition technique to the multivariate system can reduce the computational burden. Ma, Xiong, Chen, and Ding (2017) developed a modified Kalman filter based hierarchical least squares algorithm for multi-input multioutput Hammerstein systems. The highlight of that paper is to adopt the hierarchical identification to decompose the system into two fictitious subsystems, one containing the unknown parameters in the non-linear block and the other containing the unknown parameters in the linear subsystems.
The multi-innovation identification theory can improve the parameter estimation accuracy by expanding the identification innovation from a scalar innovation to an innovation vector, or from a vector innovation to an innovation matrix (Hu, Liu, & Zhou, 2014;Shi, Wang, & Ji, 2016b;Li & Liu, 2018). In this aspect, Meng (2017) derived a multi-innovation stochastic gradient algorithm for identifying the parameters of bilinear systems based on the least squares principle and the multi-innovation identification theory. Ding, Wang, Mao, and Xu (2017) developed a filtering-based multi-innovation gradient algorithm for a linear state space system with time-delay, and analysed that the parameter estimates given by the presented algorithms converge to their true values under the persistent excitation conditions. Chen, Ding, Alsaedi, and Hayat (2017) derived a filtering-based maximum likelihood multi-innovation extended gradient algorithm to estimate the parameters of controlled autoregressive autoregressive moving average systems by replacing the unmeasurable variables in the information vectors with their estimates. This paper researches the recursive identification methods for multivariate pseudo-linear autoregressive systems based on the decomposition technique and the multi-innovation identification theory. The fundamental idea is to use the decomposition technique to transform the original system into two subsystems, where one contains a system model parameter vector and a measurable information matrix and the other contains a noise model parameter matrix and an unmeasurable information vector, then to estimate each subsystem by using the least squares principle. The main contributions of this paper lie in the following: (1) A multivariate decomposition-based recursive generalised least squares (M-D-RGLS) algorithm is presented for the multivariate pseudo-linear autoregressive system based on the decomposition technique.
(2) A multivariate decomposition-based multiinnovation recursive generalised least squares (M-D-MI-RGLS) algorithm is proposed in order to improve the parameter accuracy of the M-D-RGLS algorithm.
The rest of this paper is organised as follows. Section 2 describes a multivariate pseudo-linear autoregressive system and decomposes it into two sub-identification models. Section 3 derives a M-D-RGLS algorithm. Section 4 obtains a M-D-MI-RGLS algorithm. Section 5 provides an example for illustrating the results in this paper. Finally, we offer some concluding remarks in Section 6.

System description and identification model
First of all, we give some notations in this paper. I m denotes an identity matrix of size m × m; 1 n stands for an n-dimensional column vector whose elements are 1, that is 1 n := [1, 1, . . . , 1] T ∈ R n ; 1 m × n represents a matrix of size m × n whose elements are 1; the norm of a matrix X is defined by X 2 := tr[XX T ], the superscript T stands for the matrix/vector transpose.
Consider the following multivariate pseudo-linear autoregressive system: where y(t ) := [y 1 (t ), y 2 (t ), . . . , y m (t )] T ∈ R m is the system output vector, s (t ) ∈ R m×n is the system information matrix consisting of the input-output data, θ ∈ R n is the system parameter vector to be identified, v(t ) : Without loss of generality, assume that the orders m, n and n c are known, and y(t ) = 0, s (t ) = 0 and v(t ) = 0 for t ࣘ 0. Define the related noise vector w(t ), the noise model parameter matrix θ n and the noise model information vector ψ(t ) as From Equation (2), we have Equation (3) is the noise model. Therefore, Equation (1) can be rewritten as This identification model contains a system model parameter vector θ and a noise model parameter matrix θ n . s (t ) is the system information matrix consisting of the input-output data, and thus is known, ψ(t ) is the noise model information vector consisting of the unmeasurable noise items w(t − i), i = 1, 2, … , n c , so it is unknown. Introduce two intermediate variables, According to the hierarchical identification principle and Equation (3), the system in Equation (4) can be decomposed into the following two fictitious subsystems: The hierarchical structure of these two identification models is shown in Figure 1.
By decomposing the identification model in Equation (4), two sub-identification models in Equations (7) and (8) have been obtained, one contains the system model parameter vector θ and the measurable information matrix s (t ), the other contains the noise model parameter matrix θ n and the unmeasurable information vector ψ(t ). We can deduce their generalised least squares algorithms, respectively, and use the recursive identification principle to co-ordinate the related items between these two algorithms. Suppose is the estimate of w(t ), and so on.
The flowchart of computingθ(t ) in the M-D-RGLS algorithm is shown in Figure 2.

The M-D-MI-RGLS algorithm
The multi-innovation identification theory can extract more useful information from observation data to improve the parameter estimation accuracy (Chen, Ding, Xu, Hayat, & Alsaedi, 2017;Mao & Ding, 2016;Pan, Jiang, Wan, & Ding, 2017). Based on the M-D-RGLS algorithm in Equations (22)-(30), according to the multiinnovation identification theory, we introduce an innovation length p to expand the innovation vector to a large innovation vector/matrix. Define the stacked information matrices 1 (p, t ) and 2 (p, t ), the stacked output vector Y (p, t ) and the stacked noise output matrix in Equation (22) and e 2 (t ) :=ŵ(t ) −θ T n (t − 1)ψ(t ) ∈ R m in Equation (25) are the innovation vectors. Define p ࣙ 1 as the innovation length, expand the innovation vectors e 1 (t ) and e 2 (t ) into a large innovation vector E 1 (p, t ) and an innovation matrix E 2 (p, t ): . . .
The flowchart of computingθ(t ) in the M-D-MI-RGLS algorithm is shown in Figure 3.
Obviously, we can obtain the M-D-RGLS algorithm when the innovation length p = 1. By expanding the innovation vectors e 1 (t ) and e 2 (t ) in the M-D-RGLS algorithm into a large innovation vector E 1 (p, t ) and an innovation matrix E 2 (p, t ) in the M-D-MI-RGLS algorithm, the data information of the system is used repeatedly, the accuracy of the parameter estimation is improved.

Example
Consider the following multivariate pseudo-linear autoregressive system:    In simulation, y(t ) = y 1 (t ) y 2 (t ) ∈ R 2 is the output vector, v 2 (t ) ∈ R 2 is a white noise vector with zero mean, σ 2 1 and σ 2 2 are the variance of v 1 (t) and v 2 (t). Taking the noise variances σ 2 1 = 0.20 2 and σ 2 2 = 0.30 2 , using the M-D-RGLS algorithm (i.e. the M-D-MI-RGLS algorithm with p = 1) and the M-D-MI-RGLS algorithm with p = 2 and p = 3 to estimate the parameters of this example system, we obtain the parameter estimates and their errors δ := θ (t ) − ϑ / ϑ shown in Table 1. The parameter estimation errors versus t are shown in Figure 4.
From Table 1 and Figure 4, we can draw the following conclusions: 1. The parameter estimation errors of the M-D-RGLS and the M-D-MI-RGLS algorithms become smaller with the data length t increases. 2. Under the same noise variances, the M-D-MI-RGLS algorithm has higher accurate parameter estimates than the M-D-RGLS algorithm. 3. Introducing the innovation length p can effectively improve the parameter estimation accuracy of the M-D-RGLS algorithm, as the innovation length p increases, the parameter estimates are getting more stationary.

Conclusions
This paper derives an M-D-RGLS algorithm and an M-D-MI-RGLS algorithm for identifying the multivariate pseudo-linear autoregressive system based on the decomposition technique and the multi-innovation identification theory. The simulation results indicate that the proposed algorithms are all effective and the M-D-MI-RGLS algorithm provide more accurate parameter estimation than the M-D-RGLS algorithm. The identification idea of the proposed method can be extended to study the parameter estimation problems of other scalar or multivariate, linear or nonlinear systems with coloured noises and applied to other fields.

Disclosure statement
No potential conflict of interest was reported by the authors.