Biased compensation recursive least squares-based threshold algorithm for time-delay rational models via redundant rule

This paper develops a biased compensation recursive least squares-based threshold algorithm for a time-delay rational model. The time-delay rational model is transformed into an augmented model by using the redundant rule, and then, a recursive least squares algorithm is proposed to estimate the parameters of the augmented model. Since the output of the augmented model is correlated with the noise, a biased compensation method is derived to eliminate the bias of the parameter estimates. Furthermore, based on the structures of the augmented model parameter vector and the rational model parameter vector, the unknown time delay can be computed by using a threshold given in prior. A simulated example is used to illustrate the efficiency of the proposed algorithm.


Introduction
The rational system can describe a special kind of nonlinear models which is defined as the ratio of two polynomial expressions [1]. This type of nonlinear models widely exist in modern society, for example, in life science, in chemical engineering and in economic systems [2,3]. Compared with the polynomial nonlinear model, the rational model provides a very concise and parsimonious representation for complex nonlinear systems and has excellent extrapolation properties [4][5][6]. Although the rational model has many advantages over the polynomial model and is widely used in many fields, surprisingly, there is only scattered work reported in the literature on identification of rational models. The difficulty in identifying the rational models is that the denominator polynomial terms make the rational model nonlinear in both the parameters and the regression terms. Thus the traditional linear system and polynomial nonlinear system identification algorithms cannot be directly used to rational models [7,8].
Among various identification algorithms, the offline algorithms are the most widely used algorithms for rational models. For example, Zhu provided an implicit least square (LS) algorithm for rational models; the proposed off-line algorithm is efficient in dealing with the parameter estimation problems associated with nonlinear in the parameters models [9]. Mu et al. studied a globally consistent nonlinear LS estimator for identification of a nonlinear rational system, where the proposed off-line algorithm is the globally convergent algorithm for the nonlinear rational systems [10]. It has been noted that the off-line algorithms usually use batch data to update the parameters in real time, while the new data are not involved [11,12]. On the other hand, the online algorithms have less computational efforts and can update the parameters with new data [13,14]. Therefore, the extension of online algorithms to rational model identification is becoming a hot and promising spot of present research. Recently, Zhu proposed an error back propagation parameter estimation algorithm for a class of rational models; by combining an orthogonal correlation test method, the model structure and the associate parameters can be estimated simultaneously [15]. Zhu et al. also proposed an enhanced linear Kalman filter algorithm for the parameter estimation of nonlinear rational models, in which the proposed algorithm is an online algorithm [16]. However, all the rational models in the literature are non-time-delay systems, when the rational models have unknown time delays, those above mentioned methods are invalid.
Time-delay systems often exist in engineering practice [17][18][19]. For example, in the communication network, due to the network congestion, the signals which are transmitted over a communication channel often have time delays [20,21]; in chemical processes, some variables such as chemical component concentrations are often measured through laboratory analysis, which will introduce time delays [22,23]. Recently, there exist many identification algorithms for time-delay systems [24,25]. Zhao et al. developed a variational Bayesian approach for ARX models with a Markov chain time-varying time delays, where the unknown parameters and the varying time delays can be estimated iteratively [26]. Angelosante et al. proposed a sparse linear regression-based (SLR-based) iterative algorithm to estimate the parameters of FH signals [27]. Both the variational Bayesian and the SLR-based iterative algorithms are off-line algorithms. In order to estimate the time-delay systems based on online algorithms, Ma et al. provided a Kalman filter-based LS iterative and a recursive least squares (RLS) algorithms for time-delay Hammerstein systems with the assumption that the time delay is known in prior [28]. Chen et al. proposed a RLSbased redundant parameter method for time-delay systems, in which the time delay is unknown [29]. To the best of our knowledge, the related work has not been reported in the literature on identification of timedelay rational models. Thus the focus of this paper is to develop new identification algorithms for such models.
In this paper, a biased compensation recursive least squares-based threshold algorithm is proposed for a time-delay rational model via the redundant rule. The main objective is to apply the redundant rule to transform the rational model into an augmented model, whose parameters are estimated by using a biased compensation recursive least squares-based threshold algorithm. Since the estimated parameter vector contains redundant parameters, a threshold is introduced to pick out these redundant parameters. Then based on the structures of the augmented model parameter vector and the rational model parameter vector, the unknown time delay can be obtained. The main contributions are summarized as follows. Briefly, the rest of this paper is organized as follows. Section 2 introduces the rational model. Section 3 develops a biased compensation least squaresbased threshold algorithm. Section 4 studies a biased compensation recursive least squares-based threshold algorithm. Section 5 provides an illustrative example. Finally, concluding remarks are given in Sect. 6.

The rational model
Consider the following rational model, where y(t) is the output, e(t) a stochastic white noise with zero mean and variance σ 2 , τ is an unknown integer time delay, and a(t − τ ) and b(t − τ ) are expressed as The information vectors ϕ(t − τ ) and ψ(t − τ ) are the products of past inputs , and the structures of ϕ(t − τ ) and ψ(t −τ ) are known in prior, θ a and θ b are the unknown parameters to be estimated and can be expressed as Without loss of generality, assume b 1 = 1, then one can get in which Y (t) = y(t)b 1 ψ 1 (t − τ ), and ψ 1 (t − τ ) is the first element of the vector ψ(t −τ ). Rewriting Equation (4) as where Since the time delay τ is unknown, those elements in φ(t) are also unknown; then, the LS proposed in [10] and the enhanced linear Kalman filter algorithm proposed in [16] cannot be applied for this time-delay rational model. To overcome this difficulty the redundant rule proposed in [29] is used.
Clearly, l c,d , c = 1, . . . , τ, τ + 2, . . . , M, d = 1, . . . , n + m − 1 are the redundant parameters and the true values of these parameters are equal to zero, a 1 , . . . , a n , b 2 , . . . , b m are the true parameters, and all the terms in the information vector φ p (t) are known. Then Eq. (5) can be transformed into the following augmented model, It is observed from Eqs. (6) and (7) and the other is the true part (rational model parameter vector) which can be written as The idea of the redundant rule is first to pick out the redundant part from the augmented model parameter vector θ p by using a threshold given in prior and then compute the unknown time delay based on the structures of θ and θ p . For example, if the unknown time delay is 2, the order of the true parameter is n +m −1 = 5. Assume M = 4, when apply the LS-based threshold method to estimate the unknown parameter vector θ p . The estimate of the augmented model parameter vector isθ p = [0 5 , 0 5 ,θ , 0 5 ] T (0 5 = [0, 0, 0, 0, 0]), which means the position of the first nonzero value is 11. Thus we can get τ = 10/5 = 2.

The biased compensation least squares-based threshold algorithm
Collect t input, output and noise data, respectively, and define Then it follows that Based on the LS algorithm, one can get Define Substituting Eq. (10) into Eq. (11) getŝ Since y(t) is correlated with the noise e(t), Eq. (13) means that the proposed LS algorithm is a biased algorithm. It follows that an unbiased estimateθ u (t) can be expressed as, Unfortunately, Φ T p (t)V (t) cannot be obtained because of the unknown polynomial b(i −τ ) and unknown noise e(i) in Eq. (14). That is to say, in order to obtain the unbiased estimateθ u (t), one should first obtain the estimates of b(i − τ ) and e(i). At time t − 1, the unknown noise e(i), i = 1, 2, . . . , t can be estimated by using the parameter estimateθ u (t − 1), e.g., in which (x)|θ u (t−1) means the estimate of x by usinĝ θ u (t − 1). Unlike the work in [9,10,15], the colored noise v(t) in this paper contains unknown time delay τ , which leads b(i − τ ) also be unknown. Thus, to get the unbiased parameter estimate is more challenging in this paper. One will use the redundant rule to transform the rational model into an augmented model. Then Eq. (1) can be expressed as Equation (7) can be enhanced as Clearly, the true values of l c,d , c = 1, 2, . . . , τ, τ + 2, . . . , M, d = 1, . . . , n + m − 1 are all equal 0; thus, Eq. (18) is equivalent to Eq. (7). Since e(i) is a white noise with zero mean and is independent of the past inputs and outputs, Eq. (14) can be converted as follows, where and the unbiased parameter estimateθ u (t) can be computed aŝ From Eqs. (19) and (20), we can see that at time t, can be estimated by using the unbiased parameter estimatê θ u (t − 1) without the knowledge of the unknown time delay τ . Assume that at time t, we have gotten the unbiased estimateθ u (t),θ a 1 (t), . . . , a n (t), b 2 (t), . . . , b m (t), . . . , l M,1 (t), . . . , l M,n (t), l M,n+1 (t), . . . , Since once the parameter vector has been estimated, all the redundant parameters l c,d are equal zero, while the rational model parameter estimates can be estimated aŝ Furthermore, according to the structures ofθ andθ u , we can get the unknown time delay. Then the following biased compensation least squares-based threshold (BCLS-TH) algorithm can be summarized as follows, where k represents the count of BCLS-TH iteration. However, the BCLS-TH algorithm is an off-line algorithm which has harder computational efforts and cannot update the parameters when new data becomes available. In order to overcome this difficulty, a BCRLS-TH algorithm is introduced in the next Section.

The biased compensation recursive least squares-based threshold algorithm
According to Eqs. (11) and (12), one can get Then the biased compensation recursive least squares-based threshold (BCRLS-TH) algorithm can be summarized as follows, It can be seen from Eqs. (28), (31) and (32) that one should compute the time delayτ (t − 1) at each time t − 1 to obtain the noises estimatesê(i), i = 1, . . . , t. This will lead heavy computational efforts. In order to overcome this difficulty, the noise e(i) can be computed as follows, Remark 1 When using Eq. (33) to compute the noisê e(i), i = 1, . . . , t, we do not need to compute the time delayτ (t − 1) at each time t − 1.

Remark 2
In the BCLS-TH algorithm, we use the same input and output data to update the parameter vector at each iteration k; while in the BCRLS-TH algorithm, we update the parameter vector at each time t based on the new input and output data u(t), y(t) and the used data u(t − 1), . . . , u(1), y(t − 1), . . . , y(1); thus, the BCRLS-TH algorithm can update the parameters in real time when new data becomes available.
ε, then obtain theθ u (t) and go to next step; otherwise, increase t by 1 and go to step 3. 12. Assume each element ofθ u (t) isθ s u (t), s = 1, . . . , Mn + Mm − M, and compareθ s u (t) with : if |θ s u (t)| < , then pick it out and reduce the order of the parameter vectorθ u (t) by 1; otherwise, keep the order of the parameter vectorθ u (t) unchanged. 13. Obtainθ (t) based on step 12. 14. Compute the time delay τ based on the structures ofθ u (t) andθ (t).
Remark 3 In application, systems are often disturbed by noises, which leads to the estimates of the redundant parameters not be equal to zero. In order to get the time delay τ , a threshold should be given in prior. If the absolute value of a parameter estimate is smaller than , one can regard the parameter as a redundant parameter and pick it out. However, the choice of the threshold is difficult and challenging. A large may mistake some true parameters for redundant parameters, while a small may lead some redundant parameters not be picked out from the parameter vector [29].

Example
Consider a time-delay rational model with τ = 1, Then one can get )e(t).
the input {u(t)} is taken as a persistent excitation signal sequence with zero mean and unit variance, and {e(t)} is taken as a white noise sequence with zero mean and variance σ 2 = 0.10 2 .
First, apply the RLS-TH algorithm to estimate the parameters of the time-delay rational model. The estimation errors δ := θ L S − θ p / θ p versus t are shown in Fig. 1. The parameter estimates and the estimation errors are shown in Table 1.
Next, we utilize the BCRLS-TH algorithm to estimate the parameters of the time-delay rational model. The estimation errors δ := θ u − θ p / θ p versus t are shown in Fig. 1. The parameter estimates and the estimation errors are shown in Table 2. The rational parameters chosen by using different thresholds are given in Table 3. Assume = 0.08 and t = 1000. According to Tables 1, 2 and 3, we can get that the absolute values of the estimatesl 1,1 ,l 1,2 ,l 1,3 ,l 1,4 ,l 3,1 ,l 3,2 andl 3,3 by using the RLS-TH algorithm are all smaller than 0.08, then the estimated parameter vector by using the RLS-TH algorithm can be expressed aŝ  Clearly, in the RLS-TH algorithm, some redundant parameters cannot be picked out fromθ L S (l 1,5 , l 3,4 and l 3,5 ), while in the BCRLS-TH, all the redundant parameters can be picked out fromθ u and then the unknown time delay can be obtained based on the structures ofθ u andθ . Furthermore, from Table 1, we can conclude that there is no which can be used to pick out the redundant parameters fromθ L S because of the estimates l 1,5 and a 2 . A large will mistake a 2 for the redundant parameter. On the other hand, a small will lead to the redundant parameter l 1,5 not be picked out.  5 , a 1 , a 3 , b 2 , b 3  a 1 , a 3 , b 2 , b 3 0.09 l 1,5 , a 1 , a 2 , a 3 , b 2 , b 3 , l 3,4 a 1 , a 2 , a 3 , b 2 , b 3 (True) 0.08 l 1,5 , a 1 , a 2 , a 3 , b 2 , b 3 , l 3,4 , l 3,5 a 1 , a 2 , a 3 , b 2 , b 3 (True) 0.03 l 1,4 , l 1,5 , a 1 , a 2 , a 3 , b 2 , b 3 , l 3,4 , l 3,5 a 1 , a 2 , a 3 , b 2 , b 3 (True) 0.02 l 1,4 , l 1 ,5 , a 1 , a 2 , a 3 , b 2 , b 3 , l 3,4 , l 3,5  a 1 , a 2 , a 3 , b 2 , b 3 , l 3,4 Ultimately, we can draw the following conclusions.
1. Figure 1 shows that the BCRLS-TH algorithm is more effective than the RLS-TH algorithm. 2. Tables 1 and 3 declare that when apply the RLS-TH algorithm to estimate the parameters, there will be no threshold which can be used to pick out all the redundant parameters fromθ L S . Tables 2 and 3 witness that the threshold can be easily chosen in the BCRLS-TH algorithm, e.g., the threshold can be chosen in [0.03, 0.09] and the unknown time delay is 1.

Conclusions
A BCRLS-TH algorithm is proposed for time-delay rational models in this paper. By using the redundant rule, the rational model can be transformed into an augmented model which consists of two parts: one is the redundant part and the other is the true part. Then an unbiased parameter estimate can be obtained by the BCRLS-TH algorithm, and the unknown time delay can be computed based on the two structures of the augmented model parameter vector and the rational model parameter vector. Interesting extensions of this work can also be pursued in some real rational model estimation [9,30] where time delays are likely to occur. Furthermore, the proposed method in this paper can be extended to linear systems [14,31] and bilinear systems [32,33] and applied to other fields [34,35].
To the best of our knowledge, this is the first online algorithm proposed for time-delay rational models. As a new direction, there are some potential topics associated with this work. For example, if the structures of the denominator and numerator polynomials are unknown, how to estimate the parameters and the unknown time delay? Another topic is can this method be extended to rational models with varying time delays? These topics will remain as challenging issues in future.