When modelling a system you must first estimate the form of the transfer function (order, time delay etc.), and then estimate the parameters (coefficients for the transfer function).
A common way to estimate these parameters is to formulate a cost function describing the accuracy of the estimates, and then minimise this cost function with some numerical or analytical optimisation method. Two common forms of the cost function are the Equation Error and the Output Error. These are shown for a discrete time system below:
Output Error: e(k)
u(k) and y(k) are the actual measured system input and output and y(k)hat is the model output:
$$e(k) = y(k) - \hat{y(k)}$$
$$e(k) = y(k) - \frac{\hat{b_1}z^{-1}}{1+\hat{a_1}z^{-1}}u(k)$$
And the cost function is:
$$J = \sum_{k=1}^{N}e(k)^2$$
This can be solved numerically, e.g with gradient decent or the genetic algorithm.
Equation Error: E(k)
$$E(k) = y(k) - \hat{y(k)}$$
$$E(k) = y(k) +\hat{a_1}y(k-1)-\hat{b_1}u(k-1)$$
$$J = \sum_{k=1}^{N}E(k)^2$$
This can be solved analytically by differentiating J with respect to each parameter and equating the result to zero, and then solving the resulting set of equations.
It is said that the Output Error is better for noisy data but the equation error is better otherwise since it does not suffer the local minima problem.
My Question:
It seems that by using the Output Error or Equation error the cost function has different properties, but to me they seem to be fundamentally using the exact same formula, but just re-arranged to look different. So why does the cost function end up with different properties? I.e one is solvable and one is more robust to noise.
This is what I mean by saying they use the exact same formula:
The equation error is defined as the actual measured output minus the model output:
$$E(k) = y(k) - (-\hat{a_1}y(k-1)+\hat{b_1}u(k-1))$$
Where the model output is:
$$\hat{y(k)} = -\hat{a_1}y(k-1)+\hat{b_1}u(k-1)$$
But if you re-arrange this formula so it is in transfer function form:
$$\hat{y(k)} = -\hat{a_1}y(k)z^{-1}+\hat{b}z^{-1}u(k)$$
$$\hat{y(k)}(1+\hat{a_1}z^{-1}) = \hat{b}z^{-1}u(k)$$
$$\hat{y(k)}=\frac{\hat{b}z^{-1}u(k)}{(1+\hat{a_1}z^{-1})}$$
Which is exactly the same as the Output Error. So the Output Error formula and the Equation Error formula are the same equation but just re-arranged to look different.
So why is the cost function for one solvable and for the other it is not?
And why does the result of optimisation result in a more noise-robust model when using the cost function with one and not the other?
Sorry if this is posted on the wrong site. There is no stack exchange site for control engineering, so I figured this is the next best place.
