When modelling a system you must first estimate the form of the transfer function (order, time delay etc.), and then estimate the parameters (coefficients for the transfer function).

A common way to estimate these parameters is to formulate a cost function describing the accuracy of the estimates, and then minimise this cost function with some numerical or analytical optimisation method. Two common forms of the cost function are the Equation Error and the Output Error. These are shown for a discrete time system below:

Output Error: e(k)

u(k) and y(k) are the actual measured system input and output and y(k)hat is the model output:

$$e(k) = y(k) - \hat{y(k)}$$

$$e(k) = y(k) - \frac{\hat{b_1}z^{-1}}{1+\hat{a_1}z^{-1}}u(k)$$

And the cost function is:

$$J = \sum_{k=1}^{N}e(k)^2$$

This can be solved numerically, e.g with gradient decent or the genetic algorithm.

Equation Error: E(k)

$$E(k) = y(k) - \hat{y(k)}$$

$$E(k) = y(k) +\hat{a_1}y(k-1)-\hat{b_1}u(k-1)$$

$$J = \sum_{k=1}^{N}E(k)^2$$

This can be solved analytically by differentiating J with respect to each parameter and equating the result to zero, and then solving the resulting set of equations.

It is said that the Output Error is better for noisy data but the equation error is better otherwise since it does not suffer the local minima problem.

My Question:

It seems that by using the Output Error or Equation error the cost function has different properties, but to me they seem to be fundamentally using the exact same formula, but just re-arranged to look different. So why does the cost function end up with different properties? I.e one is solvable and one is more robust to noise.

This is what I mean by saying they use the exact same formula:

The equation error is defined as the actual measured output minus the model output:

$$E(k) = y(k) - (-\hat{a_1}y(k-1)+\hat{b_1}u(k-1))$$

Where the model output is:

$$\hat{y(k)} = -\hat{a_1}y(k-1)+\hat{b_1}u(k-1)$$

But if you re-arrange this formula so it is in transfer function form:

$$\hat{y(k)} = -\hat{a_1}y(k)z^{-1}+\hat{b}z^{-1}u(k)$$

$$\hat{y(k)}(1+\hat{a_1}z^{-1}) = \hat{b}z^{-1}u(k)$$


Which is exactly the same as the Output Error. So the Output Error formula and the Equation Error formula are the same equation but just re-arranged to look different.

So why is the cost function for one solvable and for the other it is not?

And why does the result of optimisation result in a more noise-robust model when using the cost function with one and not the other?

Sorry if this is posted on the wrong site. There is no stack exchange site for control engineering, so I figured this is the next best place.

  • \$\begingroup\$ You can't mix domains (z and k) in equations, please clarify. \$\endgroup\$ – Chu Apr 24 '15 at 16:43
  • \$\begingroup\$ FYI, there is a more general engineering.stackexchange.com where this might fit better. EE.SE mods can help you migrate the question over there if tht's what you want (use the flag feature to get their help). \$\endgroup\$ – The Photon Apr 24 '15 at 18:30
  • \$\begingroup\$ General Engineering doesn't seem to have as great a following among control systems engineers as Electrical Engineering. This is indeed the best place to post and where it will receive attention. @Chu is right. You are mixing delay operators with indices for difference equations. Rethink your math. \$\endgroup\$ – docscience Apr 29 '15 at 15:38

The equations are different.

The fact that you can reconduce one to the other with some manipulation shouldn't surprise you. You are just proving that they refer to the same system's model. But one solves a linear problem, the other a strongly non-linear one. With different benefits in the outcome (as you already anticipate).

This paper examines the differences between the two criteria, and answers your question in detail.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.