Skip to main content
added 14 characters in body
Source Link
Curd
  • 16.5k
  • 36
  • 46

I guess it's just a matter of precision.

Suppose the factor 2.36 is represented by a fixed point number with only 4 bits to the right of decimal point (fractional bits), you get:

56 * 2.36 in fixed point representation with only 4 fractional bits:
(56 * round(2.36 * 2^4)) / 2^4 = 
(56 * round(37.76)) / 16 =
(56 * 38) / 16 = 133

The solution would be just to use more fractional bits (maybe 8).

56 * 2.36 in fixed point representation with 8 fractional bits:
(56 * round(2.36 * 2^8)) / 2^8 = 
(56 * round(604.16)) / 2^8 =
(56 * 604) / 256 = 132

But still then it is possible for some factors to get the same effect, only less likely/frequent.

It only works always perfectly if the fixed point representation of the factor doesn't loose any bits by rounding.

This is the case if (and only if):

  • the fractional part of the factor is a multiple of 1/2^n and
  • the fixed point representation of the factor uses at least n fractional bits.

I guess it's just a matter of precision.

Suppose the factor 2.36 is represented by a fixed point number with only 4 bits to the right of decimal point (fractional bits), you get:

56 * 2.36 in fixed point representation with only 4 fractional bits:
(56 * round(2.36 * 2^4)) / 2^4 = 
(56 * round(37.76)) / 16 =
(56 * 38) / 16 = 133

The solution would be just to use more fractional bits (maybe 8).

56 * 2.36 in fixed point representation with 8 fractional bits:
(56 * round(2.36 * 2^8)) / 2^8 = 
(56 * round(604.16)) / 2^8 =
(56 * 604) / 256 = 132

But still then it is possible for some factors to get the same effect, only less likely/frequent.

It only works always perfectly if the fixed point representation of the factor doesn't loose any bits by rounding.

This is the case if:

  • the fractional part of the factor is a multiple of 1/2^n and
  • the fixed point representation of the factor uses at least n fractional bits.

I guess it's just a matter of precision.

Suppose the factor 2.36 is represented by a fixed point number with only 4 bits to the right of decimal point (fractional bits), you get:

56 * 2.36 in fixed point representation with only 4 fractional bits:
(56 * round(2.36 * 2^4)) / 2^4 = 
(56 * round(37.76)) / 16 =
(56 * 38) / 16 = 133

The solution would be just to use more fractional bits (maybe 8).

56 * 2.36 in fixed point representation with 8 fractional bits:
(56 * round(2.36 * 2^8)) / 2^8 = 
(56 * round(604.16)) / 2^8 =
(56 * 604) / 256 = 132

But still then it is possible for some factors to get the same effect, only less likely/frequent.

It only works always perfectly if the fixed point representation of the factor doesn't loose any bits by rounding.

This is the case if (and only if):

  • the fractional part of the factor is a multiple of 1/2^n and
  • the fixed point representation of the factor uses at least n fractional bits.
added 3 characters in body
Source Link
clabacchio
  • 13.6k
  • 4
  • 45
  • 80

I guess t'sit's just a matermatter of precissionprecision.

Suppose the factor 2.36 is represented by a fixed point number with only 4 bits to the right of decimal point (fractional bits), you get:

56 * 2.36 in fixed point representation with only 4 fractional bits:
(56 * round(2.36 * 2^4)) / 2^4 = 
(56 * round(37.76)) / 16 =
(56 * 38) / 16 = 133

The solution would be just to use more fractional bits (maybe 8).

56 * 2.36 in fixed point representation with 8 fractional bits:
(56 * round(2.36 * 2^8)) / 2^8 = 
(56 * round(604.16)) / 2^8 =
(56 * 604) / 256 = 132

But still then it is possible for some factors to get the same effect, only less likely/frequent.

It only works always perfectperfectly if the fixed point representation of the factor doesn't loose any bits by rounding.

This is the case iffif:

  • the fractional part of the factor is a multiple of 1/2^n and
  • the fixed point representation of the factor uses at least n fractional bits.

I guess t's just a mater of precission.

Suppose the factor 2.36 is represented by a fixed point number with only 4 bits to the right of decimal point (fractional bits), you get:

56 * 2.36 in fixed point representation with only 4 fractional bits:
(56 * round(2.36 * 2^4)) / 2^4 = 
(56 * round(37.76)) / 16 =
(56 * 38) / 16 = 133

The solution would be just to use more fractional bits (maybe 8).

56 * 2.36 in fixed point representation with 8 fractional bits:
(56 * round(2.36 * 2^8)) / 2^8 = 
(56 * round(604.16)) / 2^8 =
(56 * 604) / 256 = 132

But still then it is possible for some factors to get the same effect, only less likely/frequent.

It only works always perfect if the fixed point representation of the factor doesn't loose any bits by rounding.

This is the case iff

  • the fractional part of the factor is a multiple of 1/2^n and
  • the fixed point representation of the factor uses at least n fractional bits.

I guess it's just a matter of precision.

Suppose the factor 2.36 is represented by a fixed point number with only 4 bits to the right of decimal point (fractional bits), you get:

56 * 2.36 in fixed point representation with only 4 fractional bits:
(56 * round(2.36 * 2^4)) / 2^4 = 
(56 * round(37.76)) / 16 =
(56 * 38) / 16 = 133

The solution would be just to use more fractional bits (maybe 8).

56 * 2.36 in fixed point representation with 8 fractional bits:
(56 * round(2.36 * 2^8)) / 2^8 = 
(56 * round(604.16)) / 2^8 =
(56 * 604) / 256 = 132

But still then it is possible for some factors to get the same effect, only less likely/frequent.

It only works always perfectly if the fixed point representation of the factor doesn't loose any bits by rounding.

This is the case if:

  • the fractional part of the factor is a multiple of 1/2^n and
  • the fixed point representation of the factor uses at least n fractional bits.
added 106 characters in body
Source Link
Curd
  • 16.5k
  • 36
  • 46

I guess t's just a mater of precission.

Suppose the factor 2.36 is represented by a fixed point number with only 4 bits to the right of decimal point (fractional bits), you get:

56 * 2.36 in fixed point representation with only 4 fractional bits:
(56 * round(2.36 * 2^4)) / 2^4 = 
(56 * round(37.76)) / 16 =
(56 * 38) / 16 = 133

The solution would be just to use more fractional bits (maybe 8).

56 * 2.36 in fixed point representation with 8 fractional bits:
(56 * round(2.36 * 2^8)) / 2^8 = 
(56 * round(604.16)) / 2^8 =
(56 * 604) / 256 = 132

But still then it is possible for some factors to get the same effect, only less likely/frequent.

It only works always perfect if the fixed point representation of the factor doesn't loose any bits by rounding.

This is the case iff

  • the fractional part of the factor is a multiple of 1/2^n and
  • the fixed point representation of the factor uses at least n fractional bits.

I guess t's just a mater of precission.

Suppose the factor 2.36 is represented by a fixed point number with only 4 bits to the right of decimal point (fractional bits), you get:

56 * 2.36 in fixed point representation with only 4 fractional bits:
(56 * round(2.36 * 2^4)) / 2^4 = 
(56 * round(37.76)) / 16 =
(56 * 38) / 16 = 133

The solution would be just to use more fractional bits (maybe 8).

56 * 2.36 in fixed point representation with 8 fractional bits:
(56 * round(2.36 * 2^8)) / 2^8 = 
(56 * round(604.16)) / 2^8 =
(56 * 604) / 256 = 132

But still then it is possible for some factors to get the same effect, only less likely/frequent.

It only works always perfect if

  • the fractional part of the factor is a multiple of 1/2^n and
  • the fixed point representation of the factor uses at least n fractional bits.

I guess t's just a mater of precission.

Suppose the factor 2.36 is represented by a fixed point number with only 4 bits to the right of decimal point (fractional bits), you get:

56 * 2.36 in fixed point representation with only 4 fractional bits:
(56 * round(2.36 * 2^4)) / 2^4 = 
(56 * round(37.76)) / 16 =
(56 * 38) / 16 = 133

The solution would be just to use more fractional bits (maybe 8).

56 * 2.36 in fixed point representation with 8 fractional bits:
(56 * round(2.36 * 2^8)) / 2^8 = 
(56 * round(604.16)) / 2^8 =
(56 * 604) / 256 = 132

But still then it is possible for some factors to get the same effect, only less likely/frequent.

It only works always perfect if the fixed point representation of the factor doesn't loose any bits by rounding.

This is the case iff

  • the fractional part of the factor is a multiple of 1/2^n and
  • the fixed point representation of the factor uses at least n fractional bits.
Source Link
Curd
  • 16.5k
  • 36
  • 46
Loading