How to deal with signed int overflows

Brief background first; I have data from CAN bus of a steering angle which is obviously in hex. The steering angle covers two bytes of a message. The specification document I have says that those two bytes form a 16-bit signed int which are the steering angle which a prescaler of 1/1024, that's all I have got (I don't have access to the source). What I am trying to do is to convert those hex values into signed int, however I am not sure how to do it correctly.

A small section of the CAN message within short time period (we are focusing on byte 2 and 3):

can0  700   [8]  00 00 99 93 55 0B EF BD
can0  700   [8]  00 00 95 95 10 0C 17 BE
can0  700   [8]  00 00 6F 97 FB 0A 17 BE
can0  700   [8]  00 00 39 99 5C 0A 40 BE
can0  700   [8]  00 00 AD 9A 62 08 EF BD
can0  700   [8]  00 00 EF 9B B5 08 40 BE
can0  700   [8]  00 00 CA 9D 9A 09 17 BE
can0  700   [8]  00 00 3E 9F 55 09 40 BE
can0  700   [8]  00 00 91 A0 ED 09 17 BE


As far as I know, typically, data in CAN messages follow this format: one byte for the actual data, one byte for the number of overflows.

For example, let's take an unsigned int of a value of 2000. Assuming byte #0 is for overflows, byte #1 is for actual data, we get:

CAN message -> [07, D0, x, x, x, x, x, x]


07 indicating that there have been 7 overflows, D0 indicating the remainder is 208, therefore:

7*255 + 208 = 2000


I understand how to do it with unsigned values. But this time in my scenario I am dealing with signed values. I am assuming one byte is for overflows, one byte is for the remainder, however I am not sure.

1. How are overflows calculated for signed values? Is it overflow += 1 when value > 127 and overflow -= when value < -128? Does it even make sense for overflows to have signedness?

2. How can I convert these bytes into signed decimal in C/C++? Let's say my byte value in hex is 91. Last time I tried storing it in int and printed it out, it printed out 145 (normal binary) and not -111 (2's complement). How can I enforce 2's complement (if that makes sense) in my code?

3. Could I be interpreting the byte format wrong? All it says that these two bytes represent the steering angle and that the steering angle is int16_t. I have monitored them change in real time and one of them changes erratically, almost like jumping from 00 to FF, whereas the other one is increasing/decreasing slowly and almost linearly with time. Ideas?

I am really stuck here, I don't even know if I am going in the correct direction.. Any suggestions and help are really appreciated!

• "Overflows" are only considered over the entire value range for the word size. Higher bytes within the same word are just that, higher bytes. – Ignacio Vazquez-Abrams Apr 14 '18 at 18:21
• "Lower" and "higher" depends on the endianness of the data, but you (for instance) don't have 0x12 0x34, you have 0x1234. – Ignacio Vazquez-Abrams Apr 14 '18 at 18:32
• No offense, but the whole question does not inspire confidence in me that you know what you are talking about. My suspicion continues to be that the values passed to you are actually binary and not hex. You are displaying them in hex because they can't be displayed in binary format since the binary is not printable. Your question is about how to manipulate single byte binary data into a correct signed integer two bytes wide. – mkeith Apr 14 '18 at 18:37
• It is never done with overflows. It is a single number. – Ignacio Vazquez-Abrams Apr 14 '18 at 18:44
• Overflow is what you get when the magnitude of the numerical result is too large for the data type of the variable where the result is supposed to be stored. – mkeith Apr 14 '18 at 21:25

Clearing up some misunderstandings will probably help.

First, your data is a 16-bit value. There's no "overflows" and "actual data" -- the 16 bits are just divided into two 8-bit pieces (bytes). To get the right binary value, you need to concatenate the bytes. In C, you can do it by starting with unsigned values and using bitwise operators, like this:

uint16_t highbyte, lowbyte, data;
highbyte = get_can_byte();   //Do whatever you normally do to get the bytes
lowbyte = get_can_byte();

data = highbyte<<8 | lowbyte;


Now you have the correct 16-bit value. If you want the result to be signed, you can simply cast the value to a signed type:

int16_t signed_data;

signed_data = (int16_t)(highbyte<<8 | lowbyte);


1. Overflows produce the same binary values regardless of whether your variable is signed or unsigned. For example, 0x7fff + 1 == 0x8000. Whether you interpret 0x8000 as 32768 or -32768 depends on the data type. (Note that signed integer overflow is technically undefined in the C standard -- this is what your CPU will do.)

2. As I said, it all depends on the data type:

uint16_t ui = 0xffff;    //65535
int16_t   i = 0xffff;    //   -1

3. What you're describing sounds reasonable for a high-precision measurement. A change of +1 or -1 in your upper byte represents only 1/256 of your total steering range. The low byte will be extremely sensitive to the exact angle.

Note that it's best to convert to a signed value as late as possible, and to do it with an explicit cast. Things like bitwise operators can behave differently or produce undefined behavior when used on signed values. In general, whenever you're doing bit manipulation, use unsigned values.

• See Jonk's comment on my answer. I believe C does not guarantee that the cast from unsigned to signed will do the right thing. However, I believe it will actually work on most modern architectures with modern compilers. If the OP wants to do it portably, I would suggest asking on a C forum, not here. – mkeith Apr 14 '18 at 19:11
• I thought about adding a pedantic standards-compliant version, but I don't want to confuse the poor questioner. :-) If they manage to find a machine where it fails, they'll discover it very quickly. To my mind, if you're mucking around with number formats and endianness-dependent code, you've given up universal portability anyway. – Adam Haun Apr 14 '18 at 19:16
• Yeah, Adam. Agree. – mkeith Apr 14 '18 at 19:43
• Thank you for a detailed and clear explanation. You've shown me the right direction and I believe it's clear to me on what to do next, thank you! – Shibalicious Apr 14 '18 at 23:10

You're over-complicating this.

can0  700   [8]  00 00 *99 93* 55 0B EF BD -> 0x9399 -> -27751
can0  700   [8]  00 00 *95 95* 10 0C 17 BE -> 0x9595 -> -27243
can0  700   [8]  00 00 *6F 97* FB 0A 17 BE -> 0x976F -> -26769
can0  700   [8]  00 00 *39 99* 5C 0A 40 BE -> 0x9939 -> -26311


etc ...

• Long day of work, my thoughts were messy. Thank you for this, I over-complicated the entire thing :) – Shibalicious Apr 14 '18 at 23:10

The most likely explanation for why one byte seems random and one is changing linearly is that the linear byte is the higher order byte (the overflows). The lower order byte appears random because it is changing rapidly compared to the update rate.

I would say that byte 3 is the high order byte and byte 2 is the low order byte. In signed integers using two's complement representation (which is almost universal nowadays) negative numbers will have high order bits set to 1. Looking at the hex, if the high order byte is greater than 0x80, it is a negative number. So in your example, all the numbers are negative numbers. 0xffff is -1, 0xfffe is -2, etc. What you have in your snippet is:

• 0x9399
• 0x9995
• 0x976f
• etc...

Since all those numbers are greater than 0x8000, they are all negative.

The best thing for you to do is to study up on two's complement representation of signed integers. It sounds like you know how to do uint16_t conversion. What may work for you is to simply do that, then assign the value to a variable of int16_t. The compiler might convert it correctly for you (it is not guaranteed by the C specification, but many compilers do it that way).

I hope this helps you a bit.

• For same-sized storage, C guarantees conversion of signed int to unsigned int. (Which doesn't sound like the direction the OP wants.) C does NOT guarantee conversion of unsigned int to signed int -- it only guarantees it where the unsigned int value has representation in the signed int (positive and fits.) Larger values can be turned into random garbage so far as the standard allows. Of course, no C compilers does that so far as I'm aware. But according to the spec, they could if they wanted to. Just as you say, I think. – jonk Apr 14 '18 at 18:57
• @jonk, Yeah. I believe it is implementation defined. Might be undefined. But it works most of the time on two's complement machines. Doing it in such a way that it is fully portable is a pain in the ass. Could convert to unit32_t, then do some tests on the unsigned number and convert it into the correct int16_t without any implementation defined steps. But it is a pain. – mkeith Apr 14 '18 at 19:01
• You are completely correct, byte 2 is the lower byte, byte 3 is the higher one (it's a little-endian system). Yours and all the other comments have cleared the things a lot to me, it's much simpler than I thought it was gonna be. To be honest I have no idea where I got the idea of number overflows being stored in the byte nearby. It kinda makes sense and gives same results but the concept is totally different, overflows are performed on the entire value, rather than lower byte. Anyway, I know what I am doing now, thank you for your patience! – Shibalicious Apr 14 '18 at 23:16
• Glad my comments helped! A lot of us have been doing this stuff for a long time so it is now easy. But we have all had to figure it out along the way, usually with help from people who have done it before. It's never easy at first. You have to figure it out. – mkeith Apr 15 '18 at 1:47

I know nothing about actual steering angle sensors in cars ...,

but just be aware that an angle sensor may report multiple turn values. i.e. one turn might be 0-1024, and the sensor can report +/-32 turns in a 16bit number.

Why would anyone do this?

Well imagine the steering wheel is exactly at the 0/360 degree point, and flipping between 0 and 360 with road vibration. What happens if you average to filter noise out? You get 180 degrees which is completely wrong.

This is always an issue that has to be handled somewhere when using free rotating angle sensors. Some systems return two angle values at 90 degrees (sin+cos) to resolve this ambiguity. Others extend the angle value to 360+180 degrees with hysteresis, while some do multi-turn angle acummulation.

• Thank you for the point, I will keep this in my mind once I do the actual conversion and look at the plot! – Shibalicious Apr 14 '18 at 23:19

The other answers do a good job of explaining the message formatting, but I'd probably grab it like this:

union
{
struct
{
uint8_t hi;    //the order of these two depends on the endian-nesss of your specific micro
uint8_t lo;    //swap if the data is garbled
};
uint16_t all;      //or sint16_t if you like: the only difference so far is at what value it wraps to the opposite end of its range

{
}


The advantage here is that the apparent computation (shift + or) is actually done entirely by the memory structure with no real effort at all, so even a stupid compiler will still generate efficient code. And you're directly writing it the way that it actually happens, which I think is a big plus for readability.

If you do this a lot, you can:

typedef union
{
struct
{
uint8_t hi;
uint8_t lo;
};
uint16_t all;
} byte_int;

byte_int steering;
byte_int throttle;
//etc.

• The problem with this solution is that the code turns endianess-dependent, and the CPU endianess is not necessarily the same as the protocol endianess. Therefore, the superior solution is to use bit shifts. It is a bad idea to write endianess-dependent code when you can get endianess-independent code instead, for little to no extra CPU overhead. – Lundin Apr 16 '18 at 8:10
• @Lundin Yes, it does become endian-dependent, and the protocol could be different from the CPU. (the latter is why we copy one byte at a time from the protocol) But I've seen lots of projects that never leave their original platforms (endianess doesn't change) and lots of idiot compilers. For example, this very operation - 16-bit shift by 8 and then bitwise or - is exactly what I've seen the object code do: it loops 8 times through a compiler library function to shift 16 bits by 1 and then it or's one byte at a time. – AaronD Apr 16 '18 at 15:33
• If the platform does change to the opposite endianess, even my version has exactly one place to fix it if you use the typedef. – AaronD Apr 16 '18 at 15:36
• It is rare that whole projects leave the platform, but individual source files frequently do - or otherwise you are doing it wrong, re-inventing the wheel over and over. – Lundin Apr 17 '18 at 7:02
• @Lundin I put my typedefs in a separate header file, which is one of two that correspond to the specific chip. (the other has a bunch of chip-specific definitions for I/O and basic functionality) Then the source code above does become portable. – AaronD Apr 17 '18 at 19:18

If you had a decimal number like 106, you might say it had one overflow, where you went past 99 and "overflowed" once to 100. You might agree that this would be better called a "carry". You do understand this, because in your post you multiplied 7 * 256 before adding 208 to get the answer of 2000. (You wrote 255 but I think that's a slip since the answer was right).

You didn't say if you were working in C or any other language. It's possible you are just staring at a CAN dump and working things out on paper. From this point of view, the simplest way to an answer is that if the number is negative (meaning, 2^15 (32768) or greater), you can subtract 2^16 (65536) from the reading to get the answer. That is, if you had F8 33, and you calculated F8 * 256 + 33, you'd get 63539. Then, 63539 - 65536 = -1997, which is the value represented by F833 when it's considered a signed number.

The information presented in the the other answers is valid and mostly recommended, but this can get you started, and might be appropriate for someone who hasn't tackled the programming end of things yet. It's also a good way to check the answer your program gives you, once you try it.