0
\$\begingroup\$

I designed a device which includes many sensors. These sensors are constantly sending information, every 400 ms, do there are series of data which reach the microcontroller (STM32F103). I want to send the data to a computer by using WiFi (I tried ESP8266).

As I'm a beginner in programming I chose HAL functions and dealing with them is really good.

enter image description here

But I'm facing many problems. For example, for sending data over serial I prefer using synchronous mode, but I was finally forced to use asynchronous mode (I didn't see any examples on the Web for USART in synchronous mode for the STM32F103C8T6 board).

My big problem is that the received data isn't only int8_t; part of them are uint32_t and others are int16_t or uint16_t.

I put them into two arrays, uint32_t data1[10] and uint16_t data2[8] but can't send them by HAL_UART_Transmit (UART_HandleTypeDef*huart, **uint8_t** ***pData,** uint16_t Size, uint32_t Timeout) or HAL_UART_Transmit_DMA(), etc.

despite trying many ways the codes didn't work

enter image description here these are my data but couldn't send them

by searching throgh the net I found suggestion of using Char enter image description here honestly i didn't get the point. how can send two arrays in this way and how change int16 or uint32 to char.....

Can anyone to explain with details how I can send these data online regularly and completely?

I know that the STM32F103 isn't a good option for online projects, but the important thing for me is sending data correctly and regularly; I am just trying to learn.

\$\endgroup\$
7
  • \$\begingroup\$ But you can and must send the data just like you said. I don't understand why you can't. The integer is just 4 bytes. \$\endgroup\$
    – Justme
    Aug 12, 2022 at 5:14
  • 1
    \$\begingroup\$ Supposed you're programming in C: Please do some research on casting a pointer to multi-byte values to a pointer to single-byte values. Make sure you understand what you're doing, blindly using some random source will give you headaches. -- As your question seems to show, you have multiple issues, but unfortunately SE/EE is not the place to get an all-in-one answer. \$\endgroup\$ Aug 12, 2022 at 6:04
  • 1
    \$\begingroup\$ This is more of a C programming question. Note that you're trying to communicate between two different machines so questions of alignment/padding and endianness may bite if you take a naïve approach using unions or whatever. \$\endgroup\$ Aug 12, 2022 at 6:13
  • \$\begingroup\$ thank you @thebusybee I'll do \$\endgroup\$
    – Mary
    Aug 12, 2022 at 6:37
  • 1
    \$\begingroup\$ dear @Justme as i said I'm really confused regarding this matter and don't know why can't send data . I'll try to share codes and what I received \$\endgroup\$
    – Mary
    Aug 12, 2022 at 6:40

3 Answers 3

1
\$\begingroup\$

This is more of a transmission problem than a serial problem; and as comments suggest, more of a C, or CS, question than embedded per se. (It's not a good fit here, but I'm going to take the lack of close votes over the last almost 2 years as tacit support.)

As such, there are multiple problems underlying the basic function: transporting data. Some of these problems may be trivial (just dump it into a port!), others may bear consideration. And, in the best/worst case (most reliable, but also heaviest overhead), elaborate protocols may be built to solve it: TCP/IP for example, a means of transmitting data reliably and in-order between clients on an unreliable medium.

Without going into detail (also, because communications are hardly my specialty..!), some examples should give some idea why you might want to make such considerations.

Raw

You could simply dump the data out the serial port.

What happens if the receiver picks up in the middle of a transmission? How long do you wait between packets? -- Is it formed as packets, or repeating continuously? What if the serial device becomes unavailable (or the data source feeding it), what happens if the data is interrupted in the middle?

Even if your data are fixed size, it's possible that the source and destination get out of sync, so it helps to have some manner of synchronizing information inserted into the data stream.

Synchronization

This can be explicitly in-band, where data are coded and intermixed with control codes; or out-of-band, such as through timing, provision of separate control signals, etc.

For example, TIA/EIA-232-F defines enable and handshake signals (though stopping short of specifying data formats; EIA-574 however, I think does make such definitions), used to announce whether the server or client is present/active, ready to send or receive, errored, etc..

Modbus, in a typical implementation, uses 8-bit bytes on an asynchronous serial medium; a wait period between packets is defined, so that all bytes within a packet must be transmitted with less than this delay between, and packets must be separated by at least this delay to ensure all receivers reset to start-of-packet state before beginning again.

A typical character-driven interface might use line-end characters to convey the end of a command, or reported data, etc. With a little prettyprinting, such formats can be made highly human-readable on a terminal display, and with use of ANSI control codes, a full interactive terminal can be constructed. This is convenient for low-rate data, low enough to be read by eye. Downsides include a large and variable overhead from the hex or decimal formatting, spacing, ANSI control codes, etc., making it harder to parse (human/machine readability are often at odds with each other).

Still other protocols might provide transparent reliable communication; TCP/IP for example can be used as an ideal (sequential, bidirectional) pipe between clients; the downside is unreliable timing, and, in case the connection itself goes down, out-of-band signals (as status variables or API calls/callbacks) must be used to indicate channel state and various statistics (ping, packet loss, error rate, connection, handshaking, etc.). Packets can also be sent and received as such, facilitating block-transfer protocols without need for control characters.

Format

Formatting is another consideration. Just for some examples: binary data might be coded to make room for control codes; this can be as basic as restricting the range of data (say 0...254, leaving 255 as a terminator perhaps), or adding an escape code (say 0...253 is normal, 254 means read the next byte as literal, 255 is terminator), etc. We might further restrict ranges so that only printable characters (32-127) are used, reserving 0-31 for control codes, and 128-255 for extended characters; this basically describes ASCII, UTF-8 and various other text formats. We might further restrict data to a convenient subset, to obtain base64 encoding for example, a staple interchange format of modern HTTP.

Further reading:
Consistent Overhead Byte Stuffing | Wikipedia
Rows of String Values (RSV Data Format) Specification | Stenway | GitHub

We might also encode data for reliability purposes: encoding N bytes in N+M bytes to provide redundancy, and therefore error detection or correction. Hamming and other codes might be a consideration here. (Asynchronous serial itself typically includes a parity function, a simple media-level version of this.) Or, as packet prefix or suffix, a checksum, CRC, signature, etc. can be used to check the data against; Modbus and USB use a CRC suffix, for instance. Data could also be "scrambled" to improve spectral properties (reducing periodicity, removing peaks in the RF signal spectrum; this is a typical component of over-the-air serial links), compressed to save [average] bandwidth, encrypted for security, etc.


Now equipped with some background, we can return to the heart of your concern.

Offhand, without any extenuating circumstances (unreliable communications, variable packet length, etc.), I would guess that you would be well served by a fixed-length packet, with a fixed delay (dead time, inactive line) between packets for synchronization. Consistent timing is generally quite feasible on embedded platforms like STM32 and ESP32, and a typical realization will require one USART (optionally fed by DMA), one timer peripheral, and probably a timer, USART or DMA interrupt (inclusive-or).

We can handle synchronization by setting a fixed time delay. Combined with baud rate, and update rate, this puts a hard limit on total data transmission capacity. Choose baud rate suitable to send everything that needs to be sent at the same time.

With a fixed packet size, coding is trivial: we can transmit raw binary. Both sides understand how many bytes to accept, and reject any beyond that limit (protip: do actually check this, to avoid buffer overruns and undefined behavior; received data should be considered unreliable and untrusted).

As for the programming interface: one of the best ways to format data for transmission (or serialization for file storage, etc.) is to pack it into a struct (particularly if the data consist of many disparate variables, structures and arrays), or flattened into a char array. I mean -- ultimately it will be read as a char array, that's what data the serial port requires, but the value of a struct is we impose structure upon that array, and C guarantees we can read it via typecast to a char array of length sizeof(struct foo).

Note that it is an open question whether we order word-sized variables as low-high or high-low bytes; you must match the endianness of both platforms to ensure this is not left ambiguous. Consider using the htons, ntohs, etc. family of functions to ensure correct endianness on all platforms.

Also be careful about variable size: use defined-width types (uint16_t, etc.). If using structs, ensure consistent byte/word packing, and other alignment concerns; refer to your compiler's/architecture's manual for specifics. See also: https://stackoverflow.com/questions/119123/why-isnt-sizeof-for-a-struct-equal-to-the-sum-of-sizeof-of-each-member

With consistent structure at both ends of the comm channel, data can be sent and received basically in-place; though you may want to provide buffering to ensure all values are updated simultaneously, and not overwritten with partial, misaligned or otherwise corrupted packets. Alternately, an event-driven data path might be used, where only successful reception triggers further action.

\$\endgroup\$
3
  • \$\begingroup\$ Well thought and thorough answer, expands quickly to introduce very advanced concepts without too much details. I do have to mention that RS-232 doesn't really use any handshake lines for signaling errors in data or link. With the exception of maybe carrier detect for modems. And indeed the suggestion to use structs is debunked by what you said, different compilers handling structs differently, even without taking endianess and struct packing into account. So memory of structs can't be sent as-is unless they happen to be identical on both ends. For example bit-fields may not be compatible. \$\endgroup\$
    – Justme
    Mar 24 at 8:49
  • \$\begingroup\$ More to say: don't assume the compiler will do the same thing in any case, but specify the data layout explicitly. This can be done in any compiler AFAIK, but how exactly, may vary. (Or if it's standard C to do so, I'm not that in-depth.) That's what I'm getting at. I feel I'd rather deal with the rigors of struct packing than hand-packing byte fields in an array; what a poorly-maintainable mess. I'm pretty sure bit fields are nonstandard C?, so would be best avoided (hand-pack them instead). \$\endgroup\$ Mar 24 at 9:32
  • \$\begingroup\$ It may not be possible to specify the data layout to work under all scenarios. I have used packed structs and even bitfields, but that's because I know the PC and MCU projects use compilers that are compatible on these two architectures. But you can't expect it to work on all compilers and architectures, for example some architectures will throw an exception if you try to access a 16-bit integer on an odd address so you can't have an odd amount of bytes before a 16-bit word, and must hand-craft the data structures anyway. Serialization is easy and you can always use tools to generate the code. \$\endgroup\$
    – Justme
    Mar 24 at 11:13
0
\$\begingroup\$

You can't use strlen() on the data array because you are not sending characters of a null terminated text string.

You always want to send as many bytes as the data array contains regardless of what the binary data stored there is.

Use sizeof() instead, or simply write how many bytes you want to send.

\$\endgroup\$
2
  • \$\begingroup\$ thanks dear @Justme , I get what you said and know that shouldn't use strlen . that picture was just an example which I found. and even though I chenged that in a way which can use in my codes but didn't work and still there are errors about data and sending that \$\endgroup\$
    – Mary
    Aug 14, 2022 at 5:07
  • \$\begingroup\$ We don't know what errors and if they are on transmit or receive side and how you interpret the binary data back to numbers or text. So if yoy say something does not work as expect, please mention what do you see and how does it differ from what you expect. The transmitting of binary data should work now if you fixed it, but we also don't know if and how you fixed it. \$\endgroup\$
    – Justme
    Aug 14, 2022 at 16:47
0
\$\begingroup\$

You can create a buffer that holds the all values then store the all sensor values in this buffer as uint8_t type.

To store different types of sensor values such as 16bit, 32bit or floating, you should make type-casting to uint8_t. This is called data packing. When you read the buffer in receiver MCU, you should unpack the buffer to convert sensor values as actual. The important thing is you must unpack the sensor values in the same sequence in which you packed them.

There are packing and unpacking functions for unsigned 16bit, 32bit and floating types. You can create signed ones.

For unsigned 16bit values, you can use this function to pack and unpack values,

void pack_u16(uint8_t *buffer, uint8_t *packet_count, const uint16_t data_u16)
{
    buffer[(*packet_count) + 0] = ((uint8_t*)&data_u16)[0];
    buffer[(*packet_count) + 1] = ((uint8_t*)&data_u16)[1];

    *packet_count+= sizeof(data_u16);
}

void unpack_u16( const uint8_t *buffer, uint8_t *packet_count,  uint16_t *data_u16)
{
    ((uint8_t*)data_u16)[0] = buffer[(*packet_count) + 0];
    ((uint8_t*)data_u16)[1] = buffer[(*packet_count) + 1];

    *packet_count+= sizeof(veri_u16);
}

For unsigned 32bit values, you can use this function to pack and unpack values,

void pack_u32(uint8_t *buffer, uint8_t *packet_count, const uint32_t data_u32)
{
    buffer[(*packet_count) + 0] = ((uint8_t*)&data_u32)[0];
    buffer[(*packet_count) + 1] = ((uint8_t*)&data_u32)[1];
    buffer[(*packet_count) + 2] = ((uint8_t*)&data_u32)[2];
    buffer[(*packet_count) + 3] = ((uint8_t*)&data_u32)[3];

    *packet_count+= sizeof(data_u32);
}

void unpack_u32(const uint8_t *buffer, uint8_t *packet_count, uint32_t *data_u32)
{
    ((uint8_t*)data_u32)[0] = buffer[(*packet_count) + 0];
    ((uint8_t*)data_u32)[1] = buffer[(*packet_count) + 1];
    ((uint8_t*)data_u32)[2] = buffer[(*packet_count) + 2];
    ((uint8_t*)data_u32)[3] = buffer[(*packet_count) + 3];

    *paket_sayaci += sizeof(data_u32);
}

For floating values, you can use this function to pack and unpack values,

void pack_float( uint8_t *buffer, uint8_t *packet_count, const float data_float)
{
    buffer[(*packet_count) + 0] = ((uint8_t*)&data_float)[0];
    buffer[(*packet_count) + 1] = ((uint8_t*)&data_float)[1];
    buffer[(*packet_count) + 2] = ((uint8_t*)&data_float)[2];
    buffer[(*packet_count) + 3] = ((uint8_t*)&data_float)[3];

    *packet_count+= sizeof(data_float);
}

void unpack_float( const uint8_t *buffer, uint8_t *packet_count, float *data_float)
{
    ((uint8_t*)data_float)[0] = buffer[(*packet_count) + 0];
    ((uint8_t*)data_float)[1] = buffer[(*packet_count) + 1];
    ((uint8_t*)data_float)[2] = buffer[(*packet_count) + 2];
    ((uint8_t*)data_float)[3] = buffer[(*packet_count) + 3];

    *packet_count+= sizeof(data_float);
}

Using example:

for sending;

#define MAX_BUFFER_LEN 255
uint8_t transmit_buffer[MAX_BUFFER_LEN];

uint8_t packet_index = 0;
uint16_t sensor_u16 = 12345;
uint32_t sensor_u32 = 123456789;
float sensor_f32 = 15.56;

pack_u16(transmit_buffer, &packet_index, sensor_u16);
pack_u32(transmit_buffer, &packet_index, sensor_u32);
pack_float(transmit_buffer, &packet_index, sensor_f32);

for receiving;

#define MAX_BUFFER_LEN 255
uint8_t receiving_buffer[MAX_BUFFER_LEN];

uint8_t packet_index = 0;
uint16_t sensor_u16;
uint32_t sensor_u32;
float sensor_f32;

read_UART_buffer(receiving_buffer); // pseudo function for read UART buffer.

unpack_u16(receiving_buffer, &packet_index, sensor_u16);
unpack_u32(receiving_buffer, &packet_index, sensor_u32);
unpack_float(receiving_buffer, &packet_index, sensor_f32);
\$\endgroup\$
6
  • \$\begingroup\$ Those fuctions may work in this case but in the end you are accessing multi-byte variables and copying them byte at a time via byte pointer to another memory location. I think same would be better achieved with simple memcpy() or for portability reasons the serialization should be done properly instead of accessing variables via byte pointers. Or just use a packed struct and send it, but a struct with all short integers may already be packed so packing may be unnecessary. \$\endgroup\$
    – Justme
    Aug 14, 2022 at 15:25
  • \$\begingroup\$ You are right. You can do it by using memcpy(). But generally, I use these functions in any application that includes serial communication and these are in my serial communication module. It is a design concept. @Justme \$\endgroup\$
    – huseyinkoc
    Aug 14, 2022 at 15:46
  • \$\begingroup\$ I just wanted to mention that your way of doing it might be not very efficient or portable so it may not be a particularly good example how to do things. For example it does not handle endianess, but if you define your serial data stream to use whatever endianess the MCU has, that's fine when using same functions on devices with same endianess, but you need to have different serialization and deserialization functions on a device with opposite endianess. \$\endgroup\$
    – Justme
    Aug 14, 2022 at 15:59
  • \$\begingroup\$ You are right about endianness handling. I have forgot mentioning that it is used in little endianness platform. \$\endgroup\$
    – huseyinkoc
    Aug 14, 2022 at 16:11
  • \$\begingroup\$ OK. But I still think this does not address the original problem. The data was already serialized in an array, the problem was using strlen to get lentgh of the serialized data. \$\endgroup\$
    – Justme
    Aug 14, 2022 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.