How to configure DMA to receive variable length USART messages?

Question

I am trying to receive USART messages on a microcontroller originating from PC which will command it to execute certain tests. I am using STM32F4, I chose to use DMA as messages on the same USART originating from the microcontroller would need to be at 2000000bps as I couldn't make it work with lower bitrate or without DMA. I am using binary rather than ASCII for both directions.

What I have designed so far "works" (updates real time almost) provided the rate of messages being sent is constant as each new message forces previous one to be flushed from the DMA register. If initiating though and having only a single instruction/USART frame being sent which happens to be less than the amount of data to be received for the HAL_UART_Receive_DMA(), message is stuck until pushed out by the next message.

An even worse case scenario would be if USART_take_size is 100 and there is a single 10 byte frame in the DMA buffer. In this case I would need to wait for another 9 (10 byte) frames before receiveing them all at once.

This is what my callback currently looks like.

void HAL_UART_RxCpltCallback(UART_HandleTypeDef *huart) {
    
    //              uart handle, global array, numb of bytes to trigger interrupt       
    HAL_UART_Receive_DMA (huart, raw_serial, USART_take_size);
    
    // function that adds received bytes to the circular buffer 
    AppendSerial(raw_serial,USART_take_size);
            
}

A thread pops bytes from the circular buffer, loops over them and based on SOF and EOF bytes extracts the message inside the frame.

My problem is that the frames that arrive are of varied length, if USART_take_size is set to be x and frame arrives and the length of it is less than x, it will have to sit in the buffer array in the extracting thread until another frame arrives which makes necessary bytes appear for the parser to obtain the embedded message.

I see three ways of solving this and each doesn't seem right:

1. Set USART_take_size to 1, which kind of defeats the whole point of DMA

2. Have a fixed frame size and pad content with zeros before EOF if content is lesser than the USART_take_size. This just seems sloppy and wasteful. It would also need to trigger on every frame even though the overall rate at that instance could be much higher.

3. Configure the DMA to trigger ISR on variable number of bytes received, provided there has been a certain amount of inactivity preceding it.

I do not know much about DMA, and while it seems to be a trade off between how many ISR get triggered and what minimum message you can read I think think there should be a way receive message after a period of inactivity. Any other approach of how to solve this would be great. Thank you!

Answer 1

I had similar problem in one of my projects. I solved it by creating long enough buffer to store longest possible message and using DMA to transfer from USART. To detect end of message I used idle line detect interrupt which trigger when USART receiver stop receiving data.

Answer 2

The STM32 DMA cannot do what you want by itself. To deal with variable length messages, it would need to be able to parse the message intelligently and determine the length using whatever encoding scheme you have decided on. This is well beyond its capabilities.

I can only really think of two options for your problem:

1. Just process each byte in the USART receive ISR. The STM32 has pretty low interrupt overhead. If your application can handle it then sometimes it is best to just go with simple, even though it might not be as elegant.

2. You can easily configure the DMA to operate as a fifo buffer. The DMA will load bytes into the buffer and you then set up a periodic timer interrupt to check the buffer for data and parse messages. This is an inbetween approach compared to option 1 in that it allows you to reduce interrupt overhead at the expense of latency. You can tune the timer polling period against fifo size to balance your particular real time processing needs. There are also tricks you can do with the DMA level triggers, though the usefulness of this will depend on your message format.