# PIC32 UART buffer overrun in MIDI receiver application

I have a system that consists of both a MIDI transmitter and MIDI receiver, that are separate units and connected between each other with cables. A PIC32MX6xx-series microcontroller forms the heart of each of these two units/sub-systems. The transmitter generates MIDI messages of notes played on an old mechanical musical keyboard that has been MIDI-fied. Each note-on/off event consists of 3 bytes of data, as per the official MIDI standard:

• MIDI channel - byte 1
• Note number - byte 2
• Velocity - byte 3

This data is sent over the PIC32's UART at the standard MIDI baud rate of 31250 bps. Furthermore, the UART conifugration is the commonly used 8 data bits, no parity, 1 stop bit. Of course, the receiving PIC32's UART is also configured exactly the same. Upon receiving the MIDI messages, the receiver performs some operations based on the received MIDI message and some other user inputs/configurations. The whole process works mostly fine even with successive notes that are played very fast.

However, the problem comes in when a large enough chord is played and the notes from the keyboard are pressed down exactly at the same time, typically with a chord size of 5 notes and larger. What happens is that most of the MIDI messages received are processed correctly, but the UART peripheral freezes at what seems to be the last received (perhaps partial) MIDI message. Thereafter, no more MIDI messages are received. An important aspect to mention is that the transmitter still transmits all the MIDI messages as it should - no problem there at all.

Now, what I do know and found out after a lot of searching on this type of issue, is that a UART buffer overrun error occurs. I know this because the UxSTAbits.OERR bit is set when this freezing phenomenon occurs. When I clear this bit, the UART operation continues as per normal. My conclusion is thus that the MIDI messages are transmitted and received faster than they are actually read and processed by the receiver, causing a buffer overrun due to the data in the FIFO buffer not being read fast enough. Consequently, this causes a loss of data, since the UART peripheral freezes while there are still MIDI messages being sent by the transmitter.

What I would like to ask: How can bypass this issue? I know the UART receive buffer is 8 levels deep. From my initial understanding, this would mean one character received for each level. However, from the datasheet it seems that a total of 13 characters can be received before the OERR bit is asserted by the PIC. Therefore, my understanding of these levels are probably wrong:

This would make some sense with the larger than 4-note chord issue, since 4 chords consist of 4x 3-byte messages = 12 bytes that are received (and processed fine), and a 5-note chord consist of 5x 3-byte messages = 15 bytes that are received, where the problem starts to occur.

This is how I handled the UART reception until now:

void main(void)
{
char dataRx[1] = {0x00};

/* --------------------------------------- */
/* Initialization code and other code here */
/* --------------------------------------- */

/* UART reception */
U1STAbits.OERR = 0; /* This happens only once */
while (1)
{
/* UART receive and constructing event string */
while (U1STAbits.URXDA == 0);
getsUART1(1, dataRx, 0);

/* Process the received data */
processIncomingByte(dataRx);
} /* while */
} /* main() */


The function processIncomingByte() determines which byte(s) of the complete MIDI message have been received thus far, and performs some operations after a complete MIDI message is received. It determines that one byte at a time, as the bytes are received over the UART.

One option that comes to mind is to somehow clear the FIFO buffer immediately after each MIDI message is received (and copied into a software-defined buffer for processing), so that a buffer overrun should (in theory) never occur. I have tried this, but it didn't solve the problem and the buffer still overruns as described. I have tried it as follows:

void main(void)
{
int rxCount = 0;
char dataBuf[3] = {0, 0, 0};

/* --------------------------------------- */
/* Initialization code and other code here */
/* --------------------------------------- */

/* UART reception */
U1STAbits.OERR = 0; /* This happens only once */
while (1)
{
while (U1STAbits.URXDA == 0);
rxCount++; /* Increment counter as soon as byte is received on UART */

/* After 3 bytes are received, i.e. a complete MIDI message */
if (rxCount == 3)
{
rxCount = 0; /* Reset counter */
getsUART1(3, dataBuf, 0);
U1STAbits.OERR = 0; /* clear the FIFO buffer */
processIncomingMessage(dataBuf); /* Process the received data */
} /* if */
} /* while */
} /* main() */


The function processIncomingMessage() in this case immediately performs the intended operations, since a complete MIDI message is passed to it in this case.

I suspect that the processing time for all the MIDI messages received are just too long to read new incoming MIDI messages when large chords are played (with notes depressed at exactly the same time), since everything works fine, even with rapid successive single notes. Therefore, another option that comes to mind is to simply increase the PIC's clock frequency. Currently it is set at 8 MHz. Could it possibly solve this issue if I just increase the PIC's clock frequency? Processing of the messages would happen much faster, while keeping the baud rate the same.

Although I am not bound by the standard MIDI baud rate of 31250 bps, I want to avoid changing the baud rate as far as possible. In any case, this will probably not solve the problem anyway. Increasing the time between transmitting MIDI messages might work, but that could introduce a tell-tale delay as more and more MIDI messages have to be transmitted in a specified time period.

Another possible solution would be to use a software-defined FIFO buffer as described in this article. However, I cannot figure out how to properly clear the hardware FIFO buffer after each byte is received on the UART in order to utilize this software-defined FIFO buffer. If I figured that out, I could just as well clear the hardware FIFO buffer after each MIDI message (3 bytes) is received, as described earlier in the second code snippet.

I know this is quite a mouthful, but I tried to describe my problem as clear and complete as I possibly can.

Any help in this regard would be greatly appreciated. This is a rather urgent situation.

• The usual approach would be something like firing an interrupt at each received character, and in that fully draining the hardware fifo to the software one (though ideally there would only be one character to move at at that point). Worry about what constitutes a "message" later, in the code that pulls data out of the software fifo to work on it. Trying to wait until several characters are received is not impossible, but substantially complicates things. – Chris Stratton Oct 19 '16 at 8:08
• @Chris: Thanks for your comment and advice. So, do you mean that I write an ISR for the UART receive interrupt (UxRX or UxRXIF) and in that ISR I just copy the FIFO contents to a software buffer and clear the interrupt flag? For reading the hardware FIFO's contents, do I just read the 8 bytes from the fifo by specifying 8 bytes in the first parameter of getsUART1()? Do I also clear the FIFO buffer by clearing U1STAbits.OERR bit? – wave.jaco Oct 19 '16 at 10:20
• See my comment at the accepted answer. Chris's suggestion of using an interrupt was a key factor in the solution of this problem. – wave.jaco Oct 26 '16 at 22:37

The datahseet example shows an overrun after 13 bytes because the software already managed to read the first 5 bytes. (This is similar to what's happening in your case.)

Processing three bytes at once is unlikely to help because processIncomingByte() probably does not actually do much when there is not yet a complete message. (And please note that there are other MIDI messages that do not have three bytes, and that it is possible to use running status, so waiting for three bytes would not work if you wanted to parse MIDI from other sources.)

If your code is too slow, increasing the clock would help. So would optimizing your code.

Your best bet would be a large software FIFO. As Chris said, you should empty the hardware FIFO from the interrupt so that it never can fill up, but it might work if you empty the hardware FIFO completely whenever you detect some byte(s) (it appears the time needed for processing one message is short enough that the 8-byte FIFO does not yet overflow).

For reproducible testing, change the sender to send fixed chords.

• Thank you for your answer. I see what you mean by the 5 bytes explaining the 13 bytes indication in the datasheet, but does that then mean the 8-level FIFO can store 16 bytes? I went through the datasheet and UART reference manual and couldn't find something mentioning 16 bytes. I just understand it as the 8-level FIFO can contain a maximum of 8 bytes. Am I understanding it wrongly? Indeed, processIncomingByte() does not do much but keep track of the byte count for the received partial messages, and populates a struct with the received data, according to the type of message sent. – wave.jaco Oct 19 '16 at 10:24
• ...continued from previous comment: Thanks for the heads-up on the MIDI message lengths. I am aware of that and have accounted for that in my code. The solution to the 3-byte message situation will also be implemented for other message types/lengths. I will increase the clock speed and see if that helps, but I would rather solve the issue with a software FIFO. Thanks also for the idea of sending fixed chords. That will certainly make developing the solution easier. – wave.jaco Oct 19 '16 at 10:27
• An eight-byte FIFO cannot store more than eight bytes, but this is only for bytes that have not yet been read. If eight bytes arrive, and five are read, then it needs five more bytes to be filled up again. – CL. Oct 19 '16 at 10:52
• That makes sense. I will try a software FIFO approach when I am at the code again later today and report back. – wave.jaco Oct 19 '16 at 13:15
• Using a software FIFO buffer (of 100 bytes) populated upon reception of each character through an interrupt service routine, as well as increasing the system and peripheral clock frequencies to 40 MHz solved the problem (which is a huge leap in performance compared to 8 MHz!). Thanks for all the assistance! Much appreciated! – wave.jaco Oct 26 '16 at 22:36