PIC32 UART buffer overrun in MIDI receiver application

Question

I have a system that consists of both a MIDI transmitter and MIDI receiver, that are separate units and connected between each other with cables. A PIC32MX6xx-series microcontroller forms the heart of each of these two units/sub-systems. The transmitter generates MIDI messages of notes played on an old mechanical musical keyboard that has been MIDI-fied. Each note-on/off event consists of 3 bytes of data, as per the official MIDI standard:

• MIDI channel - byte 1
• Note number - byte 2
• Velocity - byte 3

This data is sent over the PIC32's UART at the standard MIDI baud rate of 31250 bps. Furthermore, the UART conifugration is the commonly used 8 data bits, no parity, 1 stop bit. Of course, the receiving PIC32's UART is also configured exactly the same. Upon receiving the MIDI messages, the receiver performs some operations based on the received MIDI message and some other user inputs/configurations. The whole process works mostly fine even with successive notes that are played very fast.

However, the problem comes in when a large enough chord is played and the notes from the keyboard are pressed down exactly at the same time, typically with a chord size of 5 notes and larger. What happens is that most of the MIDI messages received are processed correctly, but the UART peripheral freezes at what seems to be the last received (perhaps partial) MIDI message. Thereafter, no more MIDI messages are received. An important aspect to mention is that the transmitter still transmits all the MIDI messages as it should - no problem there at all.

Now, what I do know and found out after a lot of searching on this type of issue, is that a UART buffer overrun error occurs. I know this because the UxSTAbits.OERR bit is set when this freezing phenomenon occurs. When I clear this bit, the UART operation continues as per normal. My conclusion is thus that the MIDI messages are transmitted and received faster than they are actually read and processed by the receiver, causing a buffer overrun due to the data in the FIFO buffer not being read fast enough. Consequently, this causes a loss of data, since the UART peripheral freezes while there are still MIDI messages being sent by the transmitter.

What I would like to ask: How can bypass this issue? I know the UART receive buffer is 8 levels deep. From my initial understanding, this would mean one character received for each level. However, from the datasheet it seems that a total of 13 characters can be received before the OERR bit is asserted by the PIC. Therefore, my understanding of these levels are probably wrong:

This would make some sense with the larger than 4-note chord issue, since 4 chords consist of 4x 3-byte messages = 12 bytes that are received (and processed fine), and a 5-note chord consist of 5x 3-byte messages = 15 bytes that are received, where the problem starts to occur.

This is how I handled the UART reception until now:

void main(void)
{ 
char dataRx[1] = {0x00};

/* --------------------------------------- */
/* Initialization code and other code here */
/* --------------------------------------- */

/* UART reception */
U1STAbits.OERR = 0; /* This happens only once */
while (1)
{        
    /* UART receive and constructing event string */        
    while (U1STAbits.URXDA == 0);
    getsUART1(1, dataRx, 0);

    /* Process the received data */
    processIncomingByte(dataRx);
} /* while */
} /* main() */

The function processIncomingByte() determines which byte(s) of the complete MIDI message have been received thus far, and performs some operations after a complete MIDI message is received. It determines that one byte at a time, as the bytes are received over the UART.

One option that comes to mind is to somehow clear the FIFO buffer immediately after each MIDI message is received (and copied into a software-defined buffer for processing), so that a buffer overrun should (in theory) never occur. I have tried this, but it didn't solve the problem and the buffer still overruns as described. I have tried it as follows:

void main(void)
{
int rxCount = 0;
char dataBuf[3] = {0, 0, 0};

/* --------------------------------------- */
/* Initialization code and other code here */
/* --------------------------------------- */

/* UART reception */
U1STAbits.OERR = 0; /* This happens only once */
while (1)
{   
    while (U1STAbits.URXDA == 0);
    rxCount++; /* Increment counter as soon as byte is received on UART */

    /* After 3 bytes are received, i.e. a complete MIDI message */
    if (rxCount == 3)
    {
        rxCount = 0; /* Reset counter */
        getsUART1(3, dataBuf, 0);
        U1STAbits.OERR = 0; /* clear the FIFO buffer */
        processIncomingMessage(dataBuf); /* Process the received data */
    } /* if */
} /* while */
} /* main() */

The function processIncomingMessage() in this case immediately performs the intended operations, since a complete MIDI message is passed to it in this case.

I suspect that the processing time for all the MIDI messages received are just too long to read new incoming MIDI messages when large chords are played (with notes depressed at exactly the same time), since everything works fine, even with rapid successive single notes. Therefore, another option that comes to mind is to simply increase the PIC's clock frequency. Currently it is set at 8 MHz. Could it possibly solve this issue if I just increase the PIC's clock frequency? Processing of the messages would happen much faster, while keeping the baud rate the same.

Although I am not bound by the standard MIDI baud rate of 31250 bps, I want to avoid changing the baud rate as far as possible. In any case, this will probably not solve the problem anyway. Increasing the time between transmitting MIDI messages might work, but that could introduce a tell-tale delay as more and more MIDI messages have to be transmitted in a specified time period.

Another possible solution would be to use a software-defined FIFO buffer as described in this article. However, I cannot figure out how to properly clear the hardware FIFO buffer after each byte is received on the UART in order to utilize this software-defined FIFO buffer. If I figured that out, I could just as well clear the hardware FIFO buffer after each MIDI message (3 bytes) is received, as described earlier in the second code snippet.

I know this is quite a mouthful, but I tried to describe my problem as clear and complete as I possibly can.

Any help in this regard would be greatly appreciated. This is a rather urgent situation.

Thanks in advance.

Answer 1

The datahseet example shows an overrun after 13 bytes because the software already managed to read the first 5 bytes. (This is similar to what's happening in your case.)

Processing three bytes at once is unlikely to help because processIncomingByte() probably does not actually do much when there is not yet a complete message. (And please note that there are other MIDI messages that do not have three bytes, and that it is possible to use running status, so waiting for three bytes would not work if you wanted to parse MIDI from other sources.)

If your code is too slow, increasing the clock would help. So would optimizing your code.

Your best bet would be a large software FIFO. As Chris said, you should empty the hardware FIFO from the interrupt so that it never can fill up, but it might work if you empty the hardware FIFO completely whenever you detect some byte(s) (it appears the time needed for processing one message is short enough that the 8-byte FIFO does not yet overflow).

For reproducible testing, change the sender to send fixed chords.