Your problem is almost certainly not due to hardware. USB uses fixed bit rates. The bit rate on the line should be 12.000 Mbps +/-0.25% regardless of how much data you transmit. If your bit rate were incorrect, you would not be able to send any data at all, and you'd see error messages in Windows. The TM4C123 has a separate PLL just for the USB module. Here's part of a block diagram from the datasheet:

At the protocol level, make sure you're using bulk endpoints to transfer data, and try not to have other devices attached to that controller on the PC side. You can see this in Device Manager's connection view. This will allow a maximum of 19 64-byte bulk packets per frame (millisecond), giving 1216 kilobytes/second of application data. This is the fastest possible real data rate for full-speed USB.
There are two possible software bottlenecks in your system. The first is on the MCU side. If there's no data available when the host (PC) sends an IN packet, the MCU will NAK and the host will try again later. If your MCU is busy doing other things, it may take a long time to get into the USB interrupt. The easiest way to detect this is by using a bus analyzer. It'll cost you several hundred dollars for the hardware, but can save you a lot of headaches in the long run. I like Teledyne-LeCroy's analyzers, personally.
The other possible bottleneck is on the PC side. For some reason, Windows does not like to max out USB bandwidth. Using multiple endpoints with asynchronous transfers running in parallel works well. If you only have one data stream, try requesting much more data than you need when you set up the transfer. Again, a bus analyzer can tell you what's going on. You might also get useful information from a software USB analyzer that runs on the PC, but I've never tried those.