I am making an application that has multiple i2c sensors controlled by an Arduino Uno. I want to get all of the raw data from the sensors and transfer them to a PC as fast as possible, on which I will perform calculation with the raw data using python. My plan is to write the arduino data onto a text file and then have python read that data and perform some calculations immediately after. I want to transfer the data as quickly as possible so it comes as close to Real Time results as possible. Should I use Ethernet, USB, or something else for comms? Should I use a raspberry Pi3 for this instead? Am I approaching this problem incorrectly? Any help would be appreciated! Thank you.
FTDI FT232RL serial-usb chips can run at 2Mbd , i.e. 200kbytes/sec. This is much faster than the I2C rate.
Say you read I2C at max rate of 400kHz (44kByte/s), you only need 887kBd to send the data back as hex-ascii (2 chars/byte).
Of course that is the sustained I2C read rate once addressed. In practice you have start/stop,addressing, control register writes, arduino code, etc etc. So your actual ability to get data to send back is probably only half that. You are likely to find that 230kbd or 460kbd are quite adequate.
So you are good with the standard usb-serial chip, just change baud rate. However, the arduinos serial routines might not be up to it.(not an arduino user myself).
Your starting place is to get the serial running a test. https://arduino.stackexchange.com/questions/296/how-high-of-a-baud-rate-can-i-go-without-errors
Realterm is able to set these (any) baud rates, and capture the data to file, timestamp, run postprocessing scripts etc. It also has a capability to convert and display hexcsv data (specifically from an I2C device)
Note that at high baud rates the Arduino might need to you to space chars you send to it. You might also find that to get max send rate, you need to handle the TX uart directly to bypass any software fifos, which can be pretty time consuming. The good news is that you don't have to buffer, when the uart sends so fast.
What you want to do will work fine.
You are correct to capture to file then process. It is much easier to get going, and to fault find later. If you have a choice, ASCII chars are easier than binary. A hexcsv format is easy for what you want to do.
A good tip is you can zip these data files if you want to keep them, they always compress massively.
The uno doesn't support this, but newer arduino models do away with the external USB to serial chip and instead directly connect the USB port to the microcontroller. I think by default it emulates a serial port over USB, but since it's not actually a serial port it's not limited to serial port baud rates and as a result will transfer data much faster. Might be worth looking at as you should be able to get much better performance.
Would you like 5.3Mbit/sec (5333333 Baud) ? Here is how to:
Maybe you will run to problem to have ready so much data at this rate :)