Do you typically need to program-in a USB communication protocol?

I have usb communication set up between a microcontroller I have and a VB.Net script. I am somewhat familiar with how the USB protocol works.

My question is: Do I need to program in start frames, tokens, and handshaking, or is this all done by the device hardware? I see on Microsoft's vb wiki that the serialport.Readline method reads a buffer up to the first new line character, and to me that looks like it does more than the USB protocol allows for. You shouldn't be able to continuously stream data to the port, it should be sent discretely in frames.

• Note that you quite often do have to think about frames if you're programming USB on the microcontroller. The simpler ones also abstract this away from you with a separate USB-serial chip, but more advanced ones let you do it yourself if e.g. you want to make a USB keyboard. Aug 4 at 7:47
• I am confused :-( A microcontroller is a physical device with an electrical interface (USB in this case). A VB.net script is a text file. How could you possibly set "communication" between a device and a text file?. Aren't you forgetting something to specify? Like which host your cable is connected to? What is running on the host side? Aug 11 at 6:52
• A better way of phrasing my question would be that I have USB communication set up between my microcontroller and my PC via a VB.Net script. I wanted to include the method in which I'm communicating because it was relevant to the question. @Ale..chenski Aug 11 at 13:27

Like any protocol stack, USB incorporates components representing various layers. High-level drivers and libraries provide higher-layer functionality on top of lower layers, effectively abstracting those lower layers away and out of sight.

In the case of USB, there are a number of so-called device classes which provide standardized interfaces on top of USB, with a coordinated protocol on the host side (implemented by a driver or a stack of drivers) and on the peripheral side (implemented by firmware or drivers). Examples of these device classes include HID devices like keyboards/mice, mass storage devices such as flash drives, image storage using PTP/MTP, and so on. These abstract away the USB protocol - for example, the FAT filesystem driver used to mount a flash drive sees a block storage device and doesn't need to worry about USB framing, packetization, etc, thanks to the USB drivers lower in the stack.

In your case, the relevant class is USB CDC. The CDC drivers present a serial port to user-land programs on your computer, which can read and write a stream of data while the underlying device class host driver handles framing it and doing all the other lower-level USB stuff. Because the specification is standardized, you benefit from knowing that your application on top of USB CDC should work regardless of what vendor your CDC-compatible device is from (as long as they follow the spec), because your system's drivers are likely implemented to the spec1

Meanwhile, your application can use the serial port without needing to worry about what physical hardware underpins it. It could be serial over USB, it could be a nine-pin D-sub connector serial port implemented with a UART on your motherboard's IO controller, or it could perhaps be a serial link on top of TCP/IP.

1 or more loosely, to give leeway for devices that may not follow the spec exactly.

• I think I get it now, I was under the assumption that I would a tleast have to adhere to a protocol format when reading/writing. I'm an electrical engineer so I'm not used to ignoring the lower level when programming (usually in C). Thank you for the help. Aug 2 at 20:49
• @Jaywalk: In fact "read until newline" is not part of the serial port abstraction. Serial port operations, which are what you get from the USB protocol stack going up to CDC and the corresponding drivers in the computer, gives you control over things like FIFO buffer size, total timeout, intercharacter timeout, etc. ReadLine is something that the library/framework is adding on top, and making a lot of compromises in the process (such as converting from bytes to characters -- did you remember the encoding -- and not really knowing what timeouts to configure to minimize latency) Aug 2 at 21:01
• So being aware of the profile-level operations can definitely help you build a better bespoke protocol (for example, length-prefixing will perform with a lot lower latency than termination by newlines or NULs or other control characters) because when you read the length and issue a fixed-size read to the serial port, it's sending a command down the USB bus to your device to "immediately forward the receive buffer when these many bytes are available"... you aren't dependent on an intercharacter gap in the transmission to cause a timeout. Aug 2 at 21:02
• Specifically the read until newline functionality is part of the abstraction for Terminal devices. These were originally console services provided by mainframes to typewriters and human interaction with typewriters are line oriented. This norm was used by hardware engineers later on to implement interfaces for devices like modems that use line-oriented AT commands because it was simpler for a hardware engineer to think that way than it is to implement complicated frame-oriented protocols like TCP/IP.. Aug 3 at 5:19
• .. today there's still lots of serial-oriented devices that use the ancient AT style commands even if their transport layer have evolved. The read line functionality is mostly there as a convenience for hardware and software developers. A lot of hardware actually don't deal directly with USB frames but output UART serial streams to something like an FTDI or a CH340 chip so all the USB packet stuff are just black boxes to hardware and software developers. Aug 3 at 5:23

It's already in the name of the function: serialport.Readline. This function operates on serial ports, not on raw USB endpoints. Your microcontroller "pretends" to be a serial port that's attached to your computer via USB. A serial port driver on your system takes care of converting the data from your program into USB packets (and vice-versa).

There's more software on your computer than just your own program. The operating system does a lot of stuff for you.

If you want to write your own driver, you can of course do that, but it's quite difficult to say the least. As long as serial communication is enough, you don't have to write one. However, if you want to create your own custom USB endpoints, you will have to write a driver for them.

• I won't bother trying to reinvent the wheel, I just didn't understand how it worked at the lower level. With the fact that it's emulating a serial port in mind the serialport methods make a lot more sense. Thank you for the help. Aug 2 at 20:50

If you have a USB serial port, then to your program, it's a serial port.

And you don't need to know or care if it is a standard serial port, USB serial port, virtual serial port extended over LAN, or something else, as long as your program sees a serial port.

• Thank you for the info and that's how I'm going to proceed. I was so caught up in how USB protocol works that I didn't realize that it might have been abstracted away. Aug 2 at 20:52

My question is: Do I need to program in start frames, tokens, and handshaking, or is this all done by the device hardware?

Ok, let me try to paint a broad picture of what you are asking about, in up-down way:

1. Your script makes calls to USB driver stack;
2. USB driver sets a linked list of data structures in PC memory, various "ring" and "transaction buffers", per requests from the script;
3. USB driver then loads the starting address for the structure and "rings" a "doorbell register" in USB host controller hardware.
4. USB host controller HARDWARE uses DMA (Direct Memory Access) to walk through the linked list of structures, on its own;
5. The HC (host controller) hardware interprets the incoming data structures and sends sequences of raw data to USB PHY - physical layer translator. Host HW follows USB protocols per SIE - Serial Interface Engine (sends proper tokens and waits/evaluates bus responses (handshakes), sending and receiving data packets to/from PHY), and calculates/verifies CRC.
6. PHY arranges parallel data into serial, appending SYNC and EOP to tokens, and performs data encoding for proper balancing of data toggling;
7. Host hardware continuously generates frame tokens on enabled ports, and SW stack watches the frame timing and schedules transaction packets in accord with bus rules;
8. Devices implement USB protocol in a similar form of SIE, receiving/analyzing packets, decoding addresses/endpoints, and responding in proper and timely manner.

With a rare exception of Low Speed protocol (like mouse/keyboard, where some enthusiasts managed to implement the USB SIE and PHY in bit-banging manner, loosely), all microprocessors with USB functionality implement all major protocol functions in hardware.

So, to answer this direct question, no, you don't program start of frames and specific tokens or wait for handshakes, it is all done in hardware under USB driver stack control.