What exactly makes the SET_ADDRESS command fail in my situation?

Question

TLDR: Can USB packets be transmitted between host and device, even if a possible hardware problem exists as evidenced by a USB error -71?

I am working on a custom board that uses USB 2.0 HS to communicate with a host computer. Previously, my computer was able to enumerate my device and start communication--however, upon receiving pre-fabricated copies of my board, enumeration no longer happens, and I'm presented with a USB error -71.

When I plug my device into my computer, dmesg gives the following output:

[ 9058.933325] usb 3-4: new high-speed USB device number 22 using xhci_hcd
[ 9059.061745] usb 3-4: Device not responding to setup address.
[ 9059.269619] usb 3-4: Device not responding to setup address.
[ 9059.481349] usb 3-4: device not accepting address 22, error -71

Many forums that I've visited online blame device- or host-side hardware issues for the error mentioned above. However, this advice doesn't make sense to me in my situation, for reasons to come.

In an effort to debug , I've used Wireshark to capture USB packets being sent between the host and the faulty device. I've even compared the packets from the faulty device to working devices that use the same MCU I use on the board (for instance, a development board for the MCU) and the packets 'agree' up to a certain packet. Here are the outputted packets from Wireshark for the faulty board:

      1 0.000000       3.1.1                 host                  USB      67     URB_INTERRUPT in
      2 0.000012       host                  3.1.1                 USB      64     URB_INTERRUPT in
      3 0.000042       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
      4 0.000087       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
      5 0.000093       host                  3.1.0                 USBHUB   64     CLEAR_FEATURE Request  [Port 3: C_PORT_CONNECTION]
      6 0.000131       3.1.0                 host                  USBHUB   64     CLEAR_FEATURE Response [Port 3: C_PORT_CONNECTION]
      7 0.000136       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
      8 0.000173       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
      9 0.035198       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     10 0.035252       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     11 0.071208       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     12 0.071301       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     13 0.107180       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     14 0.107227       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     15 0.143176       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     16 0.143225       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     17 0.143337       host                  3.1.0                 USBHUB   64     SET_FEATURE Request    [Port 3: PORT_RESET]
     18 0.143378       3.1.0                 host                  USBHUB   64     SET_FEATURE Response   [Port 3: PORT_RESET]
     19 0.211191       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     20 0.211248       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     21 0.211255       host                  3.1.0                 USBHUB   64     CLEAR_FEATURE Request  [Port 3: C_PORT_RESET]
     22 0.211295       3.1.0                 host                  USBHUB   64     CLEAR_FEATURE Response [Port 3: C_PORT_RESET]
     23 0.271315       host                  3.0.0                 USB      64     GET DESCRIPTOR Request DEVICE
     24 0.271503       3.0.0                 host                  USB      82     GET DESCRIPTOR Response DEVICE
     25 0.271530       host                  3.1.0                 USBHUB   64     SET_FEATURE Request    [Port 3: PORT_RESET]
     26 0.271592       3.1.0                 host                  USBHUB   64     SET_FEATURE Response   [Port 3: PORT_RESET]
     27 0.339156       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     28 0.339220       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     29 0.339228       host                  3.1.0                 USBHUB   64     CLEAR_FEATURE Request  [Port 3: C_PORT_RESET]
     30 0.339266       3.1.0                 host                  USBHUB   64     CLEAR_FEATURE Response [Port 3: C_PORT_RESET]
     31 0.815223       host                  3.1.0                 USBHUB   64     CLEAR_FEATURE Request  [Port 3: PORT_ENABLE]
     32 0.815280       3.1.0                 host                  USBHUB   64     CLEAR_FEATURE Response [Port 3: PORT_ENABLE]
     33 0.815548       host                  3.1.0                 USBHUB   64     SET_FEATURE Request    [Port 3: PORT_RESET]
     34 0.815590       3.1.0                 host                  USBHUB   64     SET_FEATURE Response   [Port 3: PORT_RESET]
     35 0.870742       3.1.1                 host                  USB      67     URB_INTERRUPT in
     36 0.870758       host                  3.1.1                 USB      64     URB_INTERRUPT in
     37 0.883203       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     38 0.883274       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     39 0.927234       3.1.1                 host                  USB      67     URB_INTERRUPT in
     40 0.927243       host                  3.1.1                 USB      64     URB_INTERRUPT in
     41 0.951214       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     42 0.951273       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     43 1.159190       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     44 1.159250       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     45 1.175192       3.1.1                 host                  USB      67     URB_INTERRUPT in
     46 1.175196       host                  3.1.1                 USB      64     URB_INTERRUPT in
     47 1.367190       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     48 1.367251       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     49 1.423167       3.1.1                 host                  USB      67     URB_INTERRUPT in
     50 1.423184       host                  3.1.1                 USB      64     URB_INTERRUPT in
     51 1.575195       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     52 1.575256       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     53 1.575264       host                  3.1.0                 USBHUB   64     CLEAR_FEATURE Request  [Port 3: C_PORT_RESET]
     54 1.575308       3.1.0                 host                  USBHUB   64     CLEAR_FEATURE Response [Port 3: C_PORT_RESET]
     55 1.575313       host                  3.1.0                 USBHUB   64     CLEAR_FEATURE Request  [Port 3: PORT_ENABLE]
     56 1.575356       3.1.0                 host                  USBHUB   64     CLEAR_FEATURE Response [Port 3: PORT_ENABLE]
     57 1.575512       host                  3.1.0                 USBHUB   64     CLEAR_FEATURE Request  [Port 3: PORT_ENABLE]
     58 1.575555       3.1.0                 host                  USBHUB   64     CLEAR_FEATURE Response [Port 3: PORT_ENABLE]
     59 1.575566       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     60 1.575608       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     61 1.575613       host                  3.1.0                 USBHUB   64     CLEAR_FEATURE Request  [Port 3: C_PORT_CONNECTION]
     62 1.575655       3.1.0                 host                  USBHUB   64     CLEAR_FEATURE Response [Port 3: C_PORT_CONNECTION]
     63 1.575663       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     64 1.575704       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     65 1.611188       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     66 1.611251       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     67 1.647156       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     68 1.647212       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     69 1.683213       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     70 1.683272       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]
     71 1.719178       host                  3.1.0                 USBHUB   64     GET_STATUS Request     [Port 3]
     72 1.719229       3.1.0                 host                  USBHUB   68     GET_STATUS Response    [Port 3]

When I compared this output to that of a working development board, I noticed that the transactions take different directions around packet 31, when the host issues a CLEAR_FEATURE Request [Port 3: PORT_ENABLE]. Then it almost looks like the host tries to restart the USB communication with the faulty board all over again, and eventually I get the usb error -71 in dmesg (strangely enough, not in Wireshark).

What I don't understand is something likely very naive: if device not responding to setup address, usb error -71 generally is triggered by a hardware issue, why is my device able to maintain USB communication with my computer as shown above? Wouldn't a hardware issue imply that no USB packets would be able to be sent?

EDIT

A detailed breakdown of the packets I received in the above Wireshark log are included in the following repo:

https://github.com/cosmicchasm/Array_USB_Data

Answer 1

why is my device able to maintain USB communication with my computer as shown above?

The communication shown above is not with your device, it is between the system driver and runtime registers of your host controller. What actually happens on the bus between the host port and device is unknown.

From the wireshark trace it follows only that your device only pulls-up its D+ wire, which results in "CONNECT" status on PORTSC[3] register. As the host sees the connection, it tries to "enable" the port, which can be done only by issuing PORT_RESET command.

The reset very likely does through (nothing usually can prevent it), but the return status is not shown in your trace. You should include results of PORTSC reads to understand if your device really responds to reset, or it is a FS device, or it is simply dead. I am not sure if the wireshark is capable of this.

In any case some status will be returned, and the host will proceed with the enumeration process.

So, the host will send SET_ADDRESS command, the port will send SETUP-DATA tokens out, but apparently your device does not understand it and returns nothing. Set address fails, because your device was not able to maintain USB communication.

Now about possible scenarios why this can happen:

1. Your board has no valid FW to activate internal USB functionality;
2. Your board has no clock;
3. Your board does not have proper power-on reset;
4. The device clock frequency is way out of specifications.

Usually this situation requires capturing oscilloscope traces of D+ and D- wires, properly triggered and with sufficient sampling rate and buffer depth.

Answer 2

Wouldn't a hardware issue imply that no USB packets would be able to be sent?

Not exactly. Tho only data that is exchanged between host and your device is the GET DESCRIPTOR request+response. Everything else is just the hub in front of your device.

The issue here is that the GET DESCRIPTOR Response DEVICE is run at USB full speed (not high speed) but the following switching to high speed is the point that fails somehow.

This implies that your board schematic should be OK-ish, but you need an adult to look at your layout in order to find the HF problem that prevents USB high speed to run properly.