1
\$\begingroup\$

I managed to run an M.2 SSD (Samsung PM991A) in a TypeE socket with an adapter designed by CRImier (thank you):

enter image description here

This card can be fit into an E slot, and can host an SSD (M type).

My Linux is enumerating the SSD perfectly, and it can read/write (obviously a lot slower than in an M type slot).

M slot has 4 PCIe lanes compared to E which has only 1 (https://pinoutguide.com/HD/M.2_NGFF_connector_pinout.shtml).

MY board is a NanoPI R5C (details):

M.2 Key E: PCIe2.1 x1, USB 2.0 x1, supports M.2 WiFi and Bluetooth Module.

According to Google, PCIe2.1 x1 has a speed of 500MB/s.

In terms of PCIe speed, we are always talking about Megabytes (!) per second and not Megabits!

The Samsung drive (according to a YouTube guy) is capable of much higher speeds (when it's inserted to a correct slot):

enter image description here

Now my problems with this E<->M adapter:

  1. SSD is rather slow: it can write with 30-50MBps (1/10th of PCIe2 speed)
  2. SSD becomes very hot: only during write

NVMe is mounted via fstab:

/dev/nvme0n1p1 /media/nvme exfat defaults,nouser,noexec,sync 0 0

Maybe some of these options needs to be changed for better performance?

Also, CRImer on his github repo mentions:

There are resistors on the bottom for interrupting PCI-E&other control signals if any of them interfere with operation of your SSD:

  • 1 interrupts PERST
  • 2 interrupts CLKREQ
  • 3 interrupts PEWAKE
  • 4 interrupts SUSCLK

I now have all these for lines shorted:

enter image description here

Tried communicating with the repo owner, but he was not responding.

Here is nvme's output:

# nvme smart-log /dev/nvme0n1
Smart Log for NVME device:nvme0n1 namespace-id:ffffffff
critical_warning                        : 0
temperature                             : 28 C
available_spare                         : 100%
available_spare_threshold               : 50%
percentage_used                         : 0%
endurance group critical warning summary: 0
data_units_read                         : 41303
data_units_written                      : 30419
host_read_commands                      : 177960
host_write_commands                     : 369228
controller_busy_time                    : 2
power_cycles                            : 33
power_on_hours                          : 2
unsafe_shutdowns                        : 2
media_errors                            : 0
num_err_log_entries                     : 0
Warning Temperature Time                : 0
Critical Composite Temperature Time     : 0
Temperature Sensor 1           : 28 C
Thermal Management T1 Trans Count       : 14
Thermal Management T2 Trans Count       : 0
Thermal Management T1 Total Time        : 298
Thermal Management T2 Total Time        : 0

lspci:

$ lspci
0000:00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3566 (rev 01)
0000:01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd Device a809
0001:10:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3566 (rev 01)
0001:11:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
0002:20:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3566 (rev 01)
0002:21:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)

Ultraverbose lspci for the SSD:

0000:01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd Device a809 (prog-if 02 [NVM Express])
        Subsystem: Samsung Electronics Co Ltd Device a801
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 116
        Region 0: Memory at f4200000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/32 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [70] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
                DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                        ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
                        ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s (downgraded), Width x1 (downgraded)
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
                         10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
                         EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
                         FRS- TPHComp- ExtTPHComp-
                         AtomicOpsCap: 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ OBFF Disabled,
                         AtomicOpsCtl: ReqEn-
                LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
                LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
                         EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
                         Retimer- 2Retimers- CrosslinkRes: unsupported
        Capabilities: [b0] MSI-X: Enable+ Count=13 Masked-
                Vector table: BAR=0 offset=00003000
                PBA: BAR=0 offset=00002000
        Capabilities: [100 v2] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
                AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
                        MultHdrRecCap+ MultHdrRecEn- TLPPfxPres- HdrLogCap-
                HeaderLog: 00000000 00000000 00000000 00000000
        Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00
        Capabilities: [158 v1] Power Budgeting <?>
        Capabilities: [168 v1] Secondary PCI Express
                LnkCtl3: LnkEquIntrruptEn- PerformEqu-
                LaneErrStat: 0
        Capabilities: [188 v1] Latency Tolerance Reporting
                Max snoop latency: 0ns
                Max no snoop latency: 0ns
        Capabilities: [190 v1] L1 PM Substates
                L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
                          PortCommonModeRestoreTime=10us PortTPowerOnTime=10us
                L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
                           T_CommonMode=0us LTR1.2_Threshold=0ns
                L1SubCtl2: T_PwrOn=10us
        Kernel driver in use: nvme

As you can see, NVME is driven at downgraded speed of 5GT/s. As PCIe 2.0 requires 10 bits to encode 8 bits of data, we can calculate the actual speed by this formula:

5GT/s * (8b/10b) = 4Gbps (500MB/s) as I earlier wrote.

So the system itself identifies correctly everything.

  1. How those 4 signals above can interfere with my SSD?
  2. Can any of them cause slow write speed or high temperature?
  3. Can my mount option cause slow write speed or high temperature?
\$\endgroup\$
6
  • \$\begingroup\$ Some questions you might want to answer. Even if the devices are PCIe 2.1 capable, how do you know there is a PCIe 2.1 link formed between them, if there is non standard lane config, is the link downgraded to PCIe 1? Are there retransmissions due to bad link connections? Can the SSD see or negotiate properly how much power it has available, to run at full speed? Even if the link is 500MB/sec, can the CPU transfer that much, if bandwidth is shared with something else? \$\endgroup\$
    – Justme
    Commented May 4, 2023 at 10:11
  • \$\begingroup\$ Between CPU and SSD there is this PCIe 2.1 as clean path: it's not shared with anything else. Just the SSD. CPU is powerful, and doesn't go high during copy operation \$\endgroup\$
    – Daniel
    Commented May 4, 2023 at 13:39
  • \$\begingroup\$ How can I know PCIe is downgraded to PCIe 1? How can I know if there are retransmissions? \$\endgroup\$
    – Daniel
    Commented May 4, 2023 at 13:41
  • \$\begingroup\$ You have tools such as lspci, smarctl or nvme? \$\endgroup\$
    – Justme
    Commented May 4, 2023 at 14:23
  • \$\begingroup\$ I have, added nvme and lspci outputs. \$\endgroup\$
    – Daniel
    Commented May 4, 2023 at 15:10

1 Answer 1

Reset to default
1
\$\begingroup\$

Doesn't appear to be an issue with the adapter. The lspci output shows that it has linked up at Gen 2 speed:

LnkSta: Speed 5GT/s (downgraded), Width x1 (downgraded)

It's also showing no transmission errors have been detected (which rules out retries being a cause of slowness)

CorrErr- NonFatalErr- FatalErr-

(The - in the output is indicating each type of error is not detected)

I don't know how you are measuring the speeds, but its not an unreasonable performance - a quick search online I found one example for the NanoPi running at 75MB/s write speeds for a fast NVMe disk. You'd have to do some full disk benchmarking to find out.

NVMe devices do get hot while in use - that's why the really high spec ones come with heatsinks. You don't say how hot it actually is (hot is subjective) so as long as it is within the manufacturers rating it's not unusual.

Temperatures up to 50°C would be perfectly normal. 70°C plus would be quite abnormal.

\$\endgroup\$
1
  • \$\begingroup\$ Measured with accurate thermal camera, it heats up above 100°C during write only. Speed is measured with many ways (manual copy with mc, with rsync, with cp and with dd). All of these are topping at 50MB/s. \$\endgroup\$
    – Daniel
    Commented May 6, 2023 at 21:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.