It seems that this was a complex problem partially related to clocking and also lack of series resistor on MISO line. Problems finally went away after adding 100ohm resistor on MISO and setting nRF52840 SOC SCLK line to "high drive", to make clock edges fall/rise faster.
I suspect that my problems are related to a fact that i'm currently working on a breadboard with fairly long jumper wires (10-15cm). As with every project, i'm trying to test all unknown components before i'll integrate them into a circuit/PCB. Of course, breadboard can and will introduce it's own problems, and this is probably happening here, but still, i have worked with few different SPI chips, and i have never encountered such erratic behavior of the signals on the bus before.
- First problem: slave is outputting something on MISO, but output data is unexpected/wrong. I have included oscilloscope screens in the original question, and on second, expanded one, one can see little spikes occuring on MISO. I have examined these spikes, and it seemed that they occur always at clock edge falls, when slave should shift next bit out:
Yellow - CS, Blue - Clock, Purple - MISO (MOSI is left out on these screens)
On above upper screen, one can see that slave detects clock edge fall and MISO is starting to go high (apparenty some bit in readed byte equals 1), but then as clock edge falls even slower, slave gets confused somehow and shuts MISO down even before it reaches proper logic H level (at least thats my understanding of what's happening here). Second, lower screen is for comparison, when "high drive" feature of nRF52840 SOC is enabled on SCLK pin - clock edge falls faster, and MISO acts properly with this setting - chip is outputting expected data.
- Second problem: MISO signal is ok when probed unconnected. After connecting to master, MISO signal gets distorted to the point it's unreadable by master, or there is no MISO signal at all.
Below is proper behavior (with 100ohm resistor on MISO, high drive enabled on SCLK) - this is a different flash chip than in original question - i have changed it to Micron MT25QL128ABA, but it behaved exactly the same. Probed with master connected to MISO. Yellow - CS, Blue - Clock, Purple - MISO. MOSI is left out on these screens.
Below is behavior without resistor - no MISO signal at all. Probed with master connected to MISO. Yellow - CS, Blue - Clock, Purple - MISO. MOSI is left out on these screens.
I have found numerous threads on Nordic forums, describing similar MISO problems:
I'm not sure if this is some bug or misconfiguration of SOC pin used for MISO, or is it related to long SPI bus wires in my test circuit. From what i've readed - resistors can be used on SPI bus to match impedance of receiver and trasmitter when wires/traces are long (but is 10cm long enough to cause such problems?). Fact is, that relatively small value resistor on MISO helps here. I would like to know why exactly that happens, but i lack solid low level knowledge to be sure. I will incorporate resistor on MISO on a PCB, but i'll test it with 0R value first, to see if better conditions on PCB (shorter traces) will make this work without additional resistance.