I didn't see answers to any of my questions yet, but I think I might have worked-out (at least one of) the problem(s) you were asking about:
Using a 5V GPIO correctly as an open-drain output, won't be a problem to an input limited to 3.3V, which is expecting to be driven by an open-drain output. That's because the open-drain output either provides a low-resistance path to 0V, or else is effectively not there (apart from tiny leakage current). As an open-drain output, it doesn't output 5V (unlike a push-pull output), which seems to be a part of your concern.
It is clear from the Decawave DW1000 transceiver datasheet that its RSTn signal expects to be driven by an open-drain output:
An external source should open-drain the RSTn pin once the DW1000 has been reset.
Or it could be driven by a 3.3V (or 5V) GPIO output in push-pull mode, but only for driving the RSTn signal low, and then that MCU pin must be changed to become an input (in other words, it must not drive the signal high) as explained here:
If RSTn is controlled by a GPIO of an external micro-controller care should be taken to ensure that the GPIO is configured as high-impedance as soon as it is released from the LOW state.
However, the IC you mentioned you were using as a level converter (TI SN74LVC2G17 Dual Schmitt-Trigger Buffer) is not an open-drain output device!
Since driving RSTn low resets the Decawave IC, when you stop driving that signal low to the buffer (to end the reset), you are then driving that signal high (since that TI Schmitt-trigger is not an open drain output device). However the Decawave IC datasheet says:
RSTn should never be driven high by an external source.
Therefore you should not be driving the Decawave IC RSTn signal using that TI Schmitt-trigger buffer IC.
- drive the Decawave 3.3V RSTn signal input directly from the 5V GPIO which you mentioned, in open-drain mode, if you can be sure you will never drive the output high in push-pull mode, e.g. due to programming error or other causes outside your control;
- use a suitable open-drain buffer IC between the 5V GPIO (this time configured in push-pull mode) and the 3.3V RSTn signal input.
Update: It's difficult to advice on a suitable open-drain buffer IC, without knowing the design constraints, and the reasons why the current 74LVC2G17 was chosen originally.
That 74LVC2G17 has two (non-open-drain) buffer gates per package. I found that NXP produce a 74LVC2G06 which has two open-drain, Schmitt-trigger, inverters. They seem to have the same pinout as your current 74LVC2G17 (but do double-check).
As the 74LVC2G06 gates are inverters, you would need to invert the logic of the GPIO pins driving them, in order to have equivalent behaviour to the 74LVC2G17 (and, of course, the 74LVC2G06 gates don't drive their outputs high), as shown here from the NXP datasheet.
Source NXP datasheet
At least that is a starting point in your search for a more-suitable alternative device.
If you need more open-drain gates, then the (relatively common) 74LVC06 devices may be worth investigating.