Serial DTR reset on custom Arduino-based board not working properly

Question

I have a custom atMega328 board that's essentially an Arduino Nano with a few circuits doing some custom interfacing. Nothing super crazy. However I'm running into an issue with the RESET line, which should be automatically toggled when a serial connection is made by the DTR line from an FT231XS USB to Serial converter. My setup is the same exact reset circuit most of the Arduino boards use - a 1K resistor to VCC and a series 0.1uF capacitor to convert the logic low to a short pulse. I added a removable jumper so that I can disable serial reset once the final sketch is loaded to prevent unwanted resets.

(13 Sept 2021 - Updated circuit to reflect actual reset circuit)

After burning the bootloader succesfully using a standard ISP header, I attempted to load a simple sketch to make sure USB reprogramming would work properly. However, avrdude was unable to connect to the board and gave the generic stk500_getsync(): not in sync error that's typically seen with hardware connection issues.

So, I set up a scope probe to try and capture the reset line and see what was going wrong. However, once I connected the scope probe, the reset line seemed to work fine and the sketch uploaded without issues. Disconnecting the scope probe caused the issue to come back. Thinking it could be a capacitance issue, I lightly put my finger on the reset line, and the upload worked fine again.

So, there's obviously an capacitance issue in the reset circuit. My question is - why is this happening and how do I permanently fix it? I'm following almost the exact same reset setup that most Arduino boards use, but for some reason mine only works when I add some capacitance via a scope probe or my finger.

---

Update 13 Sept 2021

Here are two scope captures from the working reset pulses. One with a 1X probe and one with a 10x probe. In both cases the reset is successful, so the probe loading the circuit is enough to make things work.

(1X probe)

(10X probe)

I'm assuming the fact that the line isn't going fully low (FT231 V_OL is 0.4V) is just an artifact of the DC blocking capacitor.

---

For completeness' sake, here's the full schematics for both the FT231XS and the AtMega328P:

Answer 1

No the setup is not exact same reset circuit of Arduino boards, and no the reason why reset signal does not go fully low is not due to DC blocking capacitor.

The USB chip is powered by 3.3V, therefore all the IO pins, including DTR pin, only toggles by 3.3V. So when the AVR is powered by 5V supply, the resistor will pull the reset up to 5V, but since DTR pin only goes low by 3.3V, the reset pin only goes down by 3.3V, so it will only go low to about 1.7V.

And it says on the ATMega328P datasheet, that maximum input voltage for the reset pin to read as low is 0.1*Vcc, or 0.5V, so the DTR pin cannot bring the reset pin low enough for the chip to reset.

The pin stays just barely over the threshold voltage and connecting an oscilloscope probe or finger adds enough resistance to ground that the AVR resets.

Fortunately, the IO pin voltage does not apply to TXD and RXD pins - the FT231 RXD input is 5V tolerant so it does not damage even if AVR TXD output is 5V, and also AVR RXD pin understands 0.6*VCC or 3.0V as logic high level so 3.3V output from FT231 is just enough margin for it to work.

Arduinos use 5V as the IO voltage supply on the USB chip so that when DTR goes from 5V to 0V, also the reset pin goes from 5V to 0V.