I completely rewrote my question as I found the source of the issue is related to the level shifter only.
This is the circuit for a 3.3V to 5V bidirectional levelshifter, used frequently.
My problem: I have the same circuit, and I built it 3 times now to go sure.
When I connect one side (3.3 or 5V) to ground the other side also goes to ground (works so far).
However, it draws 40mA current!
This caused my atmega128 to saturate the port pin current, effectively raising the voltage above 0V.
I simulated it on pspice, it says the current draw has to be at 500uA (10k pullups).
I tested it on 3 different BSS138 mosfets, they show the marking K38 on top (smt devices) which seems to be right (so it is the right mosfet).
I am clueless, why does it draw so high current in low state ?