Most likely the startup code you are using will have default handlers to exceptions you don't explicitly say you want to handle yourself in your code. These handlers do get executed, and usually by default they just go into while(1);
loop or equivalent.
When debugging, the debugger and the UI do know that now a fault happened, so most likely they stop execution and break at the fault entry point, because that is most convenient for the person debugging, and the code can't continue anyway.
Some things are not enabled by default by the CPU, like division by zero won't raise an Usage Fault handler to execute, but it all depends on your compiler startup code if it provides that.
Same thing happens if for example you enable UART receiver interrupt, but forget to install the handler, so again the default code catches it, even though it is a perfectly valid interrupt.
So yes, always install specific or generic handler to all vectors, even if these are not used. And make sure the code makes known something unexpected happens, because it is a bug anyway. During developement, blink a LED or something so it can be debugged why it happened. In release code, restart the MCU, if it can be done safely.
In general, why a fault happens, depends of course what caused it, and how much you can trust a system state after it.
If you divide by zero, and have the fault generation enabled for that, it won't of course destroy any GPIO or peripheral state.
If there is a fault from reading or writing to a memory address that is non-existent (could include the NULL address), you can't really know what has gone wrong, if a pointer has already become invalid, and you can't really continue safely from that any more.