I am new to the STM32 world. I know that all STM32 MCUs come with an internal bootloader. Is it possible to replace it with custom boatloader?
No, it is baked into the chip at manufacture time.
You can however write your own bootloader which you store in the microcontroller's flash and will be executed at power on or reset. There are a number of boot pins which determine where the processor starts executing code from when it's powered on or reset.