These solderless breadboards have pretty high contact resistance, so if you run any serious current through that, voltage will drop.
So you should really be using wires for your high current power supply, and solder them to the boards. For the ESP32 it doesn't matter, but for the motor it does.
Different PSU for motor.
Using boost for a motor is asking for trouble. It would be more logical to have a 10V supply for the high current stuff, and a buck converter to power the Pi from that. Also if your 10V drops a few volts under load, it started from 10V so there's still enough to run the Pi. With 5V, less margin.
I think the ESP32 draws ~250mA and the motor ~100mA
The motor will draw 10x that when starting, and multiply by 2 since you're boosting from 5V. If the boost outputs 10V 1A, it will consume 2A from 5V. Output power comes from input power, and power=VxI, so if you want 2x more Volts on the output, you get 2x more Amps on the input. For a buck, same rule applies, output power comes from input power, but output voltage is the lower one, which means input current is lower than input current.
"Decoupling capacitors". I have a bag of assorted electrolytic capacitors. Which should I use and where do I put them?
There are plenty of small decoupling caps on your board, but you can put a large electrolytic on the power supply.
PWM - gradually apply pwm to bring motor up to speed.
Yes.
Would this work with my circuit, with pwm before the boost converter?
Nope
The boost should be always on, powering the DRV8833 board which controls the motor.
DRV8833 has a PWM input, this is the one you should use to gently start your motor.