I am trying to power a brushless motor with a max current consumption of 12.5 A (specified in datasheet). How should I figure out the maximum current drawn by one of the phases of the ESC? Should I just divide the max current draw of the motor by three, or is the solution more complex?
With 6 step commutation each phase wire is connected for 2 out of every 3 steps, so as a rough guide you can assume that it conducts 2/3 of the average motor current. That means you can generally get away with motor wires that are one gauge smaller than the battery wires.
Getting a more exact figure is harder because the current waveform varies depending on motor characteristics and throttle level. Below is an example scope trace of current in one phase wire of a brushless model aircraft motor. In this case the battery current was measured at 15.4A. We see that current is flowing for 2 out of every 3 steps, and the current during this time is fairly flat at ~16A. Therefore the average current over a full cycle should be ~11A and the rms current should be ~13A.
And here's the same motor running at part throttle on a higher voltage battery, showing the effect of PWM:-
In this case the battery current was only 10.3A, but the waveform has become triangular and peaks at ~32A. The average and rms currents are still ~11A and ~13A, but are now higher than the battery current.