I planned to buy 2 stepper motors for small toy car. What I can find from aliexpress are mostly 12v products.
I have lipo batteries already and they are 7.2v and 5400mA. Can I use this for 12v stepper motors?
Yes, but at roughly 60% of the motor's rated capacity.
Looks like 6V is another common stepper voltage, so check to see if any will meet your performance needs. For a mobile, battery-powered application you'll want to pay attention to the power and torque densities (and compare to some de-rated 12V options). Since a stepper driver has to include current-controls (preventing the thermal failure mode), you can run a 6V stepper at 7.2V with no problems.
Steppers (also DCs and BLDCs) have a roughly triangular maximum operating region which trades off between speed and torque. Changing the supply voltage shifts the entire line, as shown in the figure. Note that some manufacturers will provide performance charts with rectangular regions - this is their rating for continuous operation, formed by taking the original triangle and chopping off the high current (thermal failure) and high speed (bearing failure) areas.
You will have to experiment to find out. If you load it lightly maybe it can work with reduced torque, if at all.
However it is never too difficult to construct a boost converter to give your motors a proper 12V rail.