No. Using electricity from the alternator makes the car consume more fuel.
The battery is only a "buffer" for the electrical energy generated by the alternator, so we can ignore it for our purposes. In the end, all the energy comes from the alternator, regardless of whether it's temporarily stored in the battery or not.
The alternator might always spin at a constant rate; however, its mechanical resistance increases as more power is drawn from it. This means that it becomes harder for the engine to spin the alternator the more electrical power you use in your car. As a result, the engine has to produce more torque, and therefore consumes more fuel.
If it didn't consume more fuel as you drew more electrical energy from the alternator, the world's energy problems would be instantly solved: Just take a single car, let it idle, and draw infinite electrical power from it. Unfortunately, that's not how this works, and we can't extract more electrical energy from a fuel than the thermal energy contained in the fuel.
Let's take a 100 horsepower car, for example: Assuming the alternator can handle an arbitrary amount of power and is 100% efficient (which it's not), you could draw up to 100HP of electrical power from the car, which is about 73kW. When you do this, the engine is at 100% load, so you'd have to keep the gas pedal floored to prevent it from stalling, and it'll consume just as much fuel as when you drive on the highway with the pedal to the metal.
Alternators also don't just always run at full power; instead, they're regulated to only generate as much power as is needed by the connected loads at any given time (lights, radio, battery charging, etc). In particular, car alternators are regulated in such a way that their output voltage remains roughly constant to prevent over-charging or over-discharging the battery. This regulation is done by varying the current through the alternator's excitation coils, which changes the strength of its internal magnetic fields, and therefore the amount of mechanical power it converts into electrical power at any given RPM. Stronger fields mean that the alternator consumes more mechanical energy and consequently generates more electrical energy. If you want all of the details, here's the datasheet of an alternator regulator IC by Infineon.