Usually it's better to leave the old battery in there as it can still help with starting the car even if it can't manage on its own. Inexpensive thin jumper cables or jumper cables connected to sub-optimal connection points can easily have too much voltage drop for the starter motor to be able turn over an engine. Sometimes you can let the old battery charge for some time (maybe 30-60 minutes) by connecting it with a running car or using an external charger and that will be enough to get it moving.
The problems are worse again in winter when batteries have reduced current capacity and oils are thick. High quality jumper cables can make all the difference. You'll know there is a voltage drop problem when the lights on the dash just about go out when the engine is supposed to be cranking. Maybe the starter just clicks. If the lights don't dim and the engine isn't turning over it could be the starter itself. There are only a few things in the circuit and it's pretty easy to troubleshoot (but if the starter is shot it can be a job better suited to a pro to replace it- some of them are very hard to access without a hoist).
However, I did spend a good part of an evening with friends trying to jump start a small crossover SUV without success, the motor just wouldn't turn over even with a freshly charged brand-new battery and high quality connections. Turned out that the engine had seized internally (it was apparently thunking lustily when shut off) and the vehicle was practically a write-off.