Figure that while soldering you are likely to bump with your soldering iron anything which is too close to the things you are trying to solder. This is likely to momentarily short things. Further, some soldering irons are constructed to have a conductive path from their tip to ground; this may help avoid ESD hazards, but may sometimes pose a problem if that path interacts badly with the circuit being soldered.
In some cases, these issues don't pose any problem. In many cases, they pose a slight concern, but no real danger. For example, if an embedded system takes an annoying amount of time to boot up, and an accidental short would disrupt its operation in such fashion as to require a reboot but have no other consequence, one might decide to solder such a device while in operation even if there was a 25% chance of bumping something. Figure that if everything works cleanly, one has saved the hassle of rebooting, and if one slips up one still would hardly be any worse off than if the circuit had been powered down first.
Of course, there are also many common scenarios where a slip up could destroy the circuit being worked upon, and some where it could pose a safety risk to the person soldering, so it's important to use good judgment. Absent a really good reason, I would be inclined to solder on live circuits only when the consequences of an accidental short would be minimal.