What are Fuses in Atmel microprocessors and when I should or need to change the default settings?
They're essentially configuration parameters, or like the chip's BIOS. There's a fantastic fuse calculator for Atmega AVRs here: http://www.engbedded.com/fusecalc/
They control things like which oscillator to use, and what speed to run at (ie. the internal 8MHz oscillator, or an external crystal), brownout detection, and the size of the boot flash.
I see blalor already has an excellent answer for the purpose of Atmel fuses.
To answer the follow-on question of "are they physical fuses or are they software programmable", the answer is neither. All modern Atmel and Microchip microcontrollers store the "fuse bits" in nonvolatile memory cells -- physically the same as SLC flash cells.
When you use a chip programmer to download a new program into the microcontroller program Flash memory, it also erases and re-programs those fuse bit memory cells.
However, the software running on a microcontroller cannot change the fuse bits -- not even if you are using a "self-programming" microcontroller -- i.e., a microcontroller that allows bootloader software running on it to re-program its own program flash memory.
Some Atmel processors (especially the low-power ones) allow software running on them to change the clock source on-the-fly by writing to some configuration register -- but whenever they are reset (or the power goes away and then comes back), they go back to using the clock source specified in the fuse bits. The only way to change those fuse bits is with a chip programmer.
Decades ago, those configuration bits (and the program memory as well) were actually stored in fuses -- a bunch of thin strands of metal; appropriate ones were selectively "blown" by applying 12 V until the metal melted and disconnected and the bit became non-conductive. As you have probably figured out, that make the microprocessor one-time programmable (OTP) -- the only way to un-blow a fuse was to throw the entire microprocessor away, pull out a fresh new one, and start all over.
We still call those configuration bits "fuse bits" for historical reason -- much like we often call the program memory "ROM" (even though it's not really "read-only" on a self-programming microcontroller), and we have "solid-state relays" that have no moving parts, and "telephone companies" that spend only a small fraction of their time actually dealing with sound, and "computers" that spend most of their time displaying pictures and playing music rather than, you know, computing numbers.
They can be a bit of a pain if you make a mistake with them, and configure your chip for use with an external oscillator when it uses a crystal or internal oscillator. You then find that you can't program your chip, and need to inject a clock signal into the oscillator pin to recover it, by setting the fuse correctly.