A computer program which is stored on a hard drive and loaded into a PC to be executed is often referred to as software. It is not permanently loaded into the memory chips connected to the CPU. A new copy of the program can be loaded onto the hard drive, and the next time the program is run the new version will be loaded and run.
Embedded computers are the ones that control a lot of the electronics that run our daily lives, but which we never see. They are in microwave ovens, cruise controls, TVs, and of course smartphones. A modern car has more than a hundred embedded computers (called microcontrollers). A microcontroller can cost as little as a quarter in production quantities.
Embedded computers almost never have a hard drive (some have SD cards, which are used like hard drives, but not for loading programs). Instead, the software is programmed, or burned into the microcontroller chip. So there is no loading of the program, it just runs from where it is. Programs for embedded CPU's like this are called firmware, rather than software, because historically it was much harder to change the program. That is not so much the case now, but the name stuck.
So you have the progression Hardware -> Firmware -> Software. Hardware is the circuitry and PCB board etc., that cannot be changed after it reaches the consumer's hands. Software is dynamic, and can be instantly updated and loaded into memory and run. Firmware is somewhere in-between, it acts like software, and can be updated albeit not as easy; but once it is, in a way it becomes part of the hardware.
Historically, firmware has been located in one of several kinds of memories described below. All of these are non-volatile (they retain their content when power is removed).
- ROM (Read-Only Memory), 1956 (original patent date). Can only be
programmed by the chip maker; so developers had to develop their
programs on separate (and expensive) emulators; and when done,
turn them over to be burned into ROMs. If a mistake slipped by,
lots of chips had to be thrown out.
- PROM (Programmable ROM), 1956. Now developers could program their
own chips, but only once. Each time they made a mistake, into the trash.
- EPROM (Erasable PROM), 1971. Now the chip could be erased, by
shining UV light through a little window on the top of the chip. So
it could be used over and over. But erasing took around 20 minutes.
Lots of coffee breaks. A variation of the EPROM was the OTP (One Time
Programmable), which was an EPROM without any window, so it couldn't be
erased. Functionally, same as a PROM.
- EEPROM (Electrically EPROM), 1978. Now the chip could be erased and
then re-programmed in seconds. Developer productivity soars!
- Flash, 1984. Another type of EEPROM, which is faster to write but
slower to erase than conventional EEPROMs. Most commonly used memory
to hold firmware today.
All of the above except for PROMs are still in use today. ROMs are used for program storage where the volume is high, and costs must be kept as low as possible -- an example is singing greeting cards. ROMs are the cheapest of any memory type used on computers.
Way back "then" (1970's), chips were programmed by taking them out of a socket on the PCB, putting the chip into a stand-alone programmer, burning the new firmware onto the chip, and replacing it on the board. Then turning power on to the board, and seeing if the program ran correctly. There was no mechanism for setting breakpoints etc. A lot of debugging was done by sending debug output to a terminal.
If you had to fix a mistake, if it was a small change sometimes it was easier to just patch the binary rather than go through the whole compile-link cycle (which could take 10 minutes or more -- these were often floppy-based development systems).
ICSP (In Circuit Serial Programming) was a huge improvement. It meant it was no longer necessary to remove the CPU chip from the board to program it. This was a necessary change, since the newer microcontrollers were coming out in SMD (Surface Mount Device) packages which cannot easily be socketed. The processor chip is connected to a header on the board, which is in turn connected to a programmer via a cable, which is connected to a PC over a USB cable.
Besides being able to update the chip over this interface, ICSP also provides a full debugging interface for the developer, for setting breakpoints, single-stepping, etc.
In addition to ICSP, a lot of microcontroller chips today can update themselves in situ. They do this by putting a small program in a protected area of program memory (so they won't accidentally overwrite themselves), then a new program is downloaded to the chip, and it updates the main program by writing to the flash. After the update is complete, the CPU is reset and the new version of the program is executed. This feature is sometimes called FOTA -- Firmware Over the Air). Apple uses it, for example, to upgrade the version of iOS on their iPhones and iPads.
In addition to referring to the programs that are actually executed by the CPU, software and firmware can refer to the code that the programmer writes, e.g. he wrote the software or she wrote the firmware. The terms can also be used as adjectives, as in software listing or firmware programmer.
In relation to firmware, you will also encounter the term assembler code. This is when the programmer writes code that matches the actual machine instructions executed by the CPU. The code is still considered to be part of the firmware. When microprocessors first came out, this was very common (usually because high level compilers weren't available or they weren't very good, or the processor's memory was too small), and often the entire program would be in assembly. Now assembly code is seldom used, and the entire firmware program is usually written in C or occasionally C++ or some other higher level language.