My GPS receiver has a small MS621FE battery:
Any ideas what might be the purpose of a battery on a GPS receiver?
This is the battery backup power for the SRAM and RTC when main power is off. Without the external backup battery, ET-662 will always execute a cold star[t] after turning on. To achieve the faster start-up offered by a hot or warm start, a backup battery must be connected. The battery voltage should be between 2.0V and 3.5V
As mentioned in the comments, the main reason this saves time is that it avoids having to wait for the "almanac" of GPS satellite positions to be downloaded.