Let me start by saying that the battery doesn't send 4.5 Ah to the device. It contains 4.5 Ah of energy and device consumes the amount of energy it consumes. GPS doesn't need 0.8 Ah. It may have 0.8 Ah battery end when it used those 0.8 Ah, the battery is empty and the device shuts down. With the lead battery, you'll have around 5 and a half times more energy to give to the GPS when the battery is full. Also I'll explain how the GPS unit gets its power. The voltage (or tension as it is said in some areas) provided by the USB 2.0 port is 5 V. The USB 2.0 port can provide current of up to 500 mA (or 0.5 A). Voltage times current is power, so the port can give to the device as much as 2.5 W of power. If the battery of the GPS is full at 5 V, it would take around 1.6 hours (\$ \frac {0.8\mbox{ } Ah}{0.5\mbox{ }A}\$) to charge it using the USB port. Depending on the battery chemistry it may take longer.
So to sum this up: Battery gives voltage and consumer takes current. The Ah rating is used to easily calculate for how much time can the battery power the device which uses A amperes. (the As cancel each-other out during division and only hours remain).
Now for the number two: You can sort of use multimeter to show battery status. Batteries are marked with something called nominal voltage which is the 6 V in our case. It doesn't mean that the battery is at 6 V the whole time. When the battery is charged, the voltage will be higher and when it discharges the voltage will be lower. You can hope to detect the various level of charge using the multimeter by measuring the voltage. Also note that the battery voltage will depend on several conditions. It will be usually higher when the battery is charging. Once the battery is disconnected from the charger it will drop. It may drop considerably when a load is attached to the battery.
DO NOT ATTEMPT TO USE THE AMPERES RANGE TO MEASURE HOW MUCH AMPERES THERE ARE LEFT in THE BATTERY!!! That range isn't used for that (although many many and once more many people thing that it is) and if you try that, the multimeter will hopefully blow a fuse. If you're unlucky it may explode and damage anything near it.
For number 1, I'd recommend that you get a commercial battery charger. You'll of course need a charger to charge the battery and the only other option would be to make one yourself and as you said, you know very little about electricity. Make sure that you get a charger for 6 V batteries. The 12 V and 24 V are much more common because they're used by automobile industry. That may also impact the next point.
Now about the main concern: Can the 6 V battery be safely used to power the USB device? Well the answer is more or less yes. As I've said, during the charging process, the voltage of the battery will increase. What you should do is use a multimeter at the volts DC range to measure the voltage of the battery. USB standard sets the appropriate voltages at 5.25 V for upper limit and 4.75 V for lower limit. Note that depending on your battery that may mean that you're not using its full capacity. For the connection itself, you could take a USB cable such as such as one used by the GPS receiver and cut off the larger A side so that there is some wire left close to the A side. Then remove the insulation and using multimeter determine which pins on the A connector go to which colors of he internal wires (there will be 4 smaller insulated wires in the cable). Another option is to get a USB A input connector and connect the battery to it.
Here's picture of the USB connector:
Connect the - side of the battery to the - side of the connector and + side of the battery to the + side of the connector. You may need to solder a 220 \$ \Omega\$ resistor to the D+ and D- pins for this to work correctly (this should indicate to the device that the USB port is a port of a dumb device which can't "talk" to the consumer and not a port of computer which needs to be asked for permission before full power can be used).
Another idea which in my opinion is better is to get a 12 V battery and use it instead of 6 V. There are several reasons for that: First 12 V battery chargers are usually more common that 6 V battery chargers (you can use chargers for car batteries with them). Next you can also get one of those car lighter to USB A converters and connect it to the battery. It will automatically provide correct output voltage for the GPS and you won't have to worry too much about the voltage of the battery. You won't have to spend time modifying the cable or getting a USB A header. You will need to modify the convertor to allow its direct connection to the battery. The center pin of the adapter should be connected to the battery's + terminal and the ring should be connected to the battery's - terminal.
About the comment:
If you're using a regulator to decrease output voltage to voltage usable for charging 6 V battery, then you're already doing a significant part of design for the charger. Research a bit more and you'll be able to make your own charger and we want to avoid that because by the time you're able to do that you'll have to learn much more than a little bit about electricity.
Also this solution suffers from the 6 V battery problems. I understand that the such solution may look nice because it works for your friend but actually its not. First read the datasheet for the battery. It's available on the link you posted. The fully charged battery has voltage of about 6.9 V under load (higher when not in use) and (using power levels we'll be using) has discharge voltage of around 5.3 V (which is above the USB specification). You can hope that it won't damage the GPS receiver but in my opinion you're making false savings fallacy. You'd be risking expensive piece of equipment for minor savings if you directly connect the battery.
If you don't directly connect the battery, you'll have to experiment with electronics.
Here again we have the problem with battery voltage. While the 6 V may look close to 5 V, it's actually pretty far away and I'll explain why: Voltage regulation circuity needs som voltage for itself to work. Here we have two ways of getting the 5 V: Using a switching regulator which can have a very low voltage drop and using a linear regulator which would have higher voltage drop.
If we go for the switching regulator we have two options: To make a switching regulator using classical components and that will be complicated because they in general require a bit more components than linear regulators or to get one of "drop-in replacement" switching regulators which already have all those components included but are more expensive and more rare.
The other option would be to use a linear regulator. It's pretty simple to use (you only need two capacitors and the regulator itself for the simplest option) but here we have problem with the dropout voltage. The most common regulators need around 2 V voltage above the voltage they're providing in order to work. We have just one (or in best case 1.5 V) so we won't be able to get the 5 V we need. The other option are low drop-out regulators which may be a simple solution if you can find a right one. Problem with them is that most of them can provide very low currents (lower than 500 mA we need) but there are some which may fit the needs. For example Microchip's TC1263 has dropout voltage of only 0.65 V at 500 mA and may work for you.
In order to get avoid this while mess I recommended use of a 12 V battery and a cigarette lighter to USB adapter which has all needed electronics included.
