Your device normally runs from 12V, and only has an internal battery to keep it going if the main power is cut. As a 'charging consumption' of 100mA is specified, I assume that it must have a built in charger. If so then all you have to do to 'trickle' charge the Lipo is keep the unit connected to 12V.
How long will it last on the internal battery? At 50mA a 500mAh battery would be dead flat in 500mAh/50mA = 10 hours. Practical backup time might be ~80% of that, or about 8 hours.
At 50mA a 3.2Ah motorcycle battery would theoretically last 3.2Ah/0.05a = 64 hours. However when used a motorcycle you don't want to run it anywhere near flat, so a practical standby time might be 16 hours (25% of battery capacity). Whether this will avoid flattening the battery depends on how often and how long the bike is ridden for, and what other loads may be draining it.
We don't know how efficient the device's Lipo charging circuit is, but with 50mA allocated to charging a linear charger would take ~10 hours from flat, while a more efficient switch-mode charger might take 4~5 hours. Either way, unless you ride for 16+ hours per day the Lipo battery will not have enough capacity to stay alive when the 12V is disconnected, so you must keep the 12V battery connected even when the bike's ignition is 'off'. The Lipo battery will then stay fully charged, so you don't have to worry about it running down or drawing extra charging current.
Whether the 3.2Ah 12V battery will have enough capacity depends on how long you ride the bike for and the gaps in between. If the average trip is enough to keep the 12V battery topped up then it should last at least 16 hours without being excessively drained, so 2 trips per day (with less than 16 hours between each trip) would be enough. However if you don't ride the bike for several days you might end up with a flat battery.
The answer to this problem is to 'trickle' charge the 12V battery, not the Lipo. A low current charger designed for 'float' charging SLA batteries will do the job. Just connect it to the 12V battery when your motorcycle is in the garage, and it will compensate for current drawn by the GPS tracker.