Congratulations on asking a good question that should have been covered by the instructor.
The battery will be damaged or destroyed if left on charge when the battery is fully charged.
A 1S LiIon cell must NEVER be charged to above 4.2V.
It MUST NOT be floated at 4.2V It can be floated at 4.1V or less.
4.1V max still gives most of capacity,
4.0V is lower capacity but still useful,
3.9V is getting "a bit low".
An easy solution is to add a zener diode across the PV panel rated at a voltage that clamps the panel without battery connected to below 4.2V under maximum light level.
The actual voltage will depend on the zener diode specifications. A 3.9V 250 mW or higher power zener is probably a bit low - higher power zeners of the same voltage will clamp the about 50 mA max panel output at somewhat higher voltages.
The 1 Watt zener listed below will probably give a safe and usable Vout at 50 mA.
1N5228 1 Watt zener datasheet here
Vout is 3.9V at 20 mA.
A better solution is to clamp the panel Vmax with a shunt regulator. An eg TL431 plus 2 resistors will allow you to precision clamp the panel voltage. Max clamp current depends on the version used by 50 mA is within the spec of many versions.
An alternative is to limit panel output using a conventional LDO (Low Dropout) regulator.
A good example is an LP2951. Datasheet here
This is the adjustable version - you do NOT want the fixed 3.3V or 5V versions.
See fig 19 for adjustable output circuit.
The DIP LP2951ACN package
Please ask if more information is required.
Are you in Australia?
If so Jaycar has LP2951ACN's available.
See Here $A2.50 each in stock.
Jaycar appears to NOT have TL431's available.
Jaycar 3.9V 1 Watt zener
I've read up on the correct Li-ion battery charging procedure, and I'll see if I can get a proper charge circuit, but if I can't, what do you recommend as an alternative?
A proper charger is a good idea BUT as long as VPV abs max is under 4.1V you do not need a "proper" charger.
However, the widely available low cost TP4056 charger modules would work well in this application - but need a PV panel zener clamp to ensure that the modules Vinmax is not exceeded.
For a basic system I recommend the 3 methods I gave in my answer.
What's the best way to both buck and boost the battery input voltage so that
the battery still charges when the solar panel voltage drops?
You do not need a buck boost converter. The panel claims to be rated at 5V at 50 mA. Under no load it should give about 6V+. In any sunlight conditions at or above medium overcast it will give more that the 4.5V or so that you need for your battery + LDO. For a zener or TL431 you need only 4.2V or less before the regulator is applied. So a boost regulator is not needed. A buck regulator is usually under 90% efficient so will give you little or no energy over just using a clamp regulator (zener or TL431) or an LDO - as per my answer.