Now the OP has finally come up with a specification, 5% current stability over the 3v to 4.2v voltage range, I can come up with a definite circuit suggestion. Reread the original answer below, and you'll see that all these modifications are mentioned. It's not worth going to a more complicated circuit if it's not needed for the specification.
simulate this circuit – Schematic created using CircuitLab
According to the data sheet, the quiescent current of the AD8541 varies by 3uA over the voltage range 3v to 4.2v. That's 1.5% of 200uA, and less at higher currents. That's sufficiently less than the 5% spec to take on trust, though if it was found to vary by more than that, lower current opamps are available, which would be expected to have a lower current variation. In the limit, the opamp's current could be sensed and controlled for, but that doesn't appear to be necessary at this specification point.
The voltage reference D1 is driven by the regulated output voltage, so the reference diode current is unaffected by battery voltage.
The variable beta current of the transistor flows in the emitter and hence the load setting resistor, so is controlled.
Depending on the sign of OA1's offset voltage, the circuit may not start up without the trickle of current from R4. The current through R4 is part of the current flowing to the load setting resistor, so is controlled.
C1 may or may not be needed for stability. It's probably not needed, though that's where you'd put it if it was found to be necessary, and it may need to be bigger.
There's still no specification for temperature performance, in spite of asking several times, so it obviously doesn't matter. The tempco is poor, as it's controlled by an ordinary silicon diode. At the low current I've shown, 5uA flowing through the diode controlled by R3, the forward voltage is in the 500mV ballpark, and the tempco is around 2mV/C. Over a 10C temperature range, the reference voltage would change by 20mV in 500mV, or by about 4%. This may or may not be acceptable.
Replace D1 with a micropower shunt voltage reference if better stability with temperature is required. The most popular reference voltage available is 1.2ish volts, so adjust R1/2/3 to suit the reference.
You do not make it clear how constant you want this current to be. It's obvious from the OP that line regulation must be better than what you show, but, how much better, and what about temperature?
The classic constant current sink, and a better version, is shown below
simulate this circuit
In the first circuit, the diodes maintain a moderately constant base voltage, and hence voltage across R1. The line regulation is reasonable. You would need to simulate it or build and measure to see whether it's adequate for you.
In the second circuit, the amplifier removes all of the variation due to the pass transistor's non-constant gm and VBE, but leaves several other errors uncorrected: the opamp and reference current variation with line voltage, the non-constant transistor beta, and of course the voltage reference is still far from precision. There are many amplifiers that a suitable for OA1, search for low power opamps. A good candidate would be the AD8541 (50uA supply, R2R, 2.7 to 5v, not expensive), but there are many others.
Replacing Q2 with a FET removes the beta variation of the bipolar. You would have to use a very low threshold device, but they exist.
Ideally you'd improve the R4/D3 voltage reference stability with a micropower voltage reference IC, of which many are available.
As an interesting way of improving further the reference line regulation, consider putting the reference diode in series with the programmable current path. How would you modify the amplifier feedback to use the now differential reference voltage? How would you make sure such a circuit would start up?
A very interesting modification of the first diagram would be to use a TLV431 instead of the NPN. These can be had very cheaply, and are a very versatile device. Intended as a 'programmable voltage zener', they behave like an NPN transistor with a very accurately defined VBE of 1.25v. Replace Q1 with a TLV431 to put an accurate voltage across R1, which you now program to draw current. You could use another one to replace R2/D1/D2 in the standard 'zener' configuration to give you (say) a 2v reference (they work over a current of 80uA and up), or you could put your programmable R around the voltage reference section to give you a variable voltage into your load resistor.