You're right about the LM117 being a 10mA constant current sink, and analysis is not too difficult:
simulate this circuit – Schematic created using CircuitLab
Those 10mA flow through VR1, to drop 1.5V across it. The wiper simply taps off some fraction between 0 and 1 of that potential difference. I'll call that fraction w, so that the voltage between nodes B and C is:
$$ V_{BC} = 1.5w $$
The voltage between nodes C and A is the voltage across R1:
$$ V_{CA} = I_1 R_1 $$
The voltage between nodes B and A is the regulator's reference voltage:
$$ V_{BA} = 1.2V $$
Applying KVL to the loop BCA:
$$
\begin{aligned}
V_{BC} + V_{CA} &= V_{BA} \\ \\
1.5w + I_1 R_1 &= 1.2
\end{aligned}
$$
KCL at node C:
$$ I_1 = I_{OUT} + 10mA $$
Putting those two equations together, and rearranging to find \$I_{OUT}\$:
$$ 1.5w + R_1(I_{OUT} + 10mA) = 1.2 $$
$$
\begin{aligned}
I_{OUT} &= \frac{1.2 - 10mA \times R_1 - 1.5w}{R1} \\ \\
&= \frac{1.2}{R_1} - 10^{-2} - \frac{1.5w}{R1} \\ \\
&\approx \frac{1.2}{R_1} - \frac{1.5}{R_1}w \\ \\
\end{aligned}
$$
Plugging in values of \$w=0\$ and \$w=1\$, with \$R_1 = 300m\Omega\$, we find maximum \$I_{OUT(MAX}\$ and minimum \$I_{OUT(MIN}\$:
$$
\begin{aligned}
I_{OUT(MAX)} &= 4A
I_{OUT(MIN)} &= -1A
\end{aligned}
$$
Obviously, the regulator cannot sink current, so in reality the minimum is probably more like a few milliamps.
4A is different from the stated maximum of 3A, and I believe this is due to two things. Firstly the regulator will likely cap its own output current to less than 4A, and secondly the regulator has an output resistance, effectively in series with R1. If you use a value for R1 of 400mΩ, which represents 300mΩ plus 100mΩ of output resistance, the actual maximum output current works out to be exactly 3A.
Anyway, with that out of the way, you still have a major hurdle, even if you did manage to get the above circuit working with the LR8. Digital potentiometers are restricted in terms of both current and voltage. I do not know of any such devices that can operate with signals that fall outside of their own power supply potentials.
Sure, you've now got only 10mA flowing through the resistor chain, and the potential difference between the ends of the chain is only 1.5V, but the absolute potential at either end can be anywhere from 0 to 300V. That means your control circuitry has to operate with a ground near the output potential of the regulator, which can vary wildly with respect to the +300V source and its own ground. This may or may not be practical, and it's certainly inconvenient.
An alternative is to build your own programmable current sink, which in theory is quite simple:
simulate this circuit
I take no responsibility for what happens if you build this, I won't guarantee anything when the source is 300V. If anything causes those transistors to fail, that opamp, and whatever's supplying \$V_{IN}\$ is toast. Ideally you'd need some kind of protection between the opamp output and Q1's base, maybe a TVS and fuse, but I haven't provided any protection here. I've chosen a couple of transistors capable of handling that voltage, but that's all.
This is a low-side current sink, programmed by supplying a voltage at IN. 1V there will produce 100mA through \$R_{LOAD}\$, with the component values shown. It works by using negative feedback to provide a very precise copy of the potential at IN across R2, thereby producing a precisely controllable current though that resistor. The transistors ensure that all that current is drawn in via the load, from the 300V source.
All you need now is a DAC output or filtered PWM voltage from a microcontroller to provide \$V_{IN}\$.
Q2 will have to be on a heatsink if you need more than about 3mA. At 100mA Q2 could dissipate up to 30W, requiring really substantial heatsinking. You may even need to use several transistors in parallel, each with appropriate emitter resistors to mitigate matching issues:
simulate this circuit
I strongly suggest you keep an eye on the comments for this answer, because there may be some sage advice about why this circuit is a good or bad idea, and what's wrong with it. Treat this as an exercise in how you might approach the problem given that a digital potentiometer is not likely to work as you would like.
