With VREF bypassed, the converter sends a lot more pulses to the transistor than it does without bypassing.
Obviously, much more frequent switching is causing increased power consumption but why it switches more frequently with smoother VREF in the first place?
Without the VREF bypass, the circuit doesn't maintain good regulation at all. It is sluggish. So it generates the pulses less often. The pulse packets act like a single longer pulse, since the inductor filters them and only "sees" their average, more-or-less.
With the VREF bypass, it actually works as designed, and the current consumption is what's needed to keep the output voltage within the hysteresis window. You did nothing to control the frequency of the pulses, so what you got is what the circuit should be doing. If you want it to do something else, then you have to change the design.
You don't have the instrumentation to measure the output voltage ripple on this circuit, since you can't just clip a scope probe there, or even a DVM. A DVM has 10Mohm input impedance - completely defeating the point of the 1Gohm feedback resistor. You'll have to add a JFET or CMOS buffer op-amp to the feedback divider node, so that you can observe the output voltage level without loading stuff down into oblivion. Then you can compare the ripple with VREF bypass and without it. You'll observe that the output ripple without the bypass will be worse, I expect, and that the circuit is in fact regulating much better with the VREF bypass.
When you'll be slowing the pulse rate down, you'd be trading off ripple for current consumption. How much of a tradeoff you allow must be actively controlled either by changing the comparator hysteresis or by limiting the pulse rate out of the timer (the pulses should be gated to occur less frequently).
Finally, the high-impedance node has no shielding and no guarding, and the relatively large loop areas likely magnetically couple into the high impedance node, and probably also into the VREF node.
the comparator has 1 pA bias current, there's no way it can create so much load on VREF
The 1pA comparator bias DC current is nothing much compared to the leakage current due to surface board contamination as well as currents induced into the VREF trace, never mind capacitive coupling. For impedances you're using, the board looks filthy. It must be cleaned from all flux, using a suitable flux remover, and then cleaned from ionic contamination with deionized water. Then it should be dried and not touched with bare fingers.
The high impedance node could use at least AC guarding: a guard trace connected to ground but also AC coupled to VREF. So you'll need to keep the VREF decoupling for that.
The 1.8V - inductor - transistor - ground AC current loop need to be as tiny as practical. The source of the AC current would be the decoupling capacitor, which should be right next to the inductor, right next to the transistor, and the transistor's emitter must return right back to the decoupling capacitor.
Same goes for the 1.8V - inductor - transistor - diode - output storage capacitor - ground loop.
The layout at the moment is very loosey-goosey. These components in the fast current loops need to be tightly spaced, and should be sitting on top of a ground plane as well.
The transistor is driven into saturation, so it turns off relatively slowly compared to how fast it turns on. You probably should make both switching events balanced, since those will affect the losses somewhat, but also will affect how much the current transient couples into the sensitive nodes. Since you seem to be OK with turn-off from saturation, make the turn-on similarly slow. You'll thus want to add a current-sense resistor into the inductor loop, and a fast instrumentation amplifier to condition the signal, so that you can observe it on the scope along with the output voltage and control signal - that way you can judge whatever changes you implement to slow down the switching.
The unloaded HV output current, with just the feedback network connected, is about 0.1uA, so the average inductor current is about 5uA, and the IIR losses are acceptable, so at least that's OK. Of course, as soon as you connect the diode multiplier etc., the inductor current will go up. I'm not sure how much leakage there is through the multiplier and how much current a Geiger tube needs in a background radiation environment, but those must also be kept in mind and you should be able to measure them. Thus - there should be a current sense resistor in the return leg of the HV circuit. I suggest using wideband amplifier for the current monitoring, since it helps to detect corona discharge - a rather fast pulsed phenomenon.
Yellow trace - comparator output, Blue trace - VREF ripple
