My scenario is a uC (an STM32) with a GPS. I need to output a 1Hz pulse that aligns with GPS PPS, even during loss of signal. Naturally, during LOS the output is only as good as my local oscillator, but I'd like to adjust relevant timer's timebase to average or recent GPS PPSs
When you buy a hardware GPSDO, e.g. from Jackson Labs, you'll find that it typically consists of a GPS receiver, a high-quality oscillator that gets disciplined and some microcontroller (in the case of Jackson Labs, they like LPC/espresso ARMs). You can actually see the NXP microcontroller on the bottom side product picture from Jackson labs (large black IC, right picture), next to the GPS receiver module (looks like a ublox AMY-6T or something). On the top of the device (left picture) you'll find the stable (and expensive) oscillator (large metal box).
The idea really is that in case of loss of signal, you need something stable to do your "holdover". That's why the better GPSDOs tend to use Oven-controlled oscillators.
So, first of all, check whether a ready-made GPSDO doesn't already fulfill your requirements of holdover stability. That'd be the easier way, and bonus, someone who's done this before has designed everything.
Notice that what Al alex.forencich describes is limited in accuracy: if let's say your counter runs at 10 MHz, then the best adjustment granularity achievable when only looking at a single second interval is 100 ppb – worse than many GSPDO's frequency accuracy! Of course, you'd remedy that with a narrower PID loop filter bandwidth, but that has other negative effects – what if your PID controller now can't keep up with the drift of your oscillator? What if the PPS edge timing has it's own error (hint: it has, typically in the order of 100 ns, at least)?
That's why the firmware on the GPSDOs probably is a bit smarter than just a PID; if you spend enough research and characterization on it, I'm sure you can model an oscillators f/V behaviour and it's drift better; extended Kalman comes to mind.
I've got zero experience modelling physical oscillators at this accuracy, nor modelling the variance behaviour of GPS receiver's frequency or timing output. That's why I'm not intending to build my own GPSDO anytime soon – but I wouldn't have to, because if I need the really high accuracy, I'd go to suppliers like Jackson Labs (not affiliated with these, btw), if I'm OK with decreased performance during holdover: ublox does sell their timing-specific GPS receiver ICs that you can just program to hold over. All they need is power, an antenna and an oscillator to control. They're pretty much what you describe in a single package.