You can always make a capacitive voltage booster and then feed that into mosfet's in follower mode. Keep in mind that capacitive voltage boosters are not efficient, they are always less efficient than 50%.
It would look something like this:
I've made arrows that shows which nodes the graph's represent.
The first graph, the one on the left, is the duty cycle of your PWM.
The second graph is the PWM the circuit actually sees.
The third graph is the filtered output of your PWM.
The fourth graph is the output of this circuit, which coincides with the voltage over the first LP filter and the duty cycle of the PWM. If you are actually trying to drive some small motor or something else, then you might as well remove the 1 kΩ at the output.
The 1 kHz sawtooth and the 3 V going into the op-amp to generate the PWM is not an actual part of the circuit, that is a model for the PWM coming out from your MCU.
The output from the first LP enters a P-MOS that is in a follower configuration. The voltage at its source will be the voltage threshold of the MOSFET, so in order to remove this offset I add a N-MOS as the final stage, also in follower configuration.
If the node that says 7.67 V would've been 5 V instead, then the output would only have been able to swing between 0V and 5 V minus the threshold of the N-MOS. In this example the threshold voltage is 1.5 V, so as long as we're feeding it with 6.5 V then the output can swing between 0 V and 5 V.
The "Some oscillator" could be a pin on the MCU or some other oscillating circuit.
I also added another LP filter for extra filtering.
However, you could also just ignore the MOSFET's in follower configuration and just hook up the 7.67 V supply to your LM358 and be done with it. I just wanted to share the MOSFET solution first to show that you don't have to throw op-amps at everything to solve every problem.