Before U3 the signal is differential, so you can't add a volume pot there. It would have to be a 4-way pot and the channel imbalance would mess up the common mode rejection of the differential amp around U3 anyway.
So it will have to be at the output of U3.
Which means you need another amp/buffer to drive the headphones. It is advisable to AC-couple the pot wiper unless the output offset of U3 is really tiny, otherwise you will hear scratching when turning the pot as the contact resistance modulates the DC as the pot wiper turns.
Note the resistor values on the OPA1622 are pretty low, they're going to load the outputs of the OPA1612 quite a bit. You'll probably get less distortion with higher resistor values.
Also, if you use a PCM1794 I guess you're not going to use that with cheap earbuds, rather a nice pair of headphones with 600R input impedance, which means this circuit may not have enough output voltage unless you tweak the resistor values. In fact you may also run out of supply voltage headroom to drive the headphones, and I have doubts about the input voltage range of the opamps. There is not much margin between your 2.68V reference and the maximum input voltage of OPA1622.
Another simpler option is to just use digital volume control.