Generally, a DC motor would take quite unkindly to such a situation. Reducing torque by removing gearing may not be a particularly good idea either, in that you may end up with insufficient torque to drive the knob itself in a reliable way. If you've tried it and the knob by itself works, then you should be fine on that count. However, do keep that in mind as a potential problem, which may need rectification depending on how you solve the problem.
Since you want the user to win, you should be looking for some way for the micro to realize that the user is even playing. There are two ways I can think of for doing this -
One way is to detect that the motor is consuming more or less current than it should at that setting of speed / voltage / duty cycle / whatever you're controlling using the h-bridge. You could have a high side or low side current shunt at the servo supply which can help you measure this. For quicker response, some sort of scheme involving a comparator may be a better bet. Note that you will have to first figure out how much current is 'normal', which may be dependent on both the supply voltage as well as the quality of the knob, etc. and to a smaller degree on temperature. You should also investigate how repeatable this normal area is
The second method is to detect if the pot is turning as much as it should, or less, or more. This is much harder to do, especially since this means you need to be aware of any and all control loops that may be involved, and also be very confident about repeatability.
Note that for both methods, having a gearhead can actually help with repeatability, and particularly more so with the second method.
Whichever method you use, once you've detected that the user is playing, your motor should pull back and no longer exert control. Depending on what sort of servo you are using, this may not be very easy to achieve. For a generic DC motor, you would basically want to let the motor winding float on atleast one end. If the servo has an internal control loop which you can't shut down, you would have to atleast power off the servo. This is probably best done with a low side switch. More details would be necessary to be sure, though.
In either case, for the period during which you haven't yet detected that the user is involved and until your controller reacts, you're in a somewhat dangerous area. You should try to minimize this time as much as possible. Be sure to include diodes for protecting against reverse EMF, something simple like an 1N4007 or better yet something fast like a MUR110. You would have two diodes each on both the motor lines. If the servo includes a control loop, you may not be able to access these lines. Such servos generally have the reverse EMF diodes anyway, though.