Maybe it will help some of you guys to see a simulation of this.
Full size can be found at http://www.flickr.com/photos/kellenjb/4991010642/lightbox/
The reason I wouldn't do it this way is because you might have sources with different resistance. Or be driving something with a much lower resistance. Or... well lots of things. It is just much cleaner to use an op-amp adder.
1) If you make the 50K pot larger, lets say 500K, you get a much smaller signal out when you are at 50% on the pot and the same signal out as the 50K when you change it to the 0% and 100%.
2) If you keep the 50K pot but change R2 and R3 to be bigger, you aren't able to get purely just the signal from 1 source over the other and the effect of the percentage that the pot is at becomes less and less important.
3) If you keep the pot and the source resistances the same but change the output to be 8ohms (such as a speaker might be) you get an effect similar to #1.