You circuits all amplify the signal a great deal.
For your first circuit, the gain is set by the resistance of the potentiometer (maximum of 1k) and the feedback resistor (1M.) The minimum gain is like 1000. You're probably not getting that much gain, but it will amplify the noise from your signal source quite well.
Your first circuit has different gain for some inputs. I/P 1 and 2 are the same, and I/P 3 and 4 are the same.
Your second circuit doesn't do anything to explicitly set the gain - it amplifies for all the transistor can give you. It'll happily amplify any noise on the inputs to a crazy volume.
I expect the real problem isn't noise. It probably sounds like crap when there's music or voice on the inputs. That'd actually be distortion, but a lot of folks call it noise.
What you need is a simple mixer with no amplification.
You want something more like this:

Mixer with optional offset function and inverted/non-inverted outputs
Inputs J1 ... J3 are standard inputs.
Input J4 has offset function provided that no plug is inserted into J4 as the switching contact of J4 is connected to the positive supply voltage in this case (via the protection resistor R8).
At J5 the inverting sum of all inputs is available. J6 outputs the non-inverting sum. P5 allows an additional attenuation of the complete signal (affects only J6).
Typical values for the parts used:
O1, O2 = LM1458, TL082, NE5532
P1...P4 = 47k linear (CV) or log (audio)
R1...R7 = 100k (for overall amplification 1)
To obtain a higher overall amplification R5 has to be increased (e.g. to 220k for overall amplification ~ 2 or to 1M for overall amplification ~ 10).
The value of R8 defines the offset range (about 0...+6V for R8=33k, a lower value of R8 will increase the offset range and vice versa).
Source
It recommends a 1458, which is a double version of the grotty old 741. Don't use that chip. Use something more modern.
You don't need P5, O2, or J6. The given circuit assumes a dual voltage power supply.
Alternatively, you build your mixer like in your first circuit, but lower all the feedback resistors to something reasonable, like 1k or 10k at the most. That'll lower the gain and avoid clipping.