I would like to at night use just my (unamplified) phone as signal for two speakers.
In general, the audio output available from mobile phone is intended to drive earphones and is not suited to drive passive loudspeakers.
For passive speakers you will have to buy or build an amplifier or connect your phone via an adapter into your existing amplifier.
If your speakers are "active speakers" or "monitors", they will have an amplifier built in to them and may be able to produce an audible output from your phone. These types of speakers typically have a volume control on one speaker.

Even active speakers expect line-level inputs, not the signal levels from earphone outputs. You can purchase attenuators that are intended to convert the audio signal levels.
I need to just switch between ...
The tradition solution for this is an audio mixer

With some knowledge of electronics, you can of course build a suitable circuit to connect your phone to your speakers and to switch the inputs of your speakers between two sources.