The problem is not only related to voltage; but also to current.
Basically; the coil operates on a set voltage and requires a set amount of current to be switch on. Commonly, this current to switch on the relay surpasses the maximum power for an output pin.
There are a few ways to control a relay from an Arduino:
You may want to use an Solid State Relay instead, these often can be controlled from 3V to 32V with very low current. Due to the
solid state nature of these relays they're also commonly faster, optically isolated, use less power and likely less prone to mechanical failure. You theoretically could dim your lights using one of these relays by turning it on/off fast (PWM signal). But there are better ways to dim lights.
You may want to use an 'Arduino Relay Board' these boards are specifically designed to be ran off an output pin. Do note that some boards are built for 5V and some even require an external 12V input.
You can design a circuit using an MOSFET (and fly back diode) to power the relay directly from the battery and use the output pin of the Arduino for the control signal. (See the other answers)
This may not work if the relay requires more power than your battery can provide, in this case you may need an step-up converter to convert your 3,x volt to 12 volt.
You may use other interfacing methods for the relay, for example you could use an IR controlled relay and send IR signals from the Arduino. In this way, power to run the relay is not consumed from your battery. The options are limitless, but you could use RS-232 or even WiFi; though; however those would require an intermediate component to do the translation.
In short; especially since it's an battery powered device; I would suggest using a solid state relay, since these typically require less power.
It's also worth to note that activating the coil of the relay usually continuously draws power (unless if you have a latched relay). So it may be worth to use the relay 'inverted'; to turn something off rather than on; if it's a device that is usually on or to use a latched relay.
Control other devices from an Arduino:
For devices that do not draw a noteworthy amount of current, you may use an 'Logic level converter', typically to interface an 5V Arduino with an 3.3V Arduino.
In some cases it may even work without the logic level converter, but there's no guarantees that it will continue to work if; for example your battery is going low.
Your question also included a note about safety, this is actually an entirely different question and too long to fully cover here.
But be sure to use a proper enclosure and it's recommended to optically (and physically, by separating traces) isolate mains voltage from the Arduino. To avoid that failure of a component could lead to your USB becoming mains voltage (110-230VAC).