I have designed many two layer PCBs using Altium Designer. Customer has asked me to design a 4 layer board. I have searched the internet but I can't find any appropriate guide for switching from a 2 layer to a multi-layer board. Are there any recommendations, guides, or tutorials that I could use?
Four-layer design with power and ground planes is refreshingly easy compared to two-layer! You'll want to familiarize yourself with the stackup editor in your tool, first off -- as to layer stackup selection, the "canonical", if nothing else because it's the simplest to lay out, stackup for a four layer board is signal - power - ground - signal, assuming your board runs at a single voltage.
When laying out, the signal traces stick to the two signal layers, and power and ground are accessed simply by dropping vias down where appropriate instead of having to run traces for them -- the lack of traces for power and ground running all over the board makes routing far easier. (Even though it's not great for signal integrity, even the most basic of four-layer stackups is still much better than a two-layer board.)
Everything that @ThreePhaseEel has said is correct. In addition, I'd like to point out that Altium is a great tool to use for generating the designs.
Also, be sure to contact your PCB fab shop to determine what their preferred stack-up is for impedance calculations etc. They may have other limitations that you may not be aware of.
A digital design using a 4 layer PCB with two signal layers and two layers for GND and VCC is easier to do than a 2 layer board, the signal traces will not interfere with power traces anymore. You will need those thermal insulation pads for the power layers. A board with no thermal insulation pads might be unsolderable because the power plane layers sucks too much heat from the soldering iron. A SMD board requires vias for GND and VCC connection to the pads, these vias must be outside the pads with a short trace between via and pad for thermal insulation.