The entire layout is basically wishful thinking. There are all those very long power traces that will radiate like crazy, lots of thin stubs, etc. All those things taken together will make it barely functional.
At first I tried using an LDO, but it heated up too much
Most likely you didn't have good thermal heat sinking from the LDOs to the board. The LDO should be sitting on a large (say 0.5"x0.5" or bigger) copper polygon. That polygon should be stitched by thermal vias to another polygon on the bottom layer. It's ideal if the polygon can be GND, since then it's just stitched to the ground plane that's on the bottom of the board.
The entire bottom of the board should be a ground plane, with very slight interruptions. Try and get most of the signal routing done on the top, with only short trace segments on the bottom ("jumpers").
All of the GND polygons on the top layer have to be stitched with many small vias to the bottom layer. Especially around the switching regulator.
The thin traces to L1 are a no-no. Use polygons instead. Power input from the connector through decoupling caps and to U7 should be a polygon as well.
Relocate J3 etc. so that there's no long trace from the switcher output anymore.
Basically, make ground a plane, and keep all other power connections as short as possible, as fat as possible if there are appreciable currents flowing across them. Reorganize components on the board so that the power distribution is very compact, since you have to relegate almost the entire bottom side to a ground plane.