Two-layer is fine. Actually, that's not bad.
The main things to beware of are:
Ethernet: treat the media side as hazardous voltage. It seems innocent enough, just an isolated cable, right? But it's rated up to 1.5kV isolation, so you need to be sure that isn't jumping to your circuit. Remove ground here, under the transformer until the connector. (It's fine to have ground up alongside, to meet the LED traces, and connector shell.) This area should be poured underneath with the common node (Bob Smith termination) instead, with adequate separation distance (a few mm) to circuit GND, and then the ESD capacitor (1nF 2kV) goes from this node to circuit GND (or preferably, to chassis if present).
You'll want to read up on Ethernet layout appnotes and what "Bob Smith termination" is, to get a better idea what to do here.
As much as possible, all components and signals should have contiguous ground above or below them. Intersections are inevitable, and routing a trace under another, necessarily creates a negative space where there is zero ground present on any layers. Stitch around such openings, to minimize the open loop area, and perimeter, of this negative space.
Automatic via stitching placement is a start, but I've highlighted some priority locations below (green circles). Notice they target either side of relatively long traces, and islands/peninsulas where ground is otherwise poorly connected.
I've also roughly sketched how better to handle the Ethernet situation: it's probably easier to route the second pair up and around the pins; this will swap the pair polarity, but this is okay, swap it on the PHY side of the transformer as well. And now that's also easier to connect. Matter of fact, I think both are better this way.
Don't worry about length matching: they could be off by some cm and it won't really matter. Since you don't have matched impedance, your priority is on keeping trace length short. This arrangement allows you to butt the transformer and connector together (isolation distance permitting), and likewise transformer and PHY.
Impedance matching is impractical on 2-layer boards, maybe unless it's ordered very thin (<1mm): required trace width for 50Ω is about half the board thickness, or 0.8mm, a ponderous size for signal routing. Just keep them short, so your too-thin traces look like small inductors -- small enough not to affect signal quality. (Traces can still be fattened a bit, where it's practical to do so.)
You may also want to bus more strongly, the signals from left side of ESP32 to CN1, and route them below the right side of ESP32. This avoids the column of vias shown above, so that just a quartet will stitch the bus crossing, and a few here and there can fill out the flat (GND over GND) areas, or their perimeters/corners. This allows GND to pour under the ESP32 right-side pins.
Alternately, you could push all these traces to the top layer, and use short jumper lengths (which can be seen elsewhere on the board) so that bottom GND pours around most of them. Those two right-side traces (in the gap between ESP32 and CN1) might be bottom-side then, with ground stitching.
What's D4 doing? Is that in case CN1 backpowers the unit, and POWER_IN is unavailable? Hmm. Depending on type, U6 might be okay with that, but maybe better not to risk it. A wired-OR switch might be used, or maybe you can just improve things as-is: take feedback after D4 (I think that one via between C4/C7 is going back to U6-1?), and probably move C6/C7 after D4 so the regulator compensation isn't quite as weird as it would be otherwise. Then the regulator compensates for D4's drop.