The two connectors on top are quite close together. When you plug in the programming connector, there is a chance you won't be able to plug the other connector.
You should put some vias in the large USB connector pads, when you solder it these will fill with solder and anchor the connector to the board. Otherwise it will only be held by the adhesive between the copper and the PCB, and it could rip off the board.
The switching regulator is a bit weird. Since the power switch is on the output, the regulator will be on constantly even if the power switch is off, or if the USB port is connected, which will drain power from the battery needlessly.
The circuit with Q1 will not work since it's a NMOS BF170, it won't turn on in this position. Even if it did, it would turn on when 5V USB is present, which is probably the opposite of what you want. Besides if you put a diode in series, the FET is unnecessary. If you want to use the FET as a zero-threshold diode, then use a PMOS that is fully on with 3.3V Vgs, with the internal diode the correct way around (cathode to "VCC"). This way it will turn off when USB 5V is applied. Then D2 is redundant and can be removed.
If the DC-DC regulator has an internal output diode, the whole circuit is also probably unnecessary. You could instead use the 5-pin version of this switching regulator, which has an ENABLE pin. Then you can disable it when it is not needed.
Or you could put the power switch on the battery side, to be able to turn it off. You could put the power switch on the input of the regulator instead, and use a SPDT switch, to power it either from the battery, or from the output of the 3V6 LDO. This eliminates Q1/D2 and gives best efficiency when running on battery, which is what you want, and a slight loss of efficiency when running on USB.
With the current schematic the micro will get 3V6 minus one diode drop, so 3V, when running on USB. Can the internal USB work with that? You should check.
Another option would be removing the switching regulator, since this Atmega has a minimum supply voltage of 2.7V, it will run fine on a coin cell voltage. You could also use something a micro which works down to 1.8V.
The current in the LED is way too high, increase R6. You can power the LED from a micro's GPIO so you can turn it off to save battery.
The cap on AREF pin is suspicious, 100pF won't decouple anything. Shouldn't it go to GND?
Since the orientation of the micro is decided by the USB pins, it would be more convenient to move the programming connector.
There are no decoupling caps, and these are required.
Each decoupling cap needs a via to the ground plane. C3 and C7 share a via which means switching regulator noise will be coupled into the micro's internal USB power supply through pin UCAP.