You cannot get those two temperatures to be the same unless they are fairly tightly thermally coupled. The peltier is dissipating power (heat), while box itself is not, and is even losing a small amount of heat to the environment through the styrofoam (though the styrofoam should reduce this to a trickle, depending on the thickness).
This means that regardless of which way you're moving watts with the peltier, you're still just moving them from one side of the peltier to the other. If the inside side of the peltier is hotter or cooler than the ambient temperature in the box, then heat will flow into or out of the air in the box and everything else inside it. The rate of this heat flow is proportional to the temperature difference between the box environment and the peltier.
More importantly, this system will only conduct heat at a certain rate, and a nice way to model this is using the concept of thermal resistance. The units of thermal resistance are usually given in °C/W. If some thermal junction or system has a thermal resistance of 2 °C/W, this means that for 1 Watt of heat flow between one side of the junction and the other (in either direction), there must be a 2 °C temperature differential between them.
No matter what, your peltier must be at a different temperature than the environment of the incubator if it is to actually regulate the temperature - make it warmer or cooler.
Just think about how the heat in your own home probably works: if you set your thermostat to 22 °C, and it is 19°C inside, do your vents produce air that is 22 °C? No, they produce air that is substantially hotter (presumably, roughly the same temperature regardless of what the thermostat is set to) until the termostat, which is not near a vent, reaches the correct temperature.
The temperature difference for your peltier and environment depends on how much heat you need to move (be it heating or cooling) and the thermal resistance between the peltier and the environment in general. Your thermal resistance is always going to be several degrees per Watt unless you have everything directly in contact using liquid or something else equally impractical. The fans help, but forced air is still at the mercy of surface area, and the lowest resistance you will practically be able to obtain will be relatively high.
Adding a second peltier will at best cut the thermal resistance in half, which is probably equal to or less than the reduction you could get simply by adding a heatsink with lots of fins to the inside part of the peltier, and having your fan pull air through the heatsink. Ultimately, the thermal resistance between the box and the inside side of the peltier is a function of surface area and air flow. Increase either (or both) and you lower the thermal resistance. In fact, adding another peltier is only reducing the thermal resistance because you're adding another peltier worth of surface area - but you can do better than this by using a heatsink with lots of fins on just a single peltier.
Reducing the thermal resistance will reduce your temperature difference between the chamber and the peltier, but you also need to adjust your expectations and understand that a temperature difference will always exist and it is, frankly, unrealistic to expect to be less than a few degrees difference.
Also keep in mind that you will need an even higher temperature differential while you are actually heating/cooling something to a temperature you want compared to simply maintaining that temperature. The larger the differential, the quicker you'll get the thing you're heating or cooling to the desired temperature. This is because you have to move joules into or out of the thing in addition to whatever the passive rate of heat gain/loss is. Once you reach this temperature, you only have to maintain it, and only have to equal this passive rate.
This also means it can take a very long time to cool down the interior of a box - are you letting the system run overnight at least?
Either way, you can't control the temperature of something without a temperature differential.
So, add a big heat sink with lots of surface area, use thermal paste between it and the peltier, get a bigger or faster fan, and just give the pelter more power.
If giving the peltier more power mostly increases the temperature of the peltier, this is because your thermal resistance is very high - another way to look at thermal resistance is how many degrees one side of the system will heat up per every watt dissipated. You can even measure this resistance by calculating how much additional power you're giving your peltier, and how much of a temperature rise this causes. And to double the heating/cooling, it means doubling the temperature difference if thermal resistance is fixed.
It is hard to guess if your incubator is mainly being limited by heat conduction or heat dissipation (moving it) or both, but peltiers also work better with smaller temperature differences, so I would first work to reduce the thermal resistance between the box and the peltier you have now, then increase the power as much as you can, and only if that still hasn't worked, add a second peltier. Also, triple check how well insulated your box is and make sure there are no air gaps that are leaking warm air. That is equivalent to hemorrhaging joules out into the environment.
