There's really no justification for it. The two heatsinks could be any size at all. It's just a question of how much thermal resistance you want to have between each surface of the Peltier device and its respective "ambient" environment.
The coldest point in a Peltier-based system is the cold face of the device itself; the thing that it is cooling must necessarily be warmer; otherwise there would be no flow of heat into the face. Similarly, the hot face is the hottest point in the system.
The size of each heatsink determines its thermal resistance. The thermal resistance multiplied by the heat flow determines the delta-T between the Peltier device and the environment. Generally, you want that delta-T on each side to be as small as possible, so you use the biggest heatsink that fits.
It often comes down to the fact that the space being cooled is fairly compact, and so only a small heatsink fits on that side. Sometimes a small fan is used on that side to improve its effectiveness. A secondary issue is that if you have large heatsinks on both sides of the device, it can be difficult to insulate the space between them effectively — if heat from the hot-side heatsink flows directly to the cold-side heatsink, the overall efficiency of the system is reduced.