Powerful high speed output drivers take up a lot of surface area of the chip. And to operate them you need significant power.
When developing a chip you have power and surface area budgets that you have to work with.
There is a limited amount surface area, putting the output drivers on the chip means some other features have been removed or simplified. Or the chip needs a much bigger, and more expensive package.
There is also a limited amount of power you can put in a package, so running an Ethernet PHY might mean you have to reduce the specifications of other parts of the chip.
CAN bus tranceivers and USB are also examples of peripherals that take up significant surface area.
I can't seem to find any die photographs, maybe someone else can?