In my opinion, it does matter. The concerns mentioned by other commenters as well do lead to system integration issues, with possible nasty, stochastic bugs.
Nathan is correct that by I2C specification the open drain drivers do not have protection diodes, which may result to the pull-ups powering a 3.3V rail if that rail is not up by the time the rail driving the pull-ups is already up. This may result to occasional brown-out conditions, and a slave or master booting incorrectly.
If the system has removable components, such as sensors connected via external connectors, ESD mitigation may require the use of TVS diodes. If zener diodes are used above rated above the rail voltage, this is not an issue. If reverse biased diodes are used, then the pull-ups need to be on the rail that comes up last.
This is typically a good strategy, given:
- the host / master does not address slaves before the pull-ups are active (typically not an issue with devices running code).
- fixed configuration of the bus, with no removable slaves.
With a variable configuration (removable slaves), if the pull-ups are on the slaves, multiple sets of pull-ups may lead to unpredictable conditions. In this case, the pull-ups may be powered from the master's side, and each slave can use a re-driver, or level translator, such as https://www.nxp.com/docs/en/data-sheet/PCA9509P.pdf, with one supply coming from the master side rail, one from the slave. This is probably the safest solution, especially if there is significant difference between the rise times of the rails.