Ideally, I want to set_output_delay only for paths from internal FFs to ulpi_dir, but it only supports targets, not sources?
Yes, it's possible in SDC. You can use the set_max_delay and set_min_delay instead. For e.g, if I really understood your requirement on ulpi_data:
set_max_delay -from [get_registers *] -to [get_ports ulpi_data] 8.200
Where \$8.200\$ ...