What is the difference between the two?
How to know which one to use at a given time?
They perform a similar function, and are both used to connect signals across different schematic sheets. The small difference is that Ports are intended to be used for hierarchical design, where you have child and parent sheets. Ports connect sheet symbols to sheets whereas off-sheet connectors cannot. Ports, like Nets, can be configured in the Net Identifier Scope from global to local, as you see fit to use in your design.
Can't we just use net name from another sheet and they will still connect thus not needing the use of either of the above tools?
Yes you can, if Net Identifier Scope is set to Global.
It's your preference at the end of the day, but I do not personally recommend this.
I always use Ports whenever a Net needs to connect to a different sheet, never Off-sheet connectors. I find it helps me mentally comprehend the design space; I know that "Signal A" needs to travel via a certain place before arriving at it's final location, so it's essentially route planning, and I know where the stops are. When a design is being reviewed by a colleague, having a top sheet with ports shows the exact route I want the signals to go.
If your design with multiple sheets has only Net labels to describe the interconnections, then the reviewer must hunt down all of the locations of the Net label, with no idea of how many locations it has stopped at. This is tedious and allows errors to creep in.