You may indeed route tracks between the pins of one row to the other row, as I saw it on a graphics card on my desk. The pitch of the pins in one row is 1mm, as it is for your HDMI connector.
However, this need a PCB manufacturer who can produce really small structures. With the pad size from your drawing, a 4mil track has a clearance of 3mil to the pads, which can not be produced by each manufacturer.
May be, you can reduce the pad size (but not drill size) to get more space between the pads, but again, manufacturers have limits on the minimum size of the pad (with respect to the drill size)
Often, the pad size is smaller on the inner layers, so routing may be easier there. (And again: Check what the manufacturer can do)
So, this is really tiny stuff, and it depends on what the manufacturer can do.
Another solution is to change to an SMD connector. A pitch of 0.5mm is quite common today.