To know what value exists at address X one may simply read it using IORD and then send it via JTAG UART to the PC. However, to simplify debug process, is there a way to have visibility of certain addresses at all times where registers of certain peripheral are mapped so debugging is easier?
It isn't entirely clear what kind of visibility you're looking for. For peripheral registers (or any other hardware nodes, for that matter), there's always SignalTap, the FPGA equivalent of a logic analyzer.
If you want to monitor the values of a few memory locations, you can add "shadow" registers for those locations that you can also connect to SignalTap. A shadow register is write-only latch that gets updated whenever the corresponding address on the memory address bus gets written to.