What is the physical significance of the group delay of a system/filter?. Why is the slope of the phase response critical in a communication system.
As already pointed out in other answers, the group delay (at a certain frequency) is the delay of the envelope of a narrowband signal centered at that frequency. Note that the term group delay is only meaningful for a few special signals (see also this answer). In general, it's more useful to directly consider a system's phase response.
The relevance of phase in communication systems is that a non-linear phase on the channel causes intersymbol interference (ISI), i.e. the symbols get mixed up in time and influence each other, and cannot be detected properly anymore by the receiver. ISI is usually mitigated by using an adaptive equalizer prior to detection.
To understand the physical relevance of the term "group delay" one should know that the definition (slope of the phase response of a system) applies to a small group of frequencies only - centered at a frequency which is much larger than the mean value of this group.
Example: Amplitude modulation with a carrirer frequency of 500 kHz and a signal bandwidth of - let`s say - 10 kHz. Then, this group of frequencies centered around 500 kHz is delayed according to the negative slope of the phase function of the corresponding system.
A pure delay is not that relevant in most systems.
It will affect timeouts, because the receiving system needs to be able to process and verify the incoming data before producing an acknowledgement, which in turn has to be transmitted back, so expected response times need to keep these things into account.
On the other hand, a non-linear phase response means that individual symbols affect each other, so e.g. a
1 following another
1 has a different voltage level than if it were preceded by a