Caps are located close to each digital IC, or small set of such ICs, to act as local reservoirs to smooth out the rapidly fluctuating current demands of such ICs. This prevents those rapidly fluctuating currents from causing fluctuating voltages on longer supply wires (PCB traces) and possibly disrupting other chips connected to those supply wires.
In some instances you will also see a large cap parallel with a small cap right next to it. The large cap provides a large reservoir, but has a significant internal resistance, so doesn't respond as quickly as a small cap can. So together the two caps can respond quickly and provide a large reservoir.
Real capacitors have both some internal resistance and inductance in series with their "ideal" capacitance. The effects are larger with larger-value capacitors, and vary with capacitor material and construction. For the current discussion, both these non-ideal characteristics act to slow the speed with which the capacitor can respond.
A good discussion can be found here: http://www.analog.com/library/analogdialogue/anniversary/21.html
An additional article on board layout for high-speed digital: http://www.ti.com/lit/an/scaa082/scaa082.pdf