Ignoring the pushbutton switches associated with the encoders, a 50-step/second step rate suggests that you can use external multiplexers to read the encoder outputs. You can find 16-to-1 multiplexers (74HC150, 74HC4067), and connect these to the two phases. Drive the address lines of the multiplexers with 4 IO lines, and use 2 lines to read the phases. You can add a 3rd multiplexer to look at the switch contacts.
You can expand the multiplexers as well, using 1 or 2 more lines as select lines.
simulate this circuit – Schematic created using CircuitLab
shows a 32-line multiplexer, using 5 select address lines and an external inverter for the most significant address. You can use two IO lines for the MSA, but you must be sure that only one is low at any time.
In practice, you would (at intervals less than 20 msec) read all of the encoder phases, then analyze each pair of bits to determine if the encoder has moved, and take whatever actions are required. For an Atmel, 20 msec is a long time, but it would be important that the software not get involved in some other task and miss a read. If the encoder has moved by 2 steps, there is no way to determine direction.
A complete setup would require 6 multiplexers - 2 per phase and 2 to read the switches, plus 3 inverters (1/2 of a hex inverter chip such as a 7HC04). Total MCU lines would be 8. 5 address lines and 3 inputs.
Total IO operations is 64. 32 address select writes, and 32 data reads. Of course, if you're only using, let's say, 20 encoders, you would only need 40 operations.