The 18 serial streams are not encoded or scrambled. The idea is to multiplex them onto one or more encoded serial streams at higher rates for onwards transmission over optical fiber.
Presuming you really do mean 600 Mbps and not 600 Msps times some number of bits, 18 x 600 Mbps = 10.8 Gbps. 10G Ethernet is 10 Gbps, so it would probably be doable without any compression if you run the serdes blocks a little faster. You could probably get away with a Kintex 7 FPGA to do the multiplexing and demultiplexing. What is the interface to the ADCs?