You need an external tri-state buffer to support increased source and sink drive while maintaining the necessary high-Z state required for Charlieplexing.
This forum post discusses a few options. Several posts suggest a tri-state amplifier with a single microcontroller pin using a circuit similar to the following:
By biasing the pin at Vcc/2, a tri-state at the microcontroller's output pin causes the buffer to enter a state where neither FET is on, thereby properly replicating a tri-state output. The price you pay is bias current for each output pin; you haven't told us if dissipation is a design constraint or not. Note that while the schematic shows a bipolar supply, it will work with a unipolar supply.