The cheapest solution from a hardware point of view would be to use a MCU that includes a USB interface (there are a lot of them in the PIC32MK/MZ/MX ranges), and just make the MCU responsible for all the communication to/from both the SD card and the USB interface. Implement the Mass Storage USB device class (or, alternatively, the Media Transfer Protocol class) in software on the MCU (there are a lot of examples and resources around: e.g. Microchip AN1169), and it will allow a computer to be able to browse and read the files when connected to the board.
This solution, however, will require heavier software development. But it is both the simplest in terms of hardware architecture, and the most flexible (if you want to implement other services through the USB port, you can. If you want the MCU to forbid some operations under specific conditions, or filter/preprocess the data on the way from the SD card to the computer, you can. And any modification on these features can be made with a simple firmware update).
This software solution would definitely be the route I'd be following if I were you.
Now, if you want this to be implemented in hardware, it is still possible: you need some USB SD card reader chip (the level shifting will typically be done by this chip). There are a handful of those: Microchip USB2660, MAX14502, and lots of them from Taiwanese manufacturers: GL823U, MA8121, ... Not sure FTDI is a good choice: they have some USB to SPI bridge, but they don't directly implement the USB Mass Storage device class, so you'd need some dedicated software on the PC to be able to browse files (maybe they have a specific firmware/chip somewhere implementing mass storage but I couldn't find it).
Then you need a way to switch the SD card bus to be connected either to the card reader chip or to your MCU. Some 1:2 analog multiplexer can do the job here (better use an analog multiplexer because the SD pins can be bidirectional). Choose one that can handle the frequency required by your SD card speed.
The MAX14502 card reader chip is particularily interesting because it is supposed to act as a middle-man between the USB/SD ports and the MCU, so you can choose to bypass the MCU and go through the reader chip, or go through the MCU. In short, the above multiplexer is builtin (see datasheet page 18), and it even includes a similar multiplexer for the USB port. That will simplify the hardware in your specific case.