I am currently attempting to mod a Nintendo 64 case to house my raspberry pi 3. I want to keep the original controller ports, but the port only has 3 pins. Would wiring a usb cable without the ground wire connected have any adverse affect?
Yes, there is a negative effect -- it won't work!
A ground is required as a reference for the USB VBUS line. Without a ground, current cannot flow through the device, so it will not work.
You need all four contacts (GND, VBUS, D+, and D+) to attach a USB device. You can't get away with fewer.