Driving a motor (or any other inductive load) requires a very low-impedance ground return system. Looking at the photo, those long, thin wires and that white solderless breadboard just aren't the right construction approach for driving a motor. There are transient effects (which you can't see without an oscilloscope) when the ULN2003 switches on and off, which can send surprisingly large currents through the ground return system, and that in turn causes "ground bounce" errors.
The diagnostic test of putting an LED in series with GND only demonstrates that there was some current flowing from the Arduino into the ground return system; it's not an indication of trouble. But it's not a good diagnostic test, because a side effect of this test is that it raises the voltage on the chip's GND. So instead of getting the full 5V supply voltage, the Arduino only gets maybe 3.5V (assuming 1.5V LED forward voltage), and the logic levels are referred to the Arduino's GND pin, not the ground return system. With such a very large voltage drop as an LED gives, that can be enough to make the logic levels be mis-interpreted by things outside the Arduino.
It's a fairly common problem, because on schematics we just drop a ground symbol anywhere and just assume that the ground return currents do whatever they need to do. On a PCB with a groundplane, that's a valid assumption; but for motor driving, the ground return system is an important part of the system as a whole. Most of the large ground current will be flowing in a loop around the motor, the ULN2003, and the power supply. The key is to make that high-current loop physically as small and compact as possible, with the widest diameter / shortest length wiring you can mange. Once you have that, the low-current Arduino ground can be connected using the kind of wire you have in the photo.