In the absence of some means to dissipate the energy stored in a solenoid, when the solenoid is switched off, it can induce high voltage spikes. The free-wheeling diode is there to provide a current path to "snub" these voltage spikes.
If the solenoid is switched by a semi-conductor switch, then it is a primary function of the diode to protect that switch. Semi-conductors are not very tolerant to high voltage spikes. If the solenoid is switched by a relay, the voltage spikes might cause arcing and burning (or welding) of relay contacts. Additionally, one end of the solenoid is often connected to a power supply rail. Voltage spikes may travel through the power rail and damage other parts of a circuit, so the diode also has the responsibility to protect those other parts.
As fraxinus points out in a comment, high voltage spikes can cause breakdown in the insulation in the solenoid, and damage the solenoid itself, so the diode needs to protect the solenoid.
Also, as TonyStewartSunnySkyGuyEE75 points out in comments and in his answer, the collapsing magnetic field may induce voltage in other wires that are not directly connected to the solenoid circuit, i.e. through mutual induction.
The leads of the solenoid have inductive properties just as the solenoid does. To reduce the inductive effects of the leads they should be twisted together.
In the industrial equipment that I have worked on, the switches for solenoids were always enclosed in metal cabinets, whether the switching device was a relay or transistor. The diodes would be on the circuit board for the transistor, or the mounting socket for the relay. I have also seen a pin-ball machine which had diodes soldered directly to solenoids. One difference between these two cases is that the solenoids in industrial settings were environmentally sealed, whereas the pin-ball machine had open wiring underneath the playing surface, and the solenoids were environmentally exposed within that area. Another difference is that the distance between the switch and the solenoid was much greater with industrial machines. Yet a third difference was that, at the time, mosfets were quite expensive, quite prone to failure, and circuit boards in general were expensive. Far more expensive than a solenoid, even a solenoid activated valve. Thus, protecting the circuit-board was paramount. Although the pinball machine had several circuit boards, it mainly relied on electromechanical switches and relays, the relays being comparable in price to the solenoids.
Whether the freewheeling diode should be placed nearer the switching device, or nearer the solenoid, is a design decision whose answer depends upon quite a few factors. If one's main concern is to protect a circuit board, the diode should probably be on the circuit board. Otherwise, factors such as ease of testing, ease of fabrication, ease of access and replacement, concerns about electrical noise etc. may play a significant role in the decision of where to place the free-wheeling diode.
A final note, added at the suggestion of TonyStewartSunnySkyGuyEE75. A single diode is not the only method to "snub" a magnetic device. The peak voltage spike occurs immediately upon the opening of the switch. To reduce the amplitude of this spike, a capacitor may be added. A (non-zener) diode may not dissipate energy fast enough for some purposes. So, a resistor may be added to aid in energy dissipation. In short, a variety of topologies are available.