How does the LDO controls the output voltage of the buck converter as well as the switching frequency of the transistor?
Can't tell you – they're tying together input voltage and ground, so this circuit is operating the 78xx outside its operational boundaries; so, this definitely depends on the properties of one or two specific 78xx implementations, and quite possibly won't work with newer 78xx models – remember, the 78xx is fourty years old now, and people didn't have good components, so they hacked together whatever worked with the parasitic properties of what they had.
Even if the Vcc == GND regime was specified in the datasheet, the temporal behaviour specifications of the 78xx are very vague, anyway, so this is really not a case of "designed, tested, worked".
You say in your profile you want to become an SMPS expert – so don't try to recreate really obsolete stuff like this.
Update: Sam pointed out (comment below) that the VCC-to-GND connection is an artifact of TI's scan, and that they are not actually tied together; well, that leaves us still with the underdefined / vendor-specific temporal/stability behaviour that makes this design highly undesirable and unreliable.