Your whole post is a bit broad. But taken separately, each question is answerable in itself, so...
... this is actually a general purpose computer, with custom program loaded in it? So the ability to interface with computer depends solely on the program loaded in this MCU?
In this particular case, you're absolutely right on both points.
Does other computer gadgets (Mouse, Keyboard, MIDI controller, etc.) all have this kind of computer inside?
They all have a controller chip. In some case, it is a generic MCU, like on your specific example here. But in other cases (e.g. high sales volume cheap mouse), it is a specific chip like you first thought it was (and in this case, the chip is not usually documented).
And if I someday wanted to produce a computer goods too, where should I begin to get the complete circuit board in the end? The chip looks like it is not removable from the board and the connection around it is very small.I guess it have to be assembled by precision automatic machine from factory only. (Not the one you can buy separately and assemble like ATMEGA on Arduino. That one is much bigger and you can actually count all the pins with your eye)
The chip you saw (STM32F103C8T6) is a LQFP48. Those have exposed pins, so they are still relatively easy to solder by hand (compared to QFN), even if the pitch is fine. But yes, of course, given the volume, they are assembled using an automated process (pick and place machine, reflow soldering). But even the arduino boards that use DIL are assembled using automated process.
So I am also wondering how can the maker (who? factory?) program into it in the first place.
They do In-Circuit-Programming. They make traces on the PCB that go from the debug pins of the chip to some custom header. Then, during the manufacturing, there is a step where a guy (usually chinese and not paid much - or, for very high volumes again, a machine) puts some cable on this header and there is a computer that then initializes the program code in the microcontroller flash. Sometimes, there isn't even a header, it is just pads on the PCB and they use pogo pins.
I found several "discovery board" that have this chip and also USB port so you can program into them. So does it means I can also program back into this STM chip on my game controller? Because after all I am using a USB connection to use the controller, and this USB connects to the MCU.
Yes, the ST discovery boards (or is it the nucleo boards? - anyway) have a "programming" part and a "demo" part. The demo part contains the MCU to demonstrate and some example peripherals. The programming part contains another chip that makes the bridge from USB to the debugging protocol so you can program ST chips with it. On these boards, you can disable the demo part and connect another board to the programming part. So you could use that to reprogram your game controller. There are also standalone, cheap devices (branded "ST-LINK v2") that can do it. However, you need to identify where on your demo board the debug pins are available (the "custom header" or "exposed pads" I was talking about above).
There could be another solution to reprogram the game controller, which is to take advantage of the bootloader already in ROM. In that case, you don't even need specific additional hardware. You just need to set the BOOT pins of the MCU to the appropriate level (see datasheet - but you also need to identify where is this pin accessible on the PCB) and you should be able to download the new firmware code directly from the game pad USB port itself.
But redeveloping a firmware, even for a game pad, is not a straightforward task. And you'll have to do a lot of reverse engineering on the game pad hardware. It's not impossible, but I wouldn't start with such a challenge.