It is not that black and white. Among the AVR family they have more than one serial protocol one is more of a pain than the other. The AVRs are nice in that you have the chip in reset so if your program does something stupid like repurpose the programming pins (yep, been there done that more than once) and the chip is designed that it is out of reset you might not be able to reprogram it, but there are sometimes other solutions.
It is all about the chip vendor and what features they want to put in there at what cost. An 8 pin part with multiple pins dedicated to programming would be a really bad idea, even dual purposing might be bad as the user might not be able to design their board to actually use those pins and make it field programmable. If you have enough pins though you could go so far as to dedicate some if you dont want to mux them. It is up to the chip vendor/designer to decide what protocol or what set they want to use. As already mentioned sometimes they want to make it so you have to buy their tools because they keep the protocol secret and proprietary. They may provide multiple solutions to make it easier on the user.
Some, a lot these days, allow for self programming, the mcu can program its own flash from code running on the mcu. So that opens the door to bootloaders. You can always make your own with your own protocol in a case like this so you dont have to rely on the vendor. But a number of them have one or two layers of bootloaders, one they place and protect you from using, I guess the AVR one is hardware the other is software, the bootloader the arduino folks place and use with a uart. It is not uncommon, esp with the ARM based ones, to have a serial, usb and maybe spi or other on the built in, placed by the chip vendor, bootloader. Strap an input pin on reset or set of pins and the bootloader takes over and doesnt boot the app but waits for you to field program. NXP, ST, Atmel, are vendors that do this.
Some, true, are jtag only, now with the reduced pin SWD jtag, they often offer that as a solution along with the others, and some are jtag only. But because we dont use parallel flash parts anymore that all programmed with the same protocol and you just needed an abstraction to read and write addresses, the jtag tools often dont/wont mess with learning the protocols of every possible chip out there, so either you use a special tool from them or you do it some other way (for example, my preference, write a program and download it and have it use the self programming, some of the solutions, (stlink, etc) actually download a program or design, and then the tool talks to the thing that is downloaded to support the task)
this is on you as the user though, to add this to the things to look for when choosing the mcu for your project. How do we program it, are you going to mass program these things, program per board, let ICT take care of it. But before you get to mass production, what is your solution for the software developers, do they end up bricking boards because there is no solution on the prototype boards to reprogram? Or do you expose something, etc. If the chip doesnt give any good solutions, then that just adds to the cost of the development. Certainly work with your software developers before designing the board, if nothing else a couple of uart pins and a strap are often enough to make a bootloader if one is not already present. Or a strap and jtag, they can have their application stop very early, if the strap is set, and then take over with jtag to either debug, download programs, or reprogram the chip.
In the old days you just removed the prom and erased it then stuck it in a programmer, or left it to erase and took the next erased one and programmed it put it back in the socket in the board. That or pulled the mcu itself out and put it in the eraser and/or programmer.
Most now have a field programming solution or you can make your own field programming solution (in software) if there are enough pins. If not make a socketed board for the developer prototypes, and a fixture for reprogramming, or just throw the parts away until the developers get it right.