In the early days of BJTs, every year the processes would improve and there would be new applications for these parts with new demands. As a result, semi-conductor companies were frequently coming out with new parts. Some were just better versions of older parts, and some were to get into a new niche that didn't have a BJT for it before.
Now consider that quite a few companies were all doing this simultaneously. Each wanted to have a product line that covered most of the market. In some ways it was a game to make your new transistor with slightly better specs than a new transistor the competition just came out with.
All this resulted in a large number of transistor models, with many of them being largely equivalent, or being outright supersets of others.
Eventually a few models got used in high-volume products. The high volume drove down the cost, so others started using that model too when the specs were good enough. When there is a particularly cheap transistor for a common application, you're going to use that one for new designs, even if 20 other transistors would work just fine too.
Now decades later, we are largely left with the few models that happened to get used in volume, have their production cost drop, and become the standard or "jellybean" part for a certain range of demands. The 2N2222A, 2N3904, and the like are in this category, although I use 2N4401 and 2N4403 for my jellybean NPN and PNP.
There is now much less development of new discrete BJTs. Look at the product line of Ztex(?) (meanwhile bought by Diodes Inc or ON-Semi or something like that). They were one of the last really specializing in and doing development on BJTs. It is one broad product line with parts covering a wide spectrum, but each being different enough from the others to make sense. The total number of parts is much smaller than is out there, but covers most of the applications. It shows what is possible when one group thinks out a single consistent product line, and that then doesn't get pressured by lots of competitive offerings popping up.