Interesting question - the answer (well my answer) is you can make a great audio amp this way. You would still have to pay some attention to the output stage, and overall design, but the use of opamps is no problem (and very common nowadays for basic, cheap amplifiers with good performance)
Although opamps are convenient tools and there are some excellent modern ones available, there are certainly still plenty of ways you can use them to achieve a poor result if you don't pay attention to the details.
This doesn't mean people will buy it though, and designers know this, so you still get high end valve based "Hi-Fi" amps costing >£2000 with 2% THD. You could maybe say the intention was to make a "bad" amplifier here as (ironically) it will make more money - unfortunately "great" means a lot of different things to a lot of different people.
You have some in the subjectivist camp that have basically decided the human ear is more accurate than any measurement tool, and can hear things none of them can see. So they can always say "Yes, your THD+n is indeed <0.001% from 20Hz-20kHz but you are not allowing for unmeasurable effect x with your design, and this is why it does not sound good to the ear"
If the desire for technical perfection were all that mattered then things like oxygen free cables costing hundreds would never get to market :-)
I think you may want to read Douglas Self's "Small Signal Audio Design" and "Audio Power Amplifier Design Handbook"
I have found him to be quite an authority on such matters. His books discuss the use of both opamps and discrete transistors. He weighs up the strengths/weaknesses which includes plenty of real life test data, and gives examples where you can obtain better performance with discrete transistors.