In a lot of cases, the choice is pretty arbitrary or based on "wherever it fits best" as ISAs grow over time. However, the MOS 6502 is a wonderful example of a chip where the ISA design was heavily influenced by trying to squeeze as much as possible out of limited transistors.
Check out this video explaining how the 6502 was reverse engineered, particularly from 34:20 onwards.
The 6502 is an 8-bit microprocessor introduced in 1975. Although it had 60% fewer gates than the Z80 it was twice as fast, and although it was more constrained (in terms of registers etc.), it made up for that with an elegant instruction set.
It contains just 3510 transistors, which were drawn out, by hand, by a small team of people crawling over some large plastic sheets which were later optically shrunk down, forming the various layers of the 6502.
As you can see below, the 6502 passes the instruction opcode and timing data into the decode ROM, then passes it into a "random control logic" component whose purpose is probably to overrule the ROM's output in certain complex situations.
At 37:00 in the video you can see a table of the decode ROM which shows what conditions the inputs must satisfy to get a "1" for a given control output. You can also find it on this page.
You can see that most of the things in this table have Xs in various positions. Let's take for instance
011XXXXX 2 X RORRORA
This means the first 3 bits of the opcode must be 011, and G must be 2; nothing else matters. If so, the output named RORRORA will go true. All the ROR opcodes start with 011; but there are other instructions which start with 011 also. These probably need to be filtered out by the "random control logic" unit.
So basically, opcodes were chosen so that instructions which needed to do the same thing as each other had something in common across their bit pattern. You can see this by looking at an opcode table; all the OR instructions start with 000, all the Store instructions start with 010, all the instructions which use zero-page addressing are of the form xxxx01xx. Of course, some instructions don't seem to "fit", because the aim is not to have a completely regular opcode format but rather to provide a powerful instruction set. And this is why the "random control logic" was necessary.
The page I mentioned above says that some of the output lines in the ROM appear twice, "We assume this has been done because they had no way of routing the output of some line where they wanted, so they put the same line at a different location again." I can just imagine the engineers hand-drawing those gates one by one and suddenly realising a flaw in the design and trying to come up with a way to avoid re-starting the whole process.