Suppose for a second that all those things are indeed "logic gates". Would it still be a useful term? Or would it be uselessly vague because it is too broad and all-encompassing? And what would you now use to refer to AND, OR, NAND, NOR, XOR, and NOT gates as a group?
This reminds me of something I read recently about tensors. All vectors and matrices are technically special cases of tensors, but no refers to vectors or matrices as tensors because it's not useful or communicative to do so. So whenever someone says tensor, they almost always mean a tensor that is not a matrix or vector. If they were referring to a vector or matrix, they would just call it that instead.
In the end, it's about communication. The components are what they are, regardless of what you decide to label them or how you decide to classify them. What matters is how useful your classification or labels are.
So, I ask you, is it useful to refer all logic circuits as logic gates? Perhaps we already have another way to refer to all logic circuits? Maybe it is staring us in the face. Could the term perhaps be "logic circuits"?