The MIPS architecture you pictured above already includes the required hardware for the BNE instruction.
The two register numbers which are part of the BNE instruction are passed into the Register File which then passes the data to the ALU. If they are equal then the zero flag is set. The instruction decode unit determines whether the branch flag is set based on a whole bunch of control signals which are already present (the aforementioned 0 flag, and a bunch of bits in the instruction opcode). Then the next address for the program counter is calculated based on the ALU on the top which handles adding the offset address to the current address.
Many people have implemented this instruction in the existing MIPS hardware so I am very confident you do not need additional control signals. For more detailed information you can look at the MIPS instruction set and architecture implementation. Here is one such example done in VHDL.