You can try a scheme like this, but you may degrade the signal by adding noise and because you are splitting the transmission line and adding an additional load on the line. This would cause problems if you listen in on a long section of cable or fast signals.
If you did remove the 50Ω termination, the line would be imbalanced, you would experience more noise\bit errors. I would leave them on.
But its possible, you can build exactly what your describing with regular internet cable called a passive tap (and they plugged a regular NIC in so the magnetics and termination resistors were there). This configuration will degrade the signal and violate ethernet specs.
However, tapping will not work with Gigabit Base-T
To transport 1 Gbit of traffic full duplex (1 Gbit in each direction
simultaneously) a very complex signal is used to reach the desired
performance and quality. In case of Gbit Ethernet the signal is called
PAM 5 modulation, meaning that each cable pair transports 5 bits
simultaneously in both directions. The PHY chips at each end of the
cable have a very complex task at hand, because they must separate the
two signals from each other. This is only possible because they know
their own signal, so they can deduct their own send signals from the
mixed signals on the line and then interpret the information sent by
their link partners.
Source: Wikipedia Network Tap
I have seen people build taps with two phys on a gigabit with an FGPA, the system also had relays to keep the connection always on and then passed traffic through a phy to an FPGA and then back out to a phy when the tap was on an monitoring traffic.