I wonder if I can use the physical ethernet layer like a serial port. Is it possible? The case would be reading signals with an electronic device without dealing with networking headers.
Yes, absolutely you can ditch the network protocol layers and send data "directly". But, you probably don't want to.
What you do is use standard Ethernet Phy's, magnetics, and connectors. But instead of using an Ethernet MAC (media access controller) you use an FPGA to send/receive data without the network overhead. This has been done for several "not quite Ethernet compatible" interfaces like Ethersound, and other industrial protocols.
One thing that you can't ditch is the packet nature. You must still transmit data in packets of 64 to about 1500 bytes (some Phy's allow packets up to 8192 bytes). You can't transmit packets smaller than 64 bytes, or larger than 1500. And you must allow for the proper "gap" between packets. But you have complete control over what is in the packets, and any header (if any).
I am glossing over lots of details, however. It's actually not all that easy, and the requirements are different depending on which Ethernet standard you want to use (10/100/1000 mbps). In some cases there are signal encoding issues to deal with.
I would advise that you not do this to Ethernet. It requires a large amount of skill to design the FPGA logic-- skill that most people do not have. And the benefits of doing this are minimal. It's much easier to simply use the standard Ethernet controllers and the associated protocol stacks than to dream up your own thing.
You can indeed run RS232 over Cat5 ethernet cable. Routers and similar devices sometimes have serial "console" ports with 8P8C modular sockets (RJ45 jacks).
However if you want to send serial data through an Ethernet NIC without actual Ethernet packets appearing on the wire, you are out of luck.
On the other hand, many manufacturers sell serial-to-ethernet converters which can be used in pairs to transparently connect serial devices using Ethernet infrastructure. Search for "Ethernet Serial"
Sometimes these come with "ComPort Redirector Software" that create a Virtual Com port on a computer. These probably work with most applications that expect a real serial port. They may not work if you are doing low-level bit twiddling.
Based on your question I think I know what you want, but I want to clear some things up first.
- Ethernet (or as the standard is called IEEE 802.3) is on layer 2 (data link layer) of the OSI model.
- The wire (the cat5/6 cable and RJ45 connectors) are actually defined in Layer 1 (Physical Layer), You may have heard of the term 100BASE-T before, that is defining the physical layer.
- Ethernet does not have to run over 100BASE-T, and 100BASE-T does not need to carry Ethernet
Now on to your question:
You can buy devices that will carry serial over 100BASE-T but will not do any encapsulation. These devices will not work with your home network as your switch/hub/router is expecting IEEE 802.3 packets to be coming down the wire and not Serial packets.
You can also get devices that will do serial over IP but it does not have to use 100BASE-T cable (or Ethernet).
Yes and no; copper is copper and RJ45 copper works great for serial connections, but don't confuse it for Ethernet unless it's talking the Ethernet protocol - switches and other ethernet devices won't approve.
Cisco certainly likes this idea, for instance; the standard Cisco serial console cable is RJ45 copper on the device end.
No, not with what is usually understood as the "physical layer" of ethernet. This includes the cable, magnetics, and the PHY (stands for "physical"). Even at the physical layer, you're not just sending arbitrary 0 and 1 levels to the other end.
There are also multiple things called "ethernet" that are different at the physical layer. There is the original, 10base-2, 10base-T, 100base-T, etc. The older slower ones used manchester encoding if I remember right. At 100 Mbit/s things were changed to achieve the higher speed. Even if you restricted yourself to the old 10 Mbit/s manchester encoded versions the answer would still be no. Data is inherently sent in packets. These have a preamble private to the phy layer which is used in part for collision detection (in some variants), clock synchronization, and start of packet identification. Then there is some out of packet signalling, like link pulses, which is handle in the physical layer. Since ethernet is transformer coupled, everything has to happen at some minimum frequency since the DC level is lost from one end to the other. This is one reason for manchester encoding.
You could set up a private ethernet with just a phy at each end and send individual packets. Technically you don't need a MAC layer if you just want to get packets of raw bits from one end to the other. In practise, it is probably easier to use a MAC layer at each end too, even if you just want packets of bits from one end to the other. Phy chips are usually designed to be driven only by specific MAC chips, or sometimes you get the PHY and MAC all integrated into one chip, called a MAC/PHY. You can still send packets of raw data at the MAC level by ignoring some of of the wrapper stuff.
To do that, you'd need to take the physical layer PHYceiver from an Ethernet card and connect it to a serial port rather than an Ethernet controller.
Note that you could only use this point-to-point. You could not use hubs or switches. Some hubs might work by luck, but switches definitely won't.
What's the outer problem? There are likely much better ways to get the job done.