A modem, in general terms, has three parts. The first is modem part, changing one signal to another, like dialup modulating and demodulating analog signals, a cable modem demodulating rf frequency or a fios modem changing fiber optic signals into a locally usable form. The second is a router, and the third an Ethernet switch (even if single port).
When a computer sends a packet, it hits the switch. An Ethernet co-processor analyzes the packet, and if on the same network, sends it back out the correct port. It often doesn't even interact with the main cpu. If on a different network, it gets sent up to the routing processor, mainly the cpu of the device. If there is a route to that intended network it forwards it. If that means out through the modem part, then the cpu encapsulates the Ethernet packet into the communication method used for the connection, then encodes it into the physical layer of the network and sends it out, at which point the other end unpacks it, analyses where it should go based on the packet headers, and repack if needed to send out again.
Read this much more accurate and detailed E.E. times article on how Cable Modems work. Trust me, if you really want to know this is gold. http://www.eetimes.com/document.asp?doc_id=1255090