It's the difference between asynchronous and synchronous data that you're referring to here.
With synchronous data such as I2C, one line (the clock line or SCL) is used to indicate to the slave that data is ready to be read on the other line (the data line or SDA). The slave also uses this clock for timing of sending out its data.
The beauty of this method is that the receiving end receives the clock that synchronises the clock and data (hence the word synchronous). This way, the slave device does not need to know what the speed of the data stream is.
Compare this with a UART (serial port as you called it). This only has one input line (receive data) and has no way of knowing for sure when a data bit is valid. Both ends have to know the baud rate that each other are transmitting at.
If you Google synchronous and asynchronous data transmission you will find a whole host of good articles explaining this in depth.
Edit: thanks to Tony for pointing this out. Although there is an advantage in having a separate clock and data in terms of synchronisation, the disadvantage is needing the extra line. SPI has a separate receive and transmit data line as well as a clock and a chip select. At least I2C has bidirectional comms with only two wires, but is a lot slower than SPI.
Long distance is also an issue for a protocol such as I2C. It is intended for on-board communication; you start to get clock to data skew as frequency increases.
You might also be interested to read about Manchester encoded data which embeds a clock within the data stream (and can be sent as a balanced signal to give long reach at high speed).