Can anyone explain how device drivers work? I have seen Arduino and other development boards communicating with the PC's USB port with the help of a FTDI chip. This thing can be made to work after we install the device driver. So, what's the role of the FTDI chip and the driver here? Why can't we do this job without a FTDI chip, directly connecting the Atmel MCU(I have worked with AVR MCUs only) and having some proper driver for it.
The FTDI chip connects on one side via a USB interface to a PC, and on the other presents a UART interface to the microcontroller; to the latter it looks as if a RS232 cable had been plugged into the device through an RS232-UART level shifter.
This has two advantages. On the PC side, the USB interface becomes a virtual COM port, which makes it easier for programs to communicate with it, since they don't need to go through a special USB driver -- they just have to talk to the COM port, and are thus not even aware a USB port is involved at all and do not need to have code to talk to a special USB driver.
Normally the driver for the FTDI chip doesn't need to be installed -- since thee are so many devices using variations of the FTDI chip, the newer versions of Windows and Linux come with an USB/FTDI driver already built-in.
On the microcontroller side, the same is true -- instead of having to write software to function as a USB device (assuming the microcontroller even has a USB port), the firmware program in the microcontroller only has to deal with interfacing with the UART which is much easier. So at both ends, the PC and the microcontroller, the software thinks there is an RS232 cable between the PC and the board, and is not aware at all there is a USB cable instead.
If an FTDI chip is not used, and the microcontroller has a USB interface, then a USB cable can be connected directly between the microcontroller cable and the PC. One can then write firmware on the microcontroller to present a USB "device" interface to the PC. (This is in contrast to USB "host" interface, which would be used for example if a USB device such as a memory stick or mouse was plugged into the microcontroller.)
If possible, programmers writing this sort of firmware try to make use of the HID (human interface device) interface, which was initially intended to be used by devices like keyboards and mice. However since the USB HID driver is always included in the OS (otherwise your keyboard wouldn't work when booting!), it never needs to be loaded, and it is relatively easy to write software on the PC to interface with it.
The most difficult situation is where a special USB protocol is needed. This makes life much more complicated for both the firmware (microcontroller side) and software (PC side) implementers. When the USB device is plugged in and enumerated, the OS (Windows or Linux etc.) will be unable to find a built-in driver for it. So the software for the device must include a special USB driver, which the user will have to install before using the device.
That's because home computers use certain standards to communicate. There is the serial port, parallel port (obsolete), USB any version, the ethernet, optic fiber maybe, (e-)sata and so on.
A microcontroller usually uses different standards such as I2C or SPI, the only common interface is the serial link (RS232).
Each of these interfaces has its plus and its minus: serial is easy to implement but slow, ethernet is good for very long cables, sata is used for mass storage devices and is very fast... Each interface usually needs some dedicated hardware, and that's why your microcontroller don't have a sata port: that's something few designers would use so it's not worth putting it in each chip, rising the costs.
But you can of course emulate hardware via software! That's quite complicated, you woud probably need some level shifter outside your chip because usually voltages differs from interface to interface, but you can. So if your chip do not have I2C dedicated HW you can write your own code and emulate it. That of course uses a lot of CPU, and sometimes that's not possible. The FTDI chip does this work for you: it takes care of signal levels (usb is a LVDS link) and probably provides some framing or whatever. And it probably has an I2C interface. Using I2C is a lot easier than USB. Microcontroller side you only have to talk to the FTDI chip in the I2C fashion, your micro probably has an I2C HW interface, and you are good to go.
Computer side the problem is that this FTDI chip is not a standard device as a mouse or a gamepad, it probably can emulate various devices and you can choose which one by telling it via I2C. The point is it needs some drivers. A driver is a piece of software that interfaces with a piece of hardware: in this case the level is just a bit over: the USB controller computer side has its own driver, but the way to communicate with the FTDI chip is custom. Maybe it needs some strange packets at the beginning to be properly initialized, maybe it has a maximum speed that is much lower than USB standard considers... Think of the driver as a piece of code that software side exposes two buffers: data_in and data_out, while hardware side takes care of how to send this buffers to your custom device.
It's just like what you are doing microcontroller side: you start I2C communication, set up some registers, start sending data.
You could do the job without any interface chip, as you suggest, but that would be much more difficult. You'd have to find a way to connect to a low level interface of your PC, that means find a free I2C port, if any, and connect your MCU there. Or another low level interface of course. You are trading tons of speed, time, reliabilty, reusability for some pennies saved on an FTDI.