You can definitely remove the chip from the arduino, and use it on a bare board or in another project. The chip on the arduino uno and older versions of the arduino duemilanove is an atmega328 (some versions have an atmega168). You can find the actual reference to the chip at the atmel website (can't post links because not enough karma) . The thing to take care of is that the pin numbers are actually quite different from those on the arduino board. Here is a page with the actual pinout, in case you want to use it in another project or read up things in the datasheet: http://www.arduino.cc/en/Hacking/PinMapping168 When you remove the chip, be careful to not bend the legs of the chip. I usually use a flat screwdriver I insert under the chip on one side, and carefully wiggle my way halfway through. Then repeat on the other side, et voila, clean pulling of the chip. Be careful when plugging it back in as well, as sometimes one leg gets a bit outside of its receiving slot and bends when pushing it back down.
What makes arduino "special" as a development board is the environment used to program it. Similar to processing (it uses the same code base), it's a kind of simplified frontend to the c++ language. The code you write in the arduino editor gets some stuff added in front and in back, and is compiled using avr-g++ to a working executable. It also gets linked to what is called the "core", which can be found in the folder hardware/cores/arduino in the Arduino folder. It initializes a few things (like timers and serial interface), and implements the helper functions we all know and love: digitalWrite, analogRead, etc... In case you want to know more about avr-gcc programming in general, there are a lot of tutorials around. I can recommend the website http://www.avrfreaks.net/, which has a lot of links and tutorials.
This binary file is then uploaded to the chip on the arduino using a program called avrdude, which again is called automatically by the processing like GUI. This program will upload the binary file through the serial port, to what is called a bootloader on the atmega328. A bootloader is a small program (usually 2kB or 4kB) that is stored permanently on the chip. If you remove it, use it in another board, and plug it back, you'll be able to use the arduino software again. The arduino chip uses a slightly modified version of the stk500 protocol, and requires a patch to avrdude. It takes a bit of will to actually overwrite the bootloader, but that is something that is quite possible. You can then either use a standard programmer to reflash the arduino. You can actually use another arduino as a flash programmer: google for arduino ISP