Inspired by this Make Magazine youtube video, I would like to use the ATtiny chips instead of the Arduino for most of my projects. The video author warns that some Arduino functions won't work on the ATtinys. Is there a complete list of what is and is not supported?
Update: To better answer the question, ATTiny's can do anything that don't require hardware the ATTiny's are missing, which include the hardware serial port, hardware i2c, or multiple interrupts. Since this is hidden away in the Arduino Libraries, unless you have a good understanding of how they work or read the actual library files, things like the Wire library (hardware i2c) will not work.
For example a ATTINY84 has 1 Interrupt, no Hardware UART (Serial or i2c) and a small handful of GPIO pins, less program space and much less ram but comes in a 14 pin package.
A ATMEGA328 has 2 Interrupts, 1 Hardware UART, more ram more program space and more GPIO, which is great, if you really need it.
Additionally, most ATTiny's have no hardware multiplication, so a compiler would have to do non-base-2 multiplication in software.
Finally, Arduinos rely on a bootloader instead of spi programming, so you need an ATTiny with atleast 4~8kb flash for them to work with the Arduino bootloader.
ATTiny44/45/84/45/2313 are popular as micro-Arduinos. They have about a fourth or eighth of the ram and flash of a ATMega328 that the official Arduino Uno uses.
There are many ATTiny's, so no single statement can be said about them all.
Also see here: http://hlt.media.mit.edu/?p=1695
The most attinys don't have a hardware USART module, so you won't be able to communicate with them via serial. For some attinys instead of dedicated I2C and SPI hardware peripherals there is a simplified USI peripheral for which you have to write a bit of glue code to implement SPI or I2C communication. Some attinys don't have ADC.
In general, you have to be more careful when choosing attiny for your project, because those tend not to have full set of features available in Arduino's atmega328. For example attiny2313 does have an UART but does not have ADC, attiny44 does have ADC, but doesn't have an UART, etc.
Just check the Atmel microcontroller selection table and a datasheet at the beginning.
A trivial if relevant point, over and above the excellent answers already posted:
Most Arduino boards, not counting the Pro Mini / Pro Micro and perhaps rare others, contain besides the main microcontroller, either a second AVR or some other USB interface part. This allows not just programming of the Arduino from a computer, but also run-time serial terminal debugging. The same USB connection also provides power to the Arduino.
While obviously the ATTiny's power requirement will have been addressed in your design, debugging code at run-time using terminal software can be quite handy.