Yes your programmer will do the job for all common atmega and attiny.
for atmel micro you have to connect this way :
GND <-> GND
SCK <-> SCK
MISO <-> MISO
MOSI <-> MOSI
RESET <-> RESET
you can add VCC, if you want to power the target from the programmer (pay attention to the maximum of current it can provide)
they are two standart header (2x3 and 2x5), but I'm not using them in my personal design as they doesn't follow the common pinout of the atmega. I use a 1x5 header in the order i listed the connection because it is so much easier to route with less via and bigger trace ! I found it with this programmer
Also dont forget to put a pull up resistor on the reset line (~ 4k). You will also need one on MISO if you are using another SPI IC.