I have an Arduino (NG) board with an ATMEGA168 AVR micro. The board has a 6 pin ICSP header. Here is what I propose to do, please tell me if it is possible:
Remove current ATMEGA168 IC
Replace with a new AT AVR of similar series
Plug in the programmer to the ICSP header on the Arduino
Use avrdude software to program the AVR
Remove AVR and install in a socket on my project board and replace the original ATMEGA168 AVR
Continue to use the Arduino as normal
I have also thought about installing the 6 pin ICSP header on my project board so I can tweek the software with the chip in place. If I implement this part of the arduino circuit on my project would this be possible.
Adding your own ICSP header is easy, however your solution will work just fine, you are just using the Arduino board as a socket with an ICSP already wired up. If your programmer will supply power to the ATMega then do not power the Arduino board any other way (do not connect USB).
Further to Davr's answer - you must always attach ground, but you can leave the V+ free if your board is powered independently.
It sounds like you've already got an AVR ISP programmer and you're just looking for something to go from that to the AVR. Adding the ICSP header to your project is easy, but if you don't want to do that, I'd suggest checking out the Simple AtmegaXX8 Mini Dev kit. I'd get the version with the ZIF socket included - it's much easier to work with than a traditional socket.
Alternatively, you can wire your own up on a piece of protoboard pretty easily, though again I'd suggest getting a ZIF socket. Sparkfun has the cheapest ZIF sockets. You'll probably also want to add a crystal (Sparkfun) since the ICSP won't work without one if you've set the fuse bits to expect an external crystal (so having one means it will always work).
Would be much simpler to just program your ATMEGA directly via adding your own ICSP header. If you already have a standalone AVR programmer, all that's needed is to connect 4 pins from the programmer to the ATMEGA (the two power pins VCC/GND are not required if your ATMEGA already has its own power supply).