My reply is a little off your original subject, but I think it's close enough to mention here.
If you want to move up (relatively speaking) from Arduino to discrete AVR projects, instead of a USBTinyISP, I would get a ATMega32u4 Breakout Board+ (from Adafruit) or equivalent. I first went the USBTinyISP route (with ATTINY2313s), and it is a great little programmer, but I haven't used it since I got my first 32u4 BB+. I got the 32u4BB+ (and later a Teensy), and it programs using a built-in bootloader and USB, so I didn't need an ISP for that. Even though it has a USB port, you can still write easy, non-USB programs to get your feet wet. And to reprogram, the only extra hardware you need is an USB cable. But then there is LUFA; a free USB library you can use to create custom USB apps (Joystick, mini-keyboard, serial-port, etc). Now here is the key - LUFA includes an ISP app that works very simply right out of the box on the 32u4BB+. So when you're ready to program discrete AVRs (I still use an ATtiny every once in a while), you can breadboard the simple circuit, load the hex file, and viola, you got programmer. Now I have a little daughter card (shield) that I can just plug in to the BB+ when I want to program something, and it was cheaper than any Arduino shield. The only thing I have found that my simple (out of the box) 32UBB+ ISP won't do, that the USBTinyISP will, is 3.3v device programming. But I'm in the process of updating my little programmer card to add that. I even like the mini-B connector on the BB+ better than the big B connector on the USBTinyISP.
BTW, I noticed Lady Ada uses an ATMega32u4BB+ (not a USBTinyISP) to program her new iCufflinks.
Bottom line: USBTinyISP = great programmer, but just programmer | ATMega32u4BB+ = OK programmer, but a lot more.