You'll want a LiPo Backpack:
https://www.adafruit.com/product/2124.
I've done just what you have in mind, but with a CircuitPlayground.
The battery plugs into the battery connector, the BAT and G pins connect to a 2-pin JST cable just like the one on the battery
https://www.adafruit.com/product/261 which plugs into the battery connector on the Gemma, and the 5v and G pins connect to 5v & ground of a USB connector:
https://www.adafruit.com/product/1833 or
https://www.adafruit.com/product/1764. The backpack stays connected to the battery and Gemma. Plug in a USB cable to charge the battery (and power the Gemma while doing so). You can wire an on/off switch to the backpack as well if desired.
Note that the USB connector will just power the changer, not let you program the Gemma; you still need to use the Gemma's USB connector for that.
Dave