Wall type USB charging can give you about 1A @ 5V, so that's around 5w per hour.
In order to do the same with Solar you would need the 5w plus about 20% for efficiency, so a 6w panel. In order to get a 6w panel you would have to go with a 12v model or make one. Prebuilts can be had for under $30 online. But its not that simple, since a 12v panel is actually around a 16-18v panel. At there rated 6w they are actually around 16-18v, the reason is simple, to charge a 12v battery you need more voltage then the battery in order to charge it, and car batts actually sit over 13v charged. So you need a StepDown from 17v or so to 5v. A cheap but effective tool for this would be the MC34063, it can output up to 1.5A and can be used to step up or step down voltages.
http://www.eevblog.com/2010/09/10/eevblog-110-lets-design-a-dc-to-dc-switchmode-converter/ Handy video blog on it.
http://www.onsemi.com/pub_link/Collateral/MC34063A-D.PDF Random Spec sheet.
http://dics.voicecontrol.ro/tutorials/mc34063/ A calculator that does all the math for you.
Add some form of battery and charge controller and your set, it wont be compact at that rating thou, some of the 6w panels are 10x12 inch.
And ive seen 5w 6v(8v) panels at around 8x12inch. So they wont fit in your pocket, but are around book size.
This is actually something I was looking at but burned most of my funds building a smaller unit designed to solar charge a custom 3.6v NiMH battery with homemade charge controller and power a homemade mintyboost. Building a solar panel is far more expensive then just buying one:(