Default Broke a flap, bit stuck inside.

I broke off the USB port cover and the little bobble off the flap is still stuck inside the handset. I've tried digging with a pin but can't get it out.

What's the best thing to do?

Remove the rear cover?
If it's broke, use a hammer. If that fails, you're not using a big enough hammer.