I have a set of Sennheiser PX100s. Luckily I got them a few years ago when they were cheap; looks like they've since been discontinued and have gone up in price. But I like them. I can't stand earbuds, I've never found a single set that stays in my ears. The PX100s are headband-style and they fold up flat. And they have good sound.
Also heard good things about Koss PortaPros, another under $50 portable headband set. Despite being over the head style, both these sets are small and compact so they won't make you look TOO weird if you're wearing them in public, although they might mess your hair up a bit if you're not careful.