Thank you for bringing this up, it's something I'd like to check on my phone. I know that just because your music has art in iTunes (whether you bought tracks, or ripped them from a CD), it doesn't translate over into the Android player. I spent a long weekend manually downloading and tagging most of my collection with Mp3tag, a free program. It checks your music library (you tell it which folder[s] to search) and allows you to edit tag information, including art. It also will search your files against a couple of databases, including Amazon, to attempt to find the artwork.
LG G2 (T-Mobile, unrooted for now)
Nexus 7 1G (unrooted)
RIP Sprint Epic 4G Touch and HTC Hero, it was a good run...OK, more like a good jog.