So I did this:
1. Add .nomedia in my music directory, eg. "Music" (only on top directory).
2. Rename "Music" directory to "Musicc"
3. Run Gallery, exit.
4. Rename "Musicc" back to "Music"
5. Run gallery and thebthumbnails no longer appear.
Also works with other directories that has thumbnails in gallery and adding .nomedia later. In my case the thumbs are still displayed.
It worked for me, might work for you. Perhaps renaming directory triggers gallery to remove unused cache.