Nope, won't show the charging icon. What I've done, if you want to ease the paranoia, lol, is hold the power button down and keep holding until the device shuts off completely. (do this with the plug disconnected) .. Now, plug the charger back in to the device. The device should power on, and you should boot up to a charging icon.
If it doesn't start up, then yeah, you may have a faulty charger. Let me know how it goes.
HTC Inspire 4G (backup phone)
HTC Flyer 7". -Daughters Tablet
Samsung Galaxy Nexus - Wifes phone
HTC One - Current phone
Asus Nexus 7 - Current tablet
Edited for those complaining.