My Twitter account has been added to the Settings > Accounts & sync screen. What I get for "Sync Contacts" is: "Sync is currently experiencing problems. It will be back shortly". "Sync Twitter" is working fine as is syncing with googlemail accounts.
My colleague has a Samsung Galaxy Nexus (also running Android 4.0.3) and I got him to add his Twitter account to the Settings > Accounts & sync screen. He is getting the exact same problem.
Is anyone else getting this? It's been broken for a few days now.