Hi monkey hung
I too noticed some laggyness to begin with the Live Wallpaper turned on. Sounds like your last phone was an iPhone 4? :P
I've been playing around with settings and have a few concepts on what might be going on:
1. First of all, under: Settings > Developer Options, there is a setting called "force GPU rendering". I haven't tried this with the Live Wallpaper on yet, but perhaps this might help the 2D UI laggyness? Let me know what you find.
2. Disable your Live Wallpaper, I did, it saved battery life and things are much snappier.
3. Close your unused apps. I know ICS is meant to deal with old apps much better now, however, I find my Galaxy Nexus to be snappier when I close down the 30 odd applications I have left in a suspended state. It doesn't take long to swipe them off from the home screen using the right "Application" touch button down the right hand corner of the screen.
4. I've noticed that dragging my finger along the screen does not give the same smooth/matte feeling of the iPhone 4. As a matter of fact, it almost jitters/vibrates across the contour glass. If I touch with a less contact force, I tend to find the performance of touch screen to be smooth, rather when I apply more force it tends to jump across the surface of the screen creating a jagged touch slide. I've also noticed the higher sensitivity of the accelerometer. If you have a very steady hand you can see the extra performance, if you hand is un-steady, it almost looks worse than the previous phone, but in reality, it was simply my inability to hold the device in one spot that gave the poor performance, really this a reflection on how responsive the phone is to my dodgy movements.