I understand where you're coming from. I had an HTC One S before the Nexus 4 and sometimes find myself missing the smaller form factor as it was easier to hold with one hand. Overall, the N4 is a superior device and and I wouldn't go back to the One S. If you are really worried, maybe look a phone with similar specs to the One S. I think it had a 4.3" screen.
Sent from my Nexus 7 using xda app-developers app
Nexus 4 running Stable CM 10.1
GSM Galaxy Nexus
HTC One S
HTC MyTouch 4g Slide