I'm on Vista x64 SP2 and I can use ADB just fine. You just need to download the SDK and then get the USB driver through there. It used to come included, but now you need to use the ADV and SDK manager and download the drivers as an SDK Component through there.
After that, make sure your phone...