Google Now definitely does *not* constantly use GPS. Right now my phone shows 4 minutes GPS use in the last 4 hours (a short walk home from work).
Google is keenly aware of the tradeoff between location accuracy and power. That's why they created the new Location APIs: see developer.android.com/google/play-services/location.html (sorry no links).
The idea is to use the least power-consuming sensor to get the desired information. For example, the phone can use the accelerometer (which takes almost no power) to see if the phone's moving. If it's not, then there's no need to update the location. If it can figure out roughly where you are from WiFi (at Starbucks again!) or the cell network then it may not need GPS. Even when the GPS is used, it needs fewer readings when you're walking than when you're driving.
Between readings, Android will try to put the GPS in a low power idle state to avoid a 2 minute cold start. Whether it can or not depends on the phone's chipset and the manufacturer's firmware.
Google Now wants to track your location so it can guess where you're going. I think it's cute, but then it doesn't use much power on my Galaxy Nexus.
The power-vampire is more likely to be a 3rd party application. As an Android developer, I know it's way way easier to just fire up the GPS when the app starts, or even before it starts, and leave it running at high speed (i.e. high power) even when I'm not using it. It takes much more effort to use GPS sparingly and to throttle it as a appropriate.
Suggestion: Leave GPS turned on for a day, and check the power usage under Settings / Battery. Then turn it off for a day and compare. You can tap the Google Services line for details, including GPS usage time.