Default [Q] Disable Hands-Free mode but keep Car Location

My Note 2 updated to Android 4.4 a few days ago. I am keeping this phone for a few more months until I upgrade, so I was wondering if I could get feedback on fixing an issue that is driving me insane.

I set up my "location" information for Home, Office and Car. Ever since I have done that, when I get into my car my phone will automatically launch "Hands-Free Mode." There are no settings to disable Hands-Free Mode permanently, I have to do it every time I get into my car.

I have a Ford, so my car already reads my text messages aloud, performs voice commands, et c. I would like to be able to keep my car location in my phone so Google Now will continue to provide me with parking location information, I just hate having to turn off Hands-Free Mode every time I get into the car.

Any suggestions would be helpful.

(Note: I don't want to have to disable S-Voice even though I don't use it. I haven't tried disabling it and getting into the car yet. I'll post more details if that is a potential solution)