Task killers aren't necessary for Android 2.1 and above. Google has optimized the system so that if you run out of memory, it will start automatically killing unused apps in the background. Even though you might see that only around 20MB of RAM is available doesn't mean it's a bad thing. It means that you can load up your recently used apps more quickly as it is already "cached" in your memory.
Killing apps may also disrupt other system processes that require it to run properly. However, if you want to save that extra battery then a task killer is good to use. Otherwise, task killers are not necessary. I would recommended downloading an app called "Watchdog" in the market. What this app does is monitors your running apps for any memory hoggers. This allows you to kill that app or uninstall it.
This is just my thoughts and may not be entirely correct. Anyone wishing to correct me is welcome to.
HTC One X
My Bubbletea Time | Nexus 7