I'm the same about swiping away recent apps (i even got an app that automatically does it for me).
I know that if it's running in the bg it is quicker to load, but I also thought that it uses battery. It does use RAM too, which I'm not worried about.
If an app consumes battery in bg, then it is most probably by its service part, not the UI part. Unfortunately, swiping away an app from recent tasks just stops its UI part, not the service part.