$ adb backup -apk -f <filename> <package names>
After confirming the backup on my device, it seemed to successfully have backed up things. The backup-file was far from empty, so I'm sure it did something.
After I manually removed the apps from my phone (as this was a test), I restored the backup with:
$ adb restore <filename>
Passed in my device password again and it showed me the packagenames I had actually backed up on the restore screen.
After the restore seemed to be successful I noticed that none of my apps/packages were installed. I shelled into my phone to check the /data folders by hand even none of the packages I had "restored" were there.
My phone is rooted, and ADB has root access.
Is this a joke from Google? "Here have a backup/restore mechanism!" *restores* "Hah just kidding, your data is now lost".