It looks like all it does is push the app to the /data/app directory anyway. It might also fix permissions. I ran chmod 777 on the apk anyway to make sure it was good.
Turns out my issue was the app was not compatible with Android 2.1. I pushed an older APK and had no problems. Thanks for the advice.
It's helpful, in part because it checks dependencies; it'll let you know if the android version of the device and of the app are different, for instance.