I got a response from Pebble that we need to force close it.. Although it solved it, I felt that the app should be intelligent enough to close itself if no Pebble is connected after a predetermined time, say after 30 seconds.
The fastest way to do a force close is using the Notification bar, press and hold on the Pebble app, click on App info, click on Force close. Now you're done.
Sent from my Nexus 7 using Tapatalk