USB Debugging allows a program or attached computer to execute commands on your phone, including installing and starting software or accessing/altering data.
It's not harmful per se, but it would be possible to exploit it to gain access to the smartphone.
In combination with e.g. SMS TAN's for your bank transactions, malware could abuse it to still gain access to your bank account.