the outer display? you're insane not to take that off. the previous fold didn't have it and the outer screen protector isn't even mentioned in samsung's warnings.
look, as a fold 2 owner you don't need to treat this phone with kid gloves. could you get a micro scratch on it that you can only see in the perfect light? yeah, you could. I used the fold 2 without the cover and have no problems.
the outside screen is much more likely to get scratched than the inside though unless you are carrying it in a bag unfolded and I don't think anyone is dumb enough to do that.