Sure, I won't be able to play a lot of games. But it's not like it has become less "fast/slow" than it was before. It's still the same phone. It's faster actually, with the setup I currently have. I've walked around with an unstable MIUI build for months. Two years ago, did you not think the Nexus S was the ****? That it was totally smooth and fast and cool and hip? Well, its smoothness hasn't changed. Your opinion has. Jelly Bean was a little laggier, but with rasbean, marmite and the seeder tweak you have nothing to complain about.
That being said, eventually you will need to upgrade. I just don't think it is as bad as some people make it look like. I'm still enjoying my Nexus S. I never knew that browsing could be so smooth.