I shrinked the swap down to 32MB and set the swappiness to 20
echo 20 > /proc/sys/vm/swappiness
This seems to have really big effect. Previously, with 128MB swap and swappiness at 60 after some use of several memory hungry apps (Browser and AndNav2) the system tended to heavily use the SLOW swap, which resulted in heavily degraded performance. It took ages for the Browser to flip orientation, scrolling pages or songs in the music player wasn't smooth at all.
Now, wtih 32MB swap and swappines @ 20 the system is much more responsive. I have been testing it for 30-60 minutes, and the performance hasn't degraded at all. Browsing and scrolling is smooth, Home opens almost instantly, apps switch immediately or with 0.5 sec delay at most.
32MB swap seems to be just enough for normal use. Browse, AndNav2, Music, Pictures, SnapPhoto, Watts, PowerManager.
According to "free" it is using 97MB of RAM and 27 of swap.