Simple answer: No, it's not harmful, bad, or wrong to go back and use an older version of a ROM. But, if you're using an old ROM, don't be surprised if something you find here that says it works on Cyanogen won't work--it may be intended only for the latest version.
The build is fundamentally different since 4.0.4, but you can still go back and use it. But, you may have to do a wipe from recovery before you flash it. Just try flashing 4.0.4 over 220.127.116.11, and if you get errors, reboot recovery, do a full wipe, and flash 4.0.4 again.