It can be done.I had a custom XXLPH 4.0.3 ICS rom that I flashed via odin onto my galaxy S2.
I'm going through the same process now, going back to an original orange stock rom.
I downloaded an integrated package via check fus downloader and tried (as read and advised to load into Odin PDA box, check re- partition, have phone in USB debug mode, download mode on phone and flash.
However be careful as I'm finding out the (compressed / encrypted) ENC2 file that I downloaded is invalid so its causing me problems to install.
However to prove a point downgrading is easy, I have a copy of gingerbread 2.3.5 that is already split into various files to fill in the PHONE, PDA, PIT, CSC boxes in Odin.
This flashed over the ICS no problem.
So hopefully when I manage to download an uncorrupted Orange stock rom I'm sure it will work too!