Use caution. There are cases where flashing one kernel over another has proven to cause stability issues. Whilst for the majority of cases it's not necessary to flash stock kernel first, be prepared to do so. In other words, have a stock kernel downloaded as a back up in case you do have issues with the phone after the kernel exchange.
Good for you. Still doesn't mean the best advice we can give someone here is to "don't worry about it, just do it, it'll be fine". You don't know what ROM / Kernel people are coming from / going to and you certainly don't know how future ROMs / Kernels will behave. Again, there ARE examples on this forum where some kernels don't like being flashed over other non-stock kernels, problems have arisen and the only option was to flash a stock kernel first.
The best advice we can give someone on this forum is to be prepared for something to not go 100%. Like you I've never had a bad flash be-it ROM, radio or kernel but I still make sure I have a nandroid of the previous working state, stock ROM, radio and Kernel downloaded and know how to flash them before I do anything. That way, should it go wrong I'm covered.
I can not stress enough how wrong it is to just say "yeah, nothing will go wrong". It DOES go wrong and it goes wrong every day. For someone.