That's because the flashed modem is not for your phone, f.e. you can't flash i8160P/L modem on i8160.
Flash the custom rom and create a tar from phone part (modem.img.md5) extracted from roms built for your model (i8160L) and flash just the phone.tar with odin.
Or write modem.bin.md5 (not the tar!) directly to modem/phone partition using terminal/adb (requires rooted phone):
dd if=/sdcard/external_sd/modem.bin.md5 of=/dev/block/mmcbk0p13