Make your own silent ogg file or better and faster download silent.ogg from Google, rename to notification.ogg. After that delete or rename file in system/media/audio/notifications/notification.ogg and copy here your silent version. Last step is to set right permissions 644.
I do similiar for...