I've search through the options and couldn't find any way to disable the login sound in Ubuntu Karmic Koala, even though I've disabled the system sounds. But as always, gconftool-2 comes to the rescue and the login sound can be turned off by running the following command in a terminal:
sudo -u gdm gconftool-2 --set /desktop/gnome/sound/event_sounds --type bool false