Logging is essential in server software. Doubly so in software that runs on someone else’s servers so you can’t drop all the source code on it to run a proper debug session. Post-mortem debugging with core dumps is useful, but often you need to know what happened before the program got into the state where it crashed.
Logging should never be done to a database – WTF were they thinking?? – if only because one of the things you want to be able to log is database failures.
Logging should be done either straight to a file, or through a simple IPC call to a logging daemon, either syslogd or something like that. One IPC call, one disk write call, done. No locks, no DB server overhead.
As for the question of excessive logging, any decent multi-level logging system quietly eats messages below the current logging threshold. You normally run in the middle of the logging range, turning up the level only briefly, to debug a severe problem. 99% of the time, the default log level should be sufficient. Also, that log level should be of a low enough volume that a person can reasonably read it in real time. If it’s going by so fast that you can’t read it, it won’t be read. The exception is higher debugging levels, where you can study a snapshot of the logs off-line, but again, this should be a 1% or less case.
As for the contention that messages should go to the user interface, what if the program doesn’t have a user interface? (Server software again.) I’m not talking about a user interface that may be bolted on top of the server software. That has its own concerns, and so can have its own messaging architecture. Anything the server wants surfaced to the user level, it can send up to the UI, sure, but that’s no reason to not do logging. There’s plenty that a program wants to say in a log that users shouldn’t see.
As for logging being like excessive comments, I find that log lines are effective substitutes for comments. Anytime the programmer wants to explain what’s happening to both future programmers and people running the software, it goes in a log message, not a comment.