The Problem With Logging

U need to go to hell.

What happened to coding horror. I can read this crap any where. This is the day the earth stood still. Is there a reason to live if you are a programmer?

At least as far as my experience goes, with embedded real time coding logging should be minimized as possible. On real time application the logging itself change the outcome. with multi threaded scenarios locking the log mechanism is a suicide. At least from my limited prospective logging other than exceptions and basic keep alive info is must NOT.

I usually use a standard logging tool (log4j for instance, yeah iā€™m a java guy :slight_smile: mostly), and have something like that for the debug messages.

http://195.251.251.19/~bkarak/weblog/2007/01022007.html

I think that something similar might apply to C# world.

There can definitely be a performance hit to overlogging, however, the decision is not a yes or no answer. When all is well, an ERROR level makes sense. Sometimes however, you may want to open the gates to DEBUG or INFO, ex. an alpha/beta release of a new feature or architecture change.

I believe the key here is flexibility, you want to have the ability to flip the switch for better forensics, but donā€™t want to become a slave to the data.

Interesting note, where I work, there have been more than a few ocassions where full logging would have helped isolate and track down intermittent nasties that we often see in prod but not in dev.

Iā€™m with you, Jeff, on this one about logging. I program for embedded systems and most embedded systems software are timing sensitive, so the more traces/logs, the more the logs may skew the software performance. I have many colleagues who do the info level type of trace at the start and end of a method which generates a lot of noise. I think itā€™s a just a lazy way of bypassing the debugger and debugging software directly from the log. Otherwise they are just copying the previous guyā€™s code and modifying it. Anyway I have always made it a point to question this practice when I am in the position, and the answer was always Eh, it is useful to have the trace there for debuggingā€¦ Oh well, they are going to have to deal with the performance problems and the unreadability of logfile.

The thing about logging is there is no right answer that is applicable for every problem.

When it comes to logging those logging hooks in your code still take time to run even when you turn that particular log off. If thatā€™s a huge impact on your program then you are better off having debug builds and use one of those if you have a problem (or ship one to the customer when they have a problem).

When it comes to production code. Iā€™ve got a couple of rules of thumb.

  1. Log when itā€™s cheap compared to the operation.
  2. Log when an exception occurs.
  3. Log when itā€™s really important and you need to pay the price.

If those logs arenā€™t enough then I use an alternative, such as conditional logs , debug builds or a debugger. Which I choose depends upon the systems constraints.

Hello,

I believe the main problem with logging is that we cannot know at coding time what is going to go wrong at runtime, so we cannot know what to log ā€¦ Thatā€™s why some of us end up logging everything. If you have an easy access to the running software, you can little by little log your way out of the outages, you should end up with meaningfull logs on critical parts.

DTrace and runtime weaving AOP techniques provide another way. Define what you want to log at runtime when facing the bug.

Good morning. Every day you may make progress. Every step may be fruitful. Yet there will stretch out before you an ever-lengthening, ever-ascending, ever-improving path. You know you will never get to the end of the journey. But this, so far from discouraging, only adds to the joy and glory of the climb.
I am from Bhutan and learning to speak English, give true I wrote the following sentence: Product information for solar wall clock from shenzhen zy pv science tech.

With love :o, Gordy.

Good morning. There are a terrible lot of lies going around the world, and the worst of it is half of them are true.
I am from Zambia and also am speaking English, please tell me right I wrote the following sentence: See our large assortment of wall sports clocks.

THX ;), Quinby.

At least as far as my experience goes, with embedded real time coding logging should be minimized as possible. On real time application the logging itself change the outcome. with multi threaded scenarios locking the log mechanism is a suicide. At least from my limited prospective logging other than exceptions and basic keep alive info is must NOT.
http://manytour.ru

If itā€™s worth saving to a logfile, itā€™s worth showing in the user interface
Here I have a few problems, A few years back (around 2006 or so), I worked with Pharmaceutical software (21CFR-Part11 anyone?); and we used to bubble major errors to the user. Typically weā€™d get a call from a customer saying they had a weird bug(bear in mind this is software theyā€™d paid millions for their couple thousand seats). Asking what it said on the screen usually turned up blanks (and often non-repeatable-yet-significant bugs turned out this way).

Iā€™m all for bubbling up to the interface so that it can be read and noted by an intelligent user, but so far Iā€™ve had no luck. Until that unhappy state of affairs changes, logs will have to do.

After looking at the original post on stackoverflow, it appears that this comment was made with an additional caveat that you have failed to mention in your post. In fact a lot of people posting comments seem to be looking at this problem from various different perspectives.

From the original comment, I tend to be a little verbose on logging because when you have to change to debug mode, the more info you have, the better.

This poster doesnā€™t seem to be considering applying this level of active logging to the live system, but instead when it comes to him attempting to debug the solution. As some have commented, the log becomes a diary of what was occurring at any particular point before an error was encountered.

Maybe the question we need to ask is, The Problem With Logging As A Method Of Debugging. Since it seems that there is a common attitude amongst some developers to just as easily replace one with the other.

Where the system is deployed makes a difference. A server in your data center can be subjected to probing and analysis to track down a problem. An embedded system deployed halfway across the world cannot be.

I do a ton of logging during development phases, and I use a variable to determine what levels of logging I want to accomplish. When I am done developing an app, I just kill the extensive logging by changing the value of the variable.

I agree with ctacke, and building in logging as needed is precisely what I do, however I use a variable to turn it off rather than having to hunt down each logging statement. I am not writing software that millions of people use, most of the software I write is to reduce manhours, so that a job previously took 100 hours now only takes a few minutes. The reduced of efficiency due to condition checking for logging really is miniscule considering the performance of the task has been time efficiency improved by a factor of over 100x in some cases.

I think you only need exception plus a stack trace to solve problems, so just log those as a default.

To me, if you have logging and tracing all over the place, the application design and code is probably in rough shape, thus the need for logging.

Selective logging may come in handy in certain situations and developers should put in logging when necessary.

For example, say you have an import process where it was doing 10 major tasks, you would probably want to log what is was doing (report back to user) and you could monitor each task within the import. In cases like that it would come in useful if there was a problem.

But overall it adds clutter to the code. The performance cost is small unless you are tracing every other line of code.

I always found that using sample Debug.Write (or Trace.Write in release mode) with a debug listeners installed should be sufficient. In some cases that is not feasible due to client a restraint so a file is required. .Net comes System.Diagnotics built in, why go log crazy?

A lot of people bash log4net, but it does have the ability to define what and who gets logged, which if you have 100s of classes and lots of assemblies can help out with large scale applications.

I think you only need exception plus a stack trace to solve problems, so just log those as a default.

To me, if you have logging and tracing all over the place, the application design and code is probably in rough shape, thus the need for logging.

Selective logging may come in handy in certain situations and developers should put in logging when necessary.

For example, say you have an import process where it was doing 10 major tasks, you would probably want to log what is was doing (report back to user) and you could monitor each task within the import. In cases like that it would come in useful if there was a problem.

But overall it adds clutter to the code. The performance cost is small unless you are tracing every other line of code.

I always found that using sample Debug.Write (or Trace.Write in release mode) with a debug listeners installed should be sufficient. In some cases that is not feasible due to client a restraint so a file is required. .Net comes System.Diagnotics built in, why go log crazy?

A lot of people bash log4net, but it does have the ability to define what and who gets logged, which if you have 100s of classes and lots of assemblies can help out with large scale applications.

Excessive logging is really just a symptom of not doing TDD.

http://carltonnettleton.com/blog/?p=260

Regarding how logging might cause deadlocks, Iā€™m not familiar with Log4Net, but the scenario I envisage would be that some complex object has been passed to the logger, the logger has attempted to convert it to a string, and the objectā€™s method for obtaining a string representation is doing complex stuff - maybe causing object state to be lazy-loaded from the database, that kind of thing.