For the last few days, I've been surveying a software project. Landing on a planet populated entirely by an alien ecosystem of source code can be overwhelming. That's why the first first thing I do is bust out my software tricorder -- static code analysis tools.
One fine day the upper management at my company decided they wanted ISO certification. Immediate effect - we are supposed to ensure we follow coding guidelines.
While coding guidelines are good to ensure consistency across the teams, often the even more important issues like code correctness, not breaking abstractions or encapsulations are most important.
I personally think the single most important metric for any commercial software developer is how well he meets his specs.
ASML is a way to programmatically write specs so there will be a way to check if code meets specs or not. TDD is another way.
Seshagiri
This is an excellent post. I need to show it to everybody at work. I’m trying to improve our software production process. I’m adding source control (and trying to get comments on the commits), creating an automated build, and trying to get some standards implemented. I’m getting resistance because the head developer thinks it’s impeding his ability to code, but I’m really just trying to make everything easier for the whole team. Perhaps this type of thinking will help him move towards a more team approach without feeling his personal space is being violated.
We’ve recently started doing code coverage of a very large code base that has never used those metrics before. We have a series of automated tests (mostly UI and output tests - not so much unit tests), and it’s been interesting to see how much of the code is covered by the tests.
Unfortunately, the instrumented code writes its results out at application exit. Most of our tests involve starting the application, performing one small operation then quitting. So a set of tests which normally takes about 2 hours to run now takes over 24 hours! So, yeah, measuring it does change things. If we make it perform more tests at a time, then we run the risk of running into memory patterns that might cause a code path to not be tested. (Not that we’re worried about that, but it just occurred to me it could happen.)
But overall, it’s been a pretty cool experience. Now it’s a question of sitting down with QA and automation and telling them, we need to add a test to do A, B, and C to cover this code, etc.
from the wikipedia article you linked to http://en.wikipedia.org/wiki/Uncertainty_principle :
==================
…
Common incorrect explanation of the uncertainty principle
In fact Heisenberg’s Uncertainty Principle means (in supersimple terms) you can know either the exact velocity of a particle or its exact location – but not both!
Interesting §article nontheless.
You’re certainly right that it’s very easy to abuse metrics to micromanage. One reason which I think your alluding to is that by applying metric maids, you end up with bad data.
Truth is, you have to be careful not to follow the adage of "You can’t manage it if you can’t measure it”, because it’s very misleading as well.
I wrote a post about this second thought last month, which I think relates.
Find it at: http://ryan-technorabble.blogspot.com/2006/08/rule.html
It seems to me that the real value of metrics is predictive – if you can find data to measure whose quantity you can predict for future development, you can use actual historical data to get a handle on how long something takes to create. Alas, there are many if’s in there – IF you can find data to measure, and (implicitly) IF you can predict this quantity for future work.
I don’t know about coding, but for writing and editing you can take a gross measure of how much documentation was created between time A and time B, from draft to published doc; if you do this over the course of several projects, you can get a somewhat accurate estimate about how much documentation a team can create in a given time period. If you can then (somehow) estimate the size of a new project, you’ve got a handle on how long it might take.
An important point, it seems to me, is to look at the metrics in a kind of gross way and to NOT qualify them with “well, this was a different kind of project” or “well, three of our folks went on vacation on that project” or the like – because every project introduces some sub-optimal elements like learning curves, unexpected difficulties (“our build server crashed four times”), personel turnover, etc… By essentially ignoring all these, you smooth them out over the course of multiple projects.
Note that I am not a statistician.
I very much agree that metrics can be useful and in the ideal case (as described in Jeff’s post) can help you find places to improve your process, and that using metrics to measure individual productivity is a much more difficult task. Getting overall team productivity numbers is good; beating an individual on the head about stats, while perhaps ok in some schools of management, seems fraught with para-productivity issues, like whether your people quit in disgust.
I’ve used SourceMonitor…
http://www.campwoodsw.com/sourcemonitor.html
To good effect. Does C/C++/C#/Java and a few others.
Hi,
Why does a ‘too small’ method is considered ‘bad’?
Is having a method that contains only one or two lines considered as a violation of some FxCop rule? If so, why?
Thanks
i think the value in knowing the rules given to you by fxcop etc is the ability to know when to break them.
tho why it’s a bad thing having a method with 7±2 parameters is anybodies guess.
Nobody likes a meter maid, but nobody likes people parking for free either.
Indeed… and “free parking” isn’t so free…
7±2 is the rough number of distinct attribute you can remember about an object, tasks you can keep track of etc. Long since forgot where I read this stat. But I think it is referenced in Steven Covey’s books on time management.
Metrics are useful for showing areas that can be improved in code. Any large scale project should use metrics to see if the code base is becoming uncontrollable. A piece of code with a complexity rating of 52 will still work, but it is tougher to maintain, fix bugs, etc. than a piece of code with a complexity of 8. Also, highly complex systems tend to have more bugs in them and are more difficult to test than simpler ones because of the higher number of logic paths.
Anyone can write code, but can you write code that is easier to maintain and update. The metrics should show how you are doing.
Metrics should be a part of a code review process. But most times its not like that because of time crunches etc.
I ran some complexity tools on a project I am working on and many classes and pages over 100 for cyclomatic complexity, grand prize being 382!
Nobody likes a meter maid, but nobody likes people parking for free either.