Nice to be back after a relaxing vacation. In this first post about product SW quality I want to focus on number 2 from my post on the 22nd of February. Lack of quality control and good quality code measures (software Metrics). Most companies measure SW quality in some form, for example number of open defects often connected to a ranking system. This is one simple way of measuring product quality, even if it standalone will not tell you very much. To be able to get a full overview of your products quality you need more than just one measure, you need to find a number of measures that give you the full picture of your product SW quality. However, first I want to elaborate some on what I mean with SW quality. There are of course a lot of things that can fit into this term. My number one focus is on product SW quality and the main quality factors for me are reliability, usability and correctness. Of course you also have many other factors like maintainability, reusability, portability etc but it is not as important as the other three in succeeding with a good customer "perceived" quality level (looking from the out side an in).
Below you can see the metrics I think you need to use on a top level:
- Reliability: Mean time between failure (MTBF), the S-curve (accumulated number of found defects over time) and SW complexity metrics (i.e. Cyclomatic complexity).
- Correctness: Open defects rank of A, B and C, % pass rate for test cases (all test areas: i.e. function test, integration test etc), test coverage, performance metrics.
- Usability: Performance metrics (latency), effectiveness, efficiency, user satisfaction (or perceived quality).
If you make sure you get reliable data on the above quality factors you are getting very good input into how to improve you product quality. More importantly you will also have the right information on your product quality status to be able to make the often hard decision on when to release you product to real customers.
Even the most mature companies I worked at best case use most of the correctness quality factor and MTBF from the reliability quality factor. This often lead to products that are released too early (i.e. with too low quality), since missing the S-curve and a good complexity metrics you miss 2 out of 3 metrics for reliability. The SW complexity metrics has been around for over 20 years, but I have yet see any company using them, even if there has been automated tools available since at least 15 years back. I myself did my master thesis at Volvo Cars in 1995 on SW complexity metrics (I do not know if they ever went ahead and implemented my suggestions :-). However in TFS 2010 you will find a automated tool for measuring Cyclomatic complexity, maybe that will make it more frequently used?
That’s it for today. In coming posts I will drill down to details for each quality factor.
Kind regards
Anders
Posts
To me correctness and usability are factors which is measured at a specific point in time (release date?) and thus it measures the quality at this release date. This says nothing about future quality.
ReplyDeleteTo me, as a SW developer, maintainability, reusability and portability means that future SW releases chance of quality should be higher thus I think they should be valued much more. Plus the code is much easier and more fun to work with if the SW code are designed using these principles.
To dare to refactor the SW code base is the key.
About reliability I agree.
Just my point of view :-)
/ Henrik
Hi Henrik,
ReplyDeleteThank you for your feedback.
I agree with you. Maybe I have not been clear on from where I am coming from. I am talking more about an organization that has not been use to more or less any measure and have low quality out put. Then I think the focus should be on correctness etc. However in the next step on a journey to increasing quality, I think it is important to also push maintainability, reusability, portability and other quality factors. Best of course would be to be able to push all quality factors at once, but in reality it is often too much for the organization to absorb at once is my experience. Or have you seen away to do this all at once it would be great to hear more about that? Do you have any experinace from complexity metrics?
Kind regards
Anders