As good as it gets: Measuring Product Quality

Working in Business Analytics,pretty much everything we measure can fit into one of three buckets: 

How much 𐄿 How Fast 𐄿 How Good

This post is all about measuring "how good".  While it is written in terms of software, you can apply the same framework to anything.

What is "Good"?

In software, How good is sometimes referred to as the "quality" of the software.  You can typically group any quality measure into one of two categories: objective quality and subjective quality.  First, an overview of each type...

Stolen from http://keydifferences.com/difference-between-objective-and-subjective.html

Objective Quality

Does the software do what its specifications dictate?

You order an orange from a restaurant and they return one that is rotten.  That orange is objectively wonky.  Don't eat that orange.

Subjective Quality

Does the thing meet the expectations of the person measuring quality (agent)? 

Note: it's really hard to measure subjective quality without first defining the "agent" who has the expectations.

Businesswoman compares fruit

You order an orange from a restaurant and the waiter gives you an apple.  It might be a fantastic apple, but you wanted an orange.  From what I read, those two fruit are incomparable.

Objective Quality

When measuring objective quality, you will start with some specification for what good software is. These specifications may be outlined in your scoping document or tracked in some other way.  However you keep track of the specifications, it's important they are explicitly documented and unambiguous.  

In software, there are two types of specifications: functional and nonfunctional.

Functional Specifications

Your product has specifications that tell you what it does and how it behaves. 

When I click on this button, the "edit your profile" page loads.  When the pH goes above 8, the water turns green.  The 3/4 inch screw is 3/4 inches long.

Nonfunctional specifications

Your product has specifications that specify how reliably it operations.

The uptime of the website is 99.999%.  The latency of the website is 200ms.  The car doesn't burst into flames when making a left turn. 

Subjective Quality

Unlike objective quality, subjective quality is a measure of how well the product meets the needs or expectations of the stakeholders.  The term "stakeholder" is very broad: it can be any party somewhere in the value-chain.  To keep it simple, we'll focus on the product user for now and call them "agents".

Oh, beautiful sandwich woman.  Who are you?

Example: There's a very disappointing restaurant I eat at.  The sandwiches I order meet their specs: 6-inches and and constructed with the correct ingredients.  Still, I'm usually disappointed when the sandwich doesn't look like the one in the ad with a beautiful woman taking a bite.  

Just like objective specifications, setting the right subjective specifications and documenting them is critical step to achieving them.  Make sure you communicate with the agents early and often!

Now what?

If you're looking for ways to measure and improve the quality of your product, I've found a lot of success with organizing specs into the framework above. With a comprehensive way to think about quality, it is easier to discuss objectives, set targets, and achieve results.  

Try it out and let me know what you find!

If you'd like to read more about measuring "how good", I recommend this great article.