Assessing The Benefits Of Technology

Chief Executive Officer at Nucleus Research, a leading provider of ROI-focused technology research and advisory services.

No one would question that in the last 50 years, technology has had a significant impact on reducing costs and increasing worker productivity. At each step, from the earliest mainframes to the cloud, increased productivity and reduced costs have been the clear drivers of technology adoption. 

What hasn’t been so clear is how to credibly quantify these benefits when building an ROI business case for a technology decision. I’ve seen far too many decisions based on little more than gut feel, the desire to reduce risk or the end of support for a version of software. With a proper business case, results can be compared between projects and organizations can make rational decisions. As the economy tightens and competition increases, making good technology decisions and being quick to discard bad ones is a competitive advantage organizations can’t overlook. But this relies on a fair measurement of the benefits.

Whether you’re a vendor describing the benefits of your solution or an organization estimating the benefits you will achieve from a technology purchase, understanding the difference between a strong benefit and a weak benefit is critical. At my company, we find that most people, when asked, will broadly categorize benefits as hard or soft. That might work in some cases, but there may be a better framework for assessing the value of a benefit. 

Rethinking How We Consider Benefits 

A business case is an estimate built on a series of smaller estimates that generate that result. Each of these mini estimates hinges on two factors: believability and, to a far lesser extent, variability. In most cases, these two factors are correlated, with an estimate having a wide variation being less believable. It’s not hard to see that the believability of the business case is built on the cumulative believability of each benefit — a nice way of saying the more fiction in the benefits, the less believable the final business case.

In reality, there are four types of benefits, not two. Let’s create a new framework that ranks benefits based on believability. We’ll use four categories to give us an idea of a strong benefit versus a dubious one.

First-Order Benefits

A first-order benefit is a benefit that will absolutely be achieved — without doubt about the outcome or the amount. If moving to the cloud will eliminate the maintenance cost on a server, the amount of which is budgeted and known, there’s a 100% chance that benefit will be achieved. Clearly, this is a strong benefit and has 100% believability in the business case.

Second-Order Benefits

A second-order benefit is essentially a first-order benefit that includes a hedging word. You can recognize it when a benefit includes words such as “expect” (or “plan,” “intend,” “hope,” etc.). For example, a benefit might be described as: “By moving half of our applications to the cloud, we should be able to reduce the energy costs by 50%.” That benefit is very likely true, but the amount is also likely to vary a bit. The benefit is believable and is strong, but it is not as strong as a first-order benefit. We consider that type of benefit as 70% believable.

Third-Order Benefits

A third-order benefit is an increase in worker productivity. It starts with two quick tests before you estimate the value of the actual benefit. First, do you believe in a broad sense that there is a benefit? And second, does the technology directly touch the user achieving that benefit? Let’s say you are trying to estimate the value of a new smartphone app for your sales team. The first test would be to look at the app and see if you agree that it delivers value. Likely true. The second test is that the app directly impacts the user. This is true because the salesperson holds the smartphone in their hand and directly interacts with the app. That makes it a third-order benefit.

How do we calculate the value of a third-order benefit? It’s not as hard as it seems. Start by recognizing that all productivity benefits follow a bell curve. Users will achieve more or less, but there’s no perfect number. Surveying users after showing them the application, measuring the time taken during a workflow and asking a vendor’s reference customers about their experiences are all reasonable approaches to making an estimate. We’ve found there are often at least two ways to generate an estimate for a productivity gain. Still, given the high degree of variability, we tend to put a 40% believability number on any third-order benefit.

Fourth-Order Benefits

A fourth-order benefit is a disconnected benefit, one with multiple steps from the technology to the benefit, and it includes joining words such as “and” or “then.” If a benefit starts to sound like a rambling story, it’s a fourth-order benefit. For example, one cloud provider insisted its application would increase profit margin by 2% and generate 10% more new products each year based on nothing more than one customer’s claims. Another vendor insisted it would increase average basket size for e-commerce sites by 20% based on a loose link between website monitoring (what it did) and a report on the internet that tied website performance to buying behavior. A fourth-order benefit has effectively no believability and is best ignored when developing a business case.

The strength of a business case is built on the strength of the benefits that drive its results. Generating a high ROI on weak benefits is more likely to damage your case rather than strengthen it. Using the four orders as a framework, you can quickly understand the strength of your proposal and assess your likelihood of success. In most cases, it’s better to discard weak benefits that do more to undermine believability than add to your ROI.


Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?


Source Article