Finally we decided to apply a little research methodology to his conundrum. We listed all the key factors that were relevant to his decision – size, price, location, proximity to employees’ residences, ambience, availability of broadband, and a couple of others. My friend defined the relative importance of each of the factors and we assigned weights to the factors accordingly. Then we rated each office on these factors on a scale of one to five. We could then calculate the weighted rating for each property – and voila! the decision was clear. He signed on the dotted line and slept peacefully that night!
Sounds simple? Well it is actually not….
Businesses can apply a rating methodology described above for business decisions in many situations – selecting a location for their office or manufacturing plant, identifying a target company to acquire, selecting a vendor or a partner, and so on..
But is the methodology reliable? Will it lead to the “right” decision?
A reliable methodology is one that gives the same answer each time, no matter who uses the methodology. In a rating methodology the final outcome is not at all definitive.
So where are the holes in it?
- First you need to be sure that you have listed all the relevant factors. If you miss an important one, you are immediately on the wrong track. This is far from trivial. In a recent study, we asked several people about the most important factors they considered while taking a home loan. We got several ones such as interest rate, amount of down payment, level of documentation/paperwork, time taken for getting an approval, and so on. We thought we understood how decisions were made. But subsequently, when they took loans, we found that some of them actually selected companies because their friend or relative worked. This factor has not featured in our initial research at all!
- Secondly, the weights will impact the final outcome. Sometimes, during research, people tell you what that they think they ought to do. So for instance, one ought to choose a healthy meal option over junk food. So for example you would give higher weight to the nutrition level than taste while selecting where to eat. However, when you go out, you may actually end up selecting junk food. So the weights were in fact not correct. Changing the weights can change your final decision. In business situations, different factors may be important to different people in an organisation, and so may their weights. In such situations, an Analytic Hierarchy Process (AHP) may be applied.
- Thirdly, the rating scale can also impact your decision. Are you using a 3 point scale or a 5 point scale or a 10 point scale. With very few points on the scale, the ranges for each rating are wider and you could miss differentiating between essentially different options. On the other hand too many points on the scale create confusion.
So what can you do? Is the methodology not as useful as you thought?
It is a very useful tool for decision making if used correctly. To ensure that your finding is reliable, you need to:
- Check how the researcher zeroed in on the factors
- Check how the weights were decided
- Examine the rationale behind the rating scales
- Finally you must do a sensitivity check. Check the outcome while varying various factors to see what factors the outcome is most sensitive to.