Chapter 7 - Quantifying the value of Information

 

Chapter 7 - Quantifying the value of Information




Accept the fact that not all information is equally valuable for making a certain decision. Certain information is more valuable than other.

If you knew what exact information was the most valuable to make a certain decision, you would perhaps straight shoot of it and not waste your energy with other kinds of information

For example, when trying to get healthy, if you know the most valuable parameter to you is muscle mass, you wont waste your time with a weighing scale, rather you would go with TNT impedence machine.

The McNamara Fallacy “The first step is to measure whatever can be easily measured. This is okay as far as it goes.

The second step is to disregard that which can’t easily be measured or to give it an arbitrary quantitative value. This is artificial and misleading.

The third step is to presume that what can’t be measured easily isn’t important. This is blindness.

The fourth step is to say that what can’t easily be measured really doesn’t exist. This is suicide.”

—Charles Handy, The Empty Raincoat (1995), describing the Vietnam-era measurement policies of Secretary of Defense Robert McNamara

The value of information regarding its effect on human behaviour is, of course, exactly equal to the value of the difference in human behaviour.

You get new information, you behave different because you know something they dont.

The Chance of Being Wrong and the Cost of Being Wrong: Expected Opportunity Loss

The cost of being wrong is the difference between the wrong choice you took and the best alternative available - that is, the one you would have shosen if you had perfect information.

TO COMPUTE THE VALUE OF MEASURING THE LIKELIHOOD OF SUCCESS, YOU HAVE TO KNOW BOTH -

  1. WHAT YOUR LOSS WOULD BE IF IT IS A BAD INVESTMENT
  2. THE CHANCE IT WILL TURN OUT TO BE A BAD INVESTMENT

Opportunity loss (OL) would be just the cost of being wrong but Expected Opportunity Loss (EOL) is slightly different.

EOL is the chance of being wrong X cost of being wrong.

Basically you can say that EOL is the probability-weighted average version of OL.




EOL if approved - 5M x 40% = 2M

EOL if rejected - 40M x 60% = 24M




Value of information depends on how much additional reduction can it bring in the EOL value after aquisicition of that information.

The bigger the difference in before and after EOLs, the higher is that value of that information

If the information is perfect, Chance of being wrong would be 0, so there would be no EOL after.

EVPI = EOL before info

Now lets take these same examples onto Ranges....

The Concept of Loss Function -

In the previous 2 static outcome example (40Mil and 5 Mil) the results were singular and not a range. But if we make the sales or profit or loss into a range of values, the model becomes more realistic.

Lets say a calibrated expert in Marketing was 90% certain that the additional sales resulting from the ad campaign could be anywhere from 150,000 units to 300,000 units. However to breakeven the amount spent of ad campaign we have to sell atleast 200,000 additional units. (25$ per unit cost and 5Mil $ ad budget)

Anything above 200,000 = profit (25x Number of units sold above 200,000)

Sales = 200,000 = No profit or loss

Anything below 200,000 = Loss

Sales = 0 = Loss of 5Mil ie the entire budget of Ad campaign

This is the situation, but lets not forget our objective here which is understanding the value of information that will reduce the uncertainity about the effect of the campaign.

So that brings the topic of loss function - A formula that computes how much we lose depending on the outcome.

Less than 200,000 units sold, Loss = (200,000 - units sold) X 25

200,000 or more sold = Loss is 0





To be Continued....

Comments

Post a Comment

Popular posts from this blog

Chapter 6 Quantifying Risk through Modeling

Image
  Quantifying Risk through Modeling It is better to be approximately right than to be precisely wrong. HOW NOT TO QUANTIFY RISK Scoring method with all that colourful heatmap is utter bullshit. There is no evidence that it works and in fact there is evidence that it works in a bad way because it increases people's confidence in their risk management without actually managing risk. More on this perhaps in Hubbard's other book called "The Failure of Risk Management", which I will have to read someday. Insurance Premiums are good example of how rightly they quantify risk to a certain dollar value. Based on the level of risk you want to mitigate, your insurance premiums will be charged. They dont score risk in these instances as "Low" "Medium" or "High" LETS TALK ABOUT THE REAL SHIT - MONTE CARLO When you have a precise value of something it is easy to add, substract, multiply and divide them, but what do you do when you dont have a ...
Read more

Chapter 1 The Challenge of Intangibles

Image
  PEOPLE WHO GAVE US UNITS OF MEASUREMENT The book starts with a quote from Lord Kelvin, this is the guy who gave us the unit of measurement for Heat. I remember his weird experiment to measure heat from my school days. Its interesting that this guy is mentioned at the start of this book. In a way this quote is not just point to him, but everyone in the past who introduced any units of measurement. The whole concept of coming up with a unit of measurement requires a sophisticated understanding, so all these guys may be a potential mentor. Anyway the quote goes like... "When you can measure what you are speaking about, an express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of science." Basically Einstein said, unless you can teach it, you have not really learnt it.. and Lord Kelv...
Read more

Chapter 3 THE ILLUSION OF INTANGIBLES: WHY IMMEASURABLES AREN'T

Image
  3 reasons why people think something cant be measured. They are rather misunderstandings. Concept - People misunderstand the concept of measurement itself. Once, you get it, you will say a lot more things are measurable than you thought before. Object - The things being measured is not very well defined. Sloppy and Ambiguous language is used. Method - Many people are simply not familiar with different methods to measure things. 3 reasons why people think something shouldnt be measured. Economic Objection - Too expensive to measure General Objection to meaningless of Statistics - You can prove anything with Stats. Circle Jerking BS. Ethical Objection - Dont wanna ask the hard questions needed to measure it. The Concept of Measurement Only an inexact man would call things exact. Measurement is not perfect nor certain. Defn of Measurement - Quantitaively expressed, reduction of uncertainity, based on one or more observations. Measurement is mere reduction of uncerta...
Read more