October 20, 2020

How to Decide (E.P. 22) | Annie Duke on Infinite Loops Podcast with Jim O’Shaughnessy

We cannot assess decision quality objectively
- What we can assess is the outcome of that decision
- The problem is that we tend to confuse a good outcome with a good decision and vice versa
Dealing with The Hindsight Bias
- Create a knowledge tracker asking yourself:
  - What information did I know, or was knowable, beforehand?
    - Could I have know other relevant information at a reasonable cost?
  - What decision did I make?
  - What things were revealed afterward?
  - What was the outcome?
  - What other outcomes could have happened and what were their probabilities?
- It’s way better to start tracking all this information as part of the decision-making process
Increased uncertainty from COVID-19, requires us to document our decision-making processes even more accurately that we normally would
When we have a thesis, we should ask ourselves how we could find out if our thesis is wrong
- “If that’s not the way you are thinking you will not be successful, except by accident” – Annie Duke

Annie Duke (@AnnieDuke) is a World Series of Poker bracelet winner, an expert in decision making, and a best-selling author
- Check out Annie’s website
- Check out these podcasts notes from Annie’s appearance on the 16z Podcast
Hosts – Jim O’Shaughnessy (@jposhaughnessy)
In this chat, Annie discusses the biases in our decision making, how to mitigate them, and much more

In Thinking in Bets, Annie was focused on the issue of uncertainty in decision-making
- We don’t have all the information we need for a decision
- We don’t have control over the outcome
- The book focused more on “why” we have to deal with uncertainty and less on “how”
How to Decide, was Annie’s exploration of “how” we can make better decisions in the face of uncertainty and imperfect information

We cannot assess decision quality objectively
- Instead, we can assess the outcome of that decision
- The problem is that we tend to confuse a good outcome with a good decision and vice versa
  - Daniel Kahneman calls this “the substitution”
A related problem is that we have the Hindsight Bias
- After the outcome happens, we are biased to think that we should have seen it coming, that it was obvious
- A real-life example: Hillary Clinton’s 2016 Presidential campaign
  - Today, everyone agrees that she screwed up badly in Michigan, Wisconsin, and Pennsylvania
    - It seems so obvious in hindsight
  - But why did nobody write about it before the election?

With regards to a past decision you made:
- Create a knowledge tracker asking yourself:
  - What information did I know, or was knowable, beforehand?
    - Could I have know other relevant information at a reasonable cost?
  - What decision did I make?
  - What things were revealed afterward?
  - What was the outcome?
  - What other outcomes could have happened and what were their probabilities?
It’s way better to start tracking all this information as part of the decision-making process
- It won’t remove all the bias, but it will make you more objective

The hindsight bias is going to be amplified as information comes at us more rapidly
More uncertainty requires that we document our decision-making processes even more accurately that we normally would

A lot of our biases come from the confidence in our own narrative and our “gut”
- Relying on our gut is attractive because nobody can examine and judge our decision-making process
- This is also the big limitation of making decisions based on intuition
- It doesn’t create any accountability or evidentiary record for the decision
Good decision-makers are aware of the limitations of their gut and compensate for them through analysis

When we have a thesis, we should ask ourselves how we could find out if our thesis is wrong
- We actively try to disprove our thesis
  - “If that’s not the way you are thinking you will not be successful, except by accident” – Annie Duke
- This way of thinking is the basis of the Scientific Method, as explained by Karl Popper
Many people are very attached to their ideas and find it difficult to try disproving them
- We like to keep a positive idea of ourselves
  - Our identity feels threatened if our beliefs are doubted

Our decisions are based on very limited information
- “We have this huge problem as decision-makers because, the stuff we know is a speck of dust, and the stuff we don’t know is like the size of the whole freaking Universe” – Annie Duke
To make things worse, there are inaccuracies in the things that we believe
Looking for the things we don’t know can help us to deal with both of our information issues
- This process also helps us to mitigate our confirmation bias

Most of us tend to avoid interacting with people who hold different beliefs
- Each person knows they are not going to change their mind so they assume the conversation would be useless
Actually, such conversations are a huge opportunity for improving, even if we are sure we are right
- We often don’t know why we hold our beliefs
  - The conversation forces us to search for more information to explain our belief
    - A great way to start exploring the things we don’t know

Often you think you know what you want to say, but when you sit to write you realize how much you don’t know
- You are forced to research more deeply, understand what you don’t know
- To explain concepts in a simple way, you have to understand them at an even deeper level
While writing, every day you think “I’m an idiot”, but at the end, you’re so much better
People who write often look at their past work and realize how wrong they were

If Annie could make any changes in the World she would:
- Ensure that markets are actually free
  - “Free market is the best expression of betting on human ingenuity” – Jim O’Shaughnessy
- Make executive, legislative and judicial branches of government actually work independently and check on each other
Technology and Decision Making
- Machine learning can tell us what and when to do something, but not why
  - Most people rely on narrative for decision making
    - Without a “why”, they may miss out on an opportunity to improve their decisions

decision making, mental models
Notes By Giorgio Parlato