Annie Duke: Innovating in Bets – a16z Podcast with Marc Andreessen and Sonal Chokshi

Check out The a16z Podcast Episode Page & Show Notes

Key Takeaways

  • Once the outcome of a decision is known, it’s hard to look back retrospectively to determine its quality
  • When analyzing a decision always look up, down, and orthogonal – ask:
    • “Could I have lost less?”
    • “Should I have lost more?”
    • “Should I have even been in this position at all?”
  • It’s very hard to get the non-consensus/right results unless you’re willing to take risks and take on the non-consensus/wrong results
  • Not making a decision IS making a decision
    • This has to do with ALL the resources you invest – time, money, attention, and energy
  • When considering options for a decision, zoom out and ask:
    • What does my life look like going with Option A/B/C a year from now?
  • “Don’t confuse confidence and certainty”
    • You can express uncertainty in a decision and still convey confidence
  • You make THOUSANDS of decisions per day
    • Getting even a little better at the art of decision-making by removing biases and thinking more probabilistically will prove very worthwhile

Intro

Decision-Making

  • Once you know the outcome of a decision, it’s hard to look back retrospectively to determine its quality (this is known as resulting)
  • Analytics show that in the NFL, if you’re on your own one-yard line on 4th down, you should always go for it
    • Why? – If you punt, you’ll likely only punt to mid-field (so the other team is essentially guaranteed 3 points anyway)
    • BUT – this is very uncommon, most teams punt in this situation
  • Human beings very often choose the less controversial option (to avoid judgment) even though it might not be the most statistically optimal

The Decision-Making Matrix

  • Marc brings up the above 2×2 grid (right/wrong is related to the outcome)
    • “Consensus/right is fine, non-consensus/right is fine, consensus/wrong is fine, non-consensus/wrong is REALLY bad”
  • Some notes:
    • For consensus/wrong decisions, that consensus almost always provides an invisible clock against blame
      • For example – If someone dies in a car crash, we don’t say, “What a moron for getting in a car”
    • If an autonomous vehicle kills a pedestrian, it currently falls under the “non-consensus/wrong category” 
      • Now consider the reaction when a pedestrian dies because of a human driver – it falls under the “consensus/wrong category”
    • Similarly, for market crashes:
      • When a crash results from human beings selling stock, people say – “The market went down today”
      • But when an algorithm causes a crash – It’s a “flash crash”
        • This results in people pointing fingers and discussing whether algorithmic trading should even be allowed

A Good Thought From Marc

  • It’s very hard to get the non-consensus/right results unless you’re willing to take risks and take on the non-consensus/wrong results
    • But people just emotionally can’t handle the non-consensus/wrong outcomes and will do nearly anything to avoid them (no matter how much we understand the above)
    • What can we do about this?
      • Revert from:
        • Giving non-consensus/right decisions too much praise
        • Giving non-consensus/wrong too much criticism

Outcomes and Process

  • In the workplace, outcomes tend to dominate
    • If sales were 10% above expected, everything’s fine and everybody goes about their day (no one’s trying to figure out why this happened)
    • But if sales were 10% lower than expected – all hell breaks loose, resulting in a discussion about what went wrong
  • “You have to make it very clear to the people who work for you that you understand good outcomes will come from good processes”
  • Advice for the workplace:
    • Focus more on forecast quality (aka the model quality) rather than outcome quality
      • In addition, know in advance what would have to be true (or the circumstance that would have to present themselves) for you to rethink/re-evaluate your model
    • Always look up, down, and orthogonal – ask:
      • “Could we have lost less?”
      • “Should we have lost more?”
        • For example:
          • If sales were 10% lower than expected, don’t just try to figure out why sales weren’t 10% higher, also consider the fact that perhaps you should have lost more, and you actually got lucky
          • In the VC world, why does no one consider the question (related to a bad investment) – “Maybe we should have invested more money and we actually got lucky?”
      • “Should we have even been in this position at all?”
    • ALSO – In investing, when discussing a win, consider the fact that you may have “oversized the bet” and actually got bailed out by a fluke win

No Decision IS a Decision

  • “Not making a decision is making a decision, we just don’t think about it that way”
    • This has to do with ALL the resources you invest – time, money, attention, and energy
  • REALIZE – There is a set of possible futures that exist for NOT making a decision, in addition to the possible futures that exist from choosing among a set of options

Time Traveling

  • When considering options for a decision, zoom out and ask:
    • What does my life look like going with Option A/B/C a year from now?
  • Related to this: “Doing certain things today is like stealing from your future self” – Sonal

Decision-Making Tips

  • When soliciting feedback, don’t do it in a group setting (to avoid the pile-on effect)
    • Go to people individually
  • “Don’t confuse confidence and certainty”
    • You can express uncertainty in a decision and still convey confidence
      • For example:
        • Option A will work out 60% of the time (what you choose)
        • Option B will work out 30% of the time
        • Option C will work out 10% of the time
    • To better express uncertainty when talking to people, use percentages
      • For example – “I’ll get this to you by Friday 80% of the time and by Monday 99% of the time”
  • Instead of asking, “Are you sure?”, ask – “How sure are you?”

Parting Advice

  • You don’t need to improve that much to get really big dividends
    • You make THOUSANDS of decisions per day
      • Getting even a little better at the art of decision-making by removing biases and thinking more probabilistically will prove very worthwhile

These notes were edited by RoRoPa Editing Services

Bookmark