Quantifying the Qualifiable

Ever since I started playing poker seriously in 2004, I have found an odd comfort in the world of incomplete information.  If everyone’s hands were turned face up, and everyone played according to proper mathematical odds, all players at a table would break even long-run (minus the rake).  A small edge would be gained, even if everyone’s hands were face up, by taking advantage of those who made poor mathematical decisions.  But the major edges come from refined analysis when the cards are face down – mathematical errors are amplified & huge profits can be generated if your assessment of potential holdings is even 10% more accurate than your opponents expectation.

The same holds true for early stage venture capital.  We are playing a game of incomplete information.  We make some money at the margin by investing at better valuations than non-institutional investors, but that’s not the game changer.  The differentiator is our analysis of the intangibles – a founding team, market dynamics and evolution, macro-trends that will affect M&A, consumer adoption, the stability of distribution channels, etc etc etc.  The list is never ending.

There are firms that have tried to quantify the entire process algorithmically – Right Side Capital Management is one, although I think they still require a level of human touch.  My issue with a quant approach to VC is that much like poker, the game can’t be solved by bots.  The only form of poker which has been roughly solved is Limit Hold’em which offers only fixed betting amounts and where risk/reward can be measured.  The dynamic betting games, No Limit Hold’em & Pot Limit Omaha have never been cracked by bots at high stakes levels with expert players.  The dynamic nature of the betting provides a nearly infinite variety of outcomes and choice architecture.  Given the level of qualitative analysis in early stage venture capital, it’s my opinion that the process can’t be solved algorithmically.

That said, the process does need to evolve formulaically.  When I started my marketing career, us brand experts/marketers sat around a table and spat out creative ideas, which, because we were smart people, were evidently likely to work.  We threw darts at a wall and ROI was typically undefinable.  That model has flipped.  Few marketers who aren’t data/analytics focused are still in business, and at a minimum, firms are expected to provide detailed analyses of impressions, best distribution channel, demographic feedback, etc.

As a budding VC, my challenge is this: how do I have any clue if I’m good at my job?  Literally.  The feedback loop in VC is 5, 7, maybe 10 years (if not more).  The sample sizes are so small that an entire lifetime might be insufficient.  My poker background cultivated a strong sense of people and personality – their competency, trustworthiness, vision – and I base a lot of my decisions on those instincts.  These instincts are actually better defined as pattern recognition.  Stuart (one of our partners) has bolstered my confidence by investing similarly.  But even if my instincts/pattern recognition are correct (which I wouldn’t know for 10-5,000 years), there’s an entire debate on whether human capital is even the dominant investment attribute.

My plan for the near term is as follows –

  1. I am going to keep an investment journal of all companies that make it to our investment committee (the ones with the most diligence) with a breakdown of whether I would invest or not & why – keeping tabs every 6-12 months on progress.
  2. To start mapping my instincts versus ultimate diligence.  Specifically, rating new leads on a 1-10 scale for attributes such as founder/market fit, distribution channels, market size, market willingness to adopt, and competition – seeing how those scores change as diligence evolves.
  3. To start quantifying my gut reactions – and tracking what % of positive reactions make it to later stages.
  4. Rely on Jim Shrager and Herbert Simon’s analyses of new venture strategy to efficiently hone in on the underlying value prop & sustainability of all businesses we see.
If any other VCs or even wannabe VCs are reading this, I’d be curious how you’re approaching the question of “how do I know if I’m good (or will ever be) at this job.”  I love my job and I want to be great.  Is it pattern recognition?  Instinctual?  Luck?

Thanks for the feedback,

About the author

Ezra Galston
Ezra Galston

Consumer focused hustling @Chicago Ventures, Young Entrepreneur @Foundation Capital, Class 18 @Kauffman Fellow, and Chicago Booth MBA. Former professional poker player, with 4 years experience doing marketing/biz dev in the online gaming industry. Launched a "poker hedge fund" in 2011, a record label in College, and produced a festival screened short film in 2006.

  • Theresa Johnson

    Nice post, very thoughtful. I think you can achieve the same sort of pattern recognition, but looking at the patterns of others, vs your own. You have access to a huge swath of current and past startups, whether successful or not. More importantly, you might have first hand access to start-ups further down the path than you’re looking to invest. Can you keep a record of what works with these teams and the eventual outcome? This at least lets cut down on the feedback loop.

    At a gut level, I also agree with the “horse” investing strategy. As a VC, you have more time than the average entrepreneur to get to know the “maze”. Therein lies the value add, and possibly ticket to success. http://cdixon.org/2013/08/04/the-idea-maze/

    • egalston

      Thanks Theresa, and I also loved Dixon’s post a couple days ago.

      My struggle is just that I don’t really have access to the data I want. VC is so qualitative that I think it’s impractical to try and speed up the long-run by using data from Crunchbase or Techcrunch or Venturebeat. So much of the pattern recognition is based on founder focus & vision which can’t come out from public sources. If I tried really hard, I could probably dig up early early decks (I know Airbnb made theirs public) but even that wouldn’t give me the interpersonal data I want.

      I’m still struggling.

  • egalston

    Also, I should point out that there are bots beating some dynamic betting poker games, but they are winning only at low to medium stakes, and are winning on the margin by making perfect mathematical decisions (based on estimated ranges & normalized betting) but are not beating experts who innovate their hand ranges and bet in non-traditional increments and patterns.

  • Hi Over here for the first time from AVC

    I find your discussion fascinating, but one aspect that concerns me is that any algorithm that seeks to deploy pattern recognition relies on feature extraction. This in turn relies on domain knowledge. And that on experience. Therefore it will overlook outliers, and real disruption particularly of unserved markets takes place in information voids.

    I may be wrong but the real value of talent spotters for VC would be those outside the envelope but none-the-less-interesting or so I would think

  • Jay

    I like how you use what you know to master your job. I feel like being able to achieve high reward more than high risk you have to learn how to notice when your gut is right. Your gut takes everything your body knows and makes a decision in an instant. And I think by you using that skill as a pro poker player directly gives you the interior of a great VC. I believe you know you have what it takes and you are just waiting for the rest of the world to see the proof.

  • Jeffrey Wan

    Really thoughtful stuff and I think you’ve identified some tough problems in VC and the investment industry as a whole. There are fewer quantifiable factors in investment than more model-able games like poker and I think that’s a tough fact to grapple with. How do process-oriented people verify their processes and mindset? What does one do?

    • egalston

      It’s tough. I think bc of my poker background and use of tools such as Hold’em Manager I have this expectation that everything is quantifiable and cab be assessed as a +EV or -EV decision. I haven’t yet found comerable tools for VC. My goal is to try and identify certain winning patterns (I think I’m making progress on this)…Bill Gurley identified 10 factors for marketplace success and I think the same analysis can ultimately be built for most industries. That said, scoring 10/10 doesn’t guarantee success whereas getting the money in in a poker game with 100% equity would guarantee success.

      • Jeffrey Wan

        This makes sense. I imagine identifying patterns outside of a vacuum must be difficult… no two situations/founders/ideas/timing factors are exactly the same in history and so identifying some factors correlated with success must be difficult. I imagine some combination of history and common sense are needed. But, I’m sure a lot of valuable qualitative details can be obtained through interaction with founders… and this is valuable non-quantifiable information to factor in a VC’s success model.

  • Hey Ezra — wannabe VC with some input from a different angle.

    I’ve been doing lots of recruiting for my company lately,
    and in doing so have been trying to collect a lot of metrics that sound very
    similar to what you’re capturing in your investment journal. I’m trying to
    quantify things like ‘cultural fit’ and ‘soft skills’ (and I rate 1-5 as of
    now, not out of 10 like you), but I think the base principles are pretty
    similar because at the end of the day we basically have a ‘best horse’
    recruiting thesis. I imagine this is similar to trying to court the best
    early-stage founders/companies/opportunities that you deal with every day.

    Here’s the twist that I’ve applied to my methodology: as I
    gather more metrics on our hiring process, I am trying to think about hiring
    the way Patrick McKinzie (patio11 on HN and elsewhere) thinks about conversion
    optimization–as a form of marketing that is amenable to engineering students. Or,
    in your case, I suppose that would mean to think about deal sourcing and
    closing as a form of marketing amenable to founders and their companies?

    By the way, a lot of this thinking was lifted from a great HN comment
    by tpatcek (another Chicagoan, and one of my favorite HN commenters), so credit
    Thomas for this idea.

    When I think about it this way, it boils down to just
    setting up a standardized funnel that aims for conversion optimization. In my
    case, a conversion means that I’ve tracked a candidate all the way through my
    funnel and successfully hired them into our company. It sounds like you have a
    marketing background so you probably have a good idea of what your funnel looks
    like, even if you haven’t thought of it in this light before. I would guess
    that it starts with you sourcing the deal, and ends with you closing a deal
    with favorable terms. As your investments mature, maybe it makes sense to
    extend this funnel to the time when you reach an exit, and see how that changes
    your metrics. For what it’s worth, defining the funnel was one of the hard
    parts of this for me because it really dictated what I had to measure and when.

    Anyway, once the funnel is set up and we’re ranking
    interactions on a standardized scale, it becomes a matter of revising these
    qualifiers (how well do these qualifiers predict how far people get into the
    funnel?) and continuing to put a lot of rigor around the hiring process. We
    want to continue to standardize every part of the recruiting process…we don’t
    think we’ll ever look back at something we’ve standardized or measured and say
    “you know, this wasn’t worth the effort or the cost in flexibility”.

    The biggest takeaways for us are realizations of some blind
    spots that we’ve been missing. Because at the end of the day when it’s time to
    make offers to our recruits, of course everyone feels all warm and fuzzy
    inside, and happy about the candidates we’ve ended up with—most would tell you
    that we’re only offering the best of the best, and that the class is very
    strong. So measuring everything from the get-go reveals some insights we haven’t
    had, and wouldn’t have been possible, before like: are we ranking too many or
    not enough kids as top prospects? Are we not doing a good enough job of
    converting our top prospects? These are insights that we missed before because
    by the end of the process, you assume that the people that stuck around are
    your top prospects because they’re the best of the rest. But is that the whole

    I’d be really interested to hear if and how you’ve applied
    similar thinking to your investment committee outcomes. Sounds like we’re
    trying to beat two different challenges with the same stick, it’d be great to
    compare notes!


Copyright © 2014. Created by Meks. Powered by WordPress.

Get BreakingVC To Your Inbox

Join hundreds of other operators & investors who get BreakingVC updates directly to their inbox.
Email address
Secure and Spam free...