Friday, December 6, 2019

The Source of Genius

The guy at the next table looks out at the amber sunset, puffs up with gravitas, and announces to his wide-eyed friend “soon all things will be connected seamlessly to a ubiquitous network." Time to change tables. I grab my drink and set off to find an area outside of Houdini’s vocal range. The bar at the Rosewood is cursed by its reputation as the place where VCs from Sand Hill Road meet, so it attracts posers playing the prophet like LA attracts actors.

Coincidentally, on my way over to the Rosewood, I saw along El Camino Real a hand-painted sign saying “clairvoyant conference” with an arrow pointing to a hotel. Imagine a conference for clairvoyants! You would not need to have sessions on “future trends,” since they would already know. But wait: Why the sign giving directions, for that matter?

Since time began, it seems people want to believe that some of us have a special knowledge of what is to come. So we’re vulnerable to those who claim such knowledge - albeit that different people fall for different images of the prophet. You might mock the trappings of the village shaman, the tarot reader, and the astrologist – but I bet you’d clear your calendar to hear the latest word from the valley’s richest VCs.

I’m not just waxing cynical. It turns out that venture capitalists typically are bad at telling the future. As an industry, VCs don’t perform that well financially. To find VCs who outperform the market, you have to selectively sample only the most successful ones - but that is true for slot machines, too. OK, a minority of VCs do tend to appear repeatedly on the winning side, so perhaps they are the real Houdinis. Maybe. Or maybe having been successful they end up getting preferential access to things that continue to make them successful. (I appreciate that advantage, working at Stanford.)

Whatever the reason, the fact that at least some VCs are repeatedly successful has created the mystique that there does exist, somewhere along Sand Hill Road, somebody who does know what’s next. Hence all the puffery at the Rosewood. How are we to know that he’s not really a visionary?

Professor Elizabeth Pontikes and I took a careful look at this question. We collected data on thousands of firms in the software business and looked at their fates over time – including both successes and failures in the data. We found that firms herd into “hot” markets that have been blessed by the VCs. But we also found that the VCs herd into markets too, following each other in financing frenzies. The resulting hype cycles lead to bad outcomes for companies; firms getting funded in these waves are the least likely to ultimately succeed by going public. So much for Houdini.

Perhaps even more interesting, the VCs themselves seem to be aware of this problem. While VCs herd into hot markets, at the same time they try to avoid investing in firms that do so. The VCs prefer instead to invest in those who pioneered what is now a hot market (and survived). As a kid I had a precocious classmate who would jump to the front when he saw a trend, pointing resolutely forward in Napoleonic fashion, proclaiming “follow me!” I’ll have to check; perhaps he grew up to be a VC.


Remember, the most remarkable changes are not predicted by our experts. These acts of genius are pioneered by those foolish enough to ignore the consensus. Be skeptical of those who claim to know what's next.

Then What's the Fuss About "Unicorns"?

OK, Barnett, if we're so bad at predicting genius, then why does everyone make such a fuss about spotting "unicorns"? I ran into this question recently from three very different groups: some top executives of a large American manufacturing firm, a roomful of Russians, and several thousand Chinese entrepreneurs in Shanghai. Very different groups. Very different topics. But one question was asked in every venue: "What business is the next 'unicorn'?"


If you can read then by now you are tired of hearing about unicorns. The internet tells me that the trope refers to startups valued at $1 billion. Most pundits talk about them only after they are valuable, but anybody can look in the rear view mirror. What my audiences worldwide want to know is how to see them coming.

First let's get something straight: Statistics tells us that amazing exceptions will happen, every now and then, at random. In fact, amazing exceptions will even come in bunches every now and then at random, in the same way that my music player will randomly serve up three straight "Steely Dan" songs in a row. Call them what you want; black swans, unicorns, whatever. Unusual exceptions happen at random.

But with unicorn businesses there is a pattern we can see in advance. This pattern won't tell you who is the next unicorn. It will tell you where not to look, however.

Let me explain. Research shows that waves of exuberance about businesses tend to be biased. Since we're all looking to each other to find the next new thing, once a market space starts trending it's bound to get hyped beyond its real potential; that's what the "hype cycle" is all about.  All that buzz makes it much easier to start companies in a hyped space. Ironically, this makes it so that many of the least competitive firms are the ones that herd into the hot markets where everyone wants to invest.

Want to find the next unicorn? Listen to where the buzz is coming from and run the other way. I can't tell you who will be the next unicorn, but I can tell you it will come from where we least expect it.

Then How Do We Organize to Create Genius?

If we cannot predict genius very well, what is a business leader to do if she wants her firm to innovate?

One way to answer this question is to recall an old debate in the management of innovation. Mid-20th century economists, most notably John Kenneth Galbraith, predicted that large companies would be responsible for most of the groundbreaking innovations in our future. History has proved them dead wrong.

In fact, the opposite has turned out to be true. Startups – entrepreneurial companies – are the engine of innovation in the modern economy, and as a result they grow disproportionately faster than large, established firms. And this is true both in terms of their average growth rates and in terms of which companies are more likely to be super-high-growth outliers.

Big firm fans: Before you argue by naming your favorite big-firm innovator, review the quality research on this subject. A large academic literature has thoroughly looked at “Gibrat’s Law of Proportionate Effect” in firm growth rates. That's a baseline model where small, new companies and large, established ones innovate and grow proportionate to their size. If that were true, then entrepreneurship and intrapreneurship would be equally effective. But this literature consistently finds disproportionate growth rates (and more dispersed growth rates) among smaller, newer organizations. We can debate nuances, such as whether large organizations help by investing in R&D that later appears in the innovations of startups, or that there are some exceptional big-company innovators, but there is no doubt of the main finding: Entrepreneurs outperform intrapreneurs.

But what about the studies showing that innovation helps big firms improve? Two problems here. First, when innovation does help large firms, the improvements typically don't kick in until after a difficult and painful period of adjustment. In fact, often large companies fail outright while trying to adjust to innovation. Second, this is a false comparison anyway. Sure, compared to its tired, old self, an innovative big firm is improving. But the correct comparison is to the entrepreneur.

Now you may be wondering, don’t new companies also fail at a higher rate? Yes, they do, but the growth created by the high-growth survivors more than compensates for the economic losses due to failure. Plus, failures fuel future entrepreneurship; the “creative destruction” described by Schumpeter. In any case, if you’re looking for big success, look to entrepreneurs – not to the intrapreneurs funded by large, established companies.

But why?

To understand why entrepreneurs outperform intrapreneurs, you first have to understand how great new innovations take hold. The process happens in two steps:

Step 1: Variation.

Groundbreaking new innovations are typically "nonconsensus" ideas when they start out. That is, some think they are a good idea, but many disagree because (by definition) groundbreaking innovations defy conventional wisdom. I recall when the consensus said that internet search was not a good business. After all, we had seen the failure of Lycos, Excite, Alta Vista, and a bunch of other search firms. So when Google came along few people wanted to fund them. Now, my marketing colleagues tell me that search is the greatest business in the history of humankind. In this way, we're often very bad at predicting which innovations will succeed. But we're so good at retrospectively rationalizing, we forget how bad we are at predicting.

Of course, not all nonconsensus ideas are brilliant. In fact, often they are really bad ideas. We won't know for sure until an entrepreneur proves their genius (or folly) by experimentation. Nonconsensus ideas are not necessarily better, they are just higher variance.

To see what I mean, Consider these two distributions:


Let's say these are plots of the number of innovations produced, ranging from really foolish (far left) to really genius (far right). The top distribution is a high-variance distribution, while the bottom distribution is low variance. Both have the same average, but the top distribution has much more genius (and folly) than the one on the bottom. Nonconsensus ideas are high variance ideas, like the top distribution. If what we want are some really great innovations, we want to generate high-variance distributions like the one on the top.

Entrepreneurs are unpredictable, and uncontrollable. So they are very high variance, looked at as a group. Many are complete fools, and some of the fools will turn out to have been geniuses once we see how history develops. By contrast, intrapreneurs operate within a corporation. They need to justify their expenses. And, even if they are given a great deal of latitude, they must at some point explain themselves. The need to answer to a big, established firm renders intrapreneurship low variance.

For instance, I remember when Hewlett Packard was the model of big-firm intrapreneurship back in the 1990s. I was studying their venture into video networking, a very entrepreneurial move. Jim Olson ran their video division just like a startup, even imitating the physical surroundings and culture of a start-up company. But every year he would have to attend the budget meetings, where the meager returns of the video division were dwarfed by the billions being earned in other parts of the company. So it was that the division began to market a printer that would sit atop your TV, since corporate understood printers. This move helped them in the budgetary negotiations, but in so doing it moved them from high-variance entrepreneurship to the safely low-variance world of intrapreneurship.

Step 2: Selection.

Once an idea is surfaced, the innovation process has just begun. The process of "selection" involves testing the idea in the market. You've probably heard a lot about this process: The innovative firm quickly gets its idea into the market in an effort to improve by failing fast and cheap. This process improves the idea by iterating through trial and error. What you may not realize is that the real beauty of the selection process is when the market test leads to unexpected outcomes - discovery. The most innovative applications of ideas are not the anticipated applications, but rather those that materialize along they way. There is a long list of innovative companies that became raging successes after discovering their unexpected brilliance, including Airbnb, NetApp, and Apple (to name just a few).

Image result for apple lemmingsWhen it comes to discovery, entrepreneurship again has an advantage over intrapreneurship. The intrapreneur's iterative tests are seen through the lens of the parent organization. Imagine when Airbnb was first discovering itself. Had its model been tested by the folks at Hilton or Sheraton, odds are they would not have recognized the very different logic at work. Similarly, NetApp's innovative file servers would have made no sense to IBM - nor would have Apple's odd experiment with iTunes. All these experiments were based on logics that made no sense to the establishment.

Now you may be thinking: Intrapreneurs also engage in discovery. Yes, but they do so through the lens of the status quo. When it comes to discovery, the lens of the status quo creates blinders. By contrast, the entrepreneur can see things in ways that the intrapreneur cannot.

Implications for startups: If you're getting push-back because your idea is nonconsensus, that's a good thing. If the big-company people like your idea, it is time to worry. And if your plans are not working out as planned, that's a good thing; you're discovering.

Implications for big, established companies: Don't reward good innovation. Reward high-variance innovation, good or bad. Evaluate your innovation processes in terms of how they affect variation and selection. Stamping out foolishness? Then you're eliminating any chance of genius.

The Price of Genius

On this note, I am often approached by company executives who read all this and then say something along the lines of "But, Bill, I want innovation - but I want 'good' innovation."

Well, there's your problem. 

Searching for "good" innovation means you want creativity, but without any of the foolish errors. But creativity is not about eliminating error. Creativity's payoff is the occasional burst of genius - and you don't get that by eliminating error. If you were, for example, to start telling the musicians how to look, sound, and act, you'll never get the next Bob Dylan. Systems that sometimes give us the rare genius also give us a lot of foolishness along the way.

Creative systems should be judged, but not by their "average output." Instead, you should judge creative systems by their variance - their ability to produce extreme outcomes whether good or bad. As Professor James March explains, high-variance systems are the most creative. They produce lots of foolishness and, every now and then, a moment of brilliance. If you plan away the foolishness you might improve the "average" result. But planning away foolishness will also reduce variance - which means you eliminate any chance of genius. 


In short, foolishness is the price of genius.


You probably accept this idea when it comes to artistic creativity. But what about in business? Well-meaning business leaders plan away variance all the time. Sometimes that makes sense, of course. I don't want a lot of creative experimentation when we're operating an airport. But when leaders want innovation, they typically put systems in place that reduce variance and raise the average. They ask for "intelligent" innovation, for creativity without foolishness. Don't look to leaders like this for the next breakthrough innovation. They are unwilling to pay the price of genius.



References

Barnett, William P. and Elizabeth G. Pontikes. 2017. "The Nonconsensus Entrepreneur: Organizational Responses to Vital Events." Administrative Science Quarterly. 

March, James G. 1991. "Exploration and Exploitation in Organizational Learning." Organization Science.