The “Pruning Process of AI Investing”
Parallels from Brain Development and what it teaches us about Innovation, Boom-and-Bust Cycles, Investing, and AI.
Welcome to issue #11 of Indiscrete Musings
I write about the world of Cloud Computing and Venture Capital and will most likely fall off the path from time to time. You can expect a bi-weekly to monthly update on specific sectors with Cloud Computing or uncuffed thoughts on the somewhat opaque world that is Venture Capital. I’ll be mostly wrong and sometimes right. Views my own.
Please feel free to subscribe, forward, and share. For more random musings, follow @MrRazzi17
I think that’s exactly what you see going on here. Every experiment [on the Internet] is getting tried. Many of them are going to succeed, and many of them are going to fail. — Jeff Bezos, Internet summit, 1999
The quote above sounds timely and apt as it relates to this new world of AI that we entered at hypersonic speed in 2023. The words “Prompts,” “LLMs,” “GPT,” and “Generative AI” may have sounded like another language circa 2022 and 2021. While I’m optimistic about the future and where the world of AI is going (and can go, assuming the lack of regulatory constraints), the exuberance is far too nostalgic when compared to other historical technology breakthroughs.
Whenever I start to feel smart (which is rare), I take a good look at 19-month-old twins–they learn at a staggering pace. Research shows that preliterate children learn a new word every two hours they are awake, and they are on their way to knowing approximately 45,000 words by high school graduation. Young children possess an extraordinary aptitude for learning under the premise that their surroundings facilitate such cognitive development. In contrast, adults encounter more significant challenges in assimilating vast amounts of new information. For example, acquiring proficiency in a second language proves to be significantly more formidable for a middle-aged adult compared to a young child. The rationale behind this discrepancy not only holds intrigue within the realm of child development but also furnishes a valuable perspective for contemplating the dynamics of innovation, technology breakthroughs, and investing.
From the moment of a child's birth until the age of three, there is a remarkable surge in the number of synapses—neuronal connections—within the brain. To provide perspective, a toddler boasts an impressive one quadrillion synaptic connections (source), twice the number found in an adult brain. Children exhibit brains that are notably more active, interconnected, and adaptable compared to those of their adult counterparts. However, this phase of synaptic abundance is succeeded by a substantial pruning process.
This pruning process, often referred to as the Hebbian process after psychologist Donald Hebbs, involves reinforcing beneficial synaptic connections through experience while eliminating those that remain inactive. Astonishingly, estimates suggest that young children shed approximately 20 billion synaptic connections each day. This intricate process serves to fine-tune the brain, enabling it to navigate and thrive adeptly within its specific environment. By the time individuals reach adulthood, synaptic selection has played a pivotal role in shaping their brains for success.
While the idea of synaptic overproduction and subsequent pruning may seem extravagant, especially considering its cost in terms of neural components and energy, evolution has permitted this apparently lavish process to persist. Nature's ingenuity becomes apparent when examining models of neural networks, revealing that the overproduction/pruning approach is remarkably flexible and more dependable at preserving information than a simplified feed-forward network. The strategy of commencing with numerous alternatives and progressively refining to the most essential ones proves to be a resilient and effective process despite its initial appearance of inefficiency.
The relevance of this intricate neural development process extends beyond the realms of biology. VCs take note (myself included!) that understanding the complexities of neural development offers a valuable framework for contemplating innovation and adaptability within the world of investing. The resilient and dynamic nature of synaptic development provides insights that can be applied to strategic decision-making in various industries, underscoring the importance of embracing diverse alternatives before honing in on the most advantageous ones.
The process of neural overproduction followed by pruning closely aligns with the dynamics observed in the emergence of new industries, particularly in the case of AI. An appreciation of this parallel offers VC’s three notable advantages. Firstly, it serves as a theoretically sound model of innovation backed by empirical testing by researchers (imagine having some data!). Secondly, it provides VCs with a foundational understanding of market manias or bubbles (tailored toward public investing but also relevant in private market investing). Lastly, it underscores that the innovation process frequently unveils new areas to invest in.
James Utterback, in his insightful work "Mastering the Dynamics of Innovation," delineates three phases in industry innovation. The initial phase termed the fluid phase, represents a period characterized by extensive experimentation, mirroring the surge in synaptic connections. Subsequently, the transitional phase ensues, wherein evolutionary forces dictate the selection of a dominant product design—akin to the pruning process observed in neural development. The final phase, the specific phase, witnesses more modest changes in product or process, a stage commonly encountered by adults. These phases collectively suggest a recurring pattern: a sharp increase in the number of companies during the early stages of an industry, followed by a pronounced decline as the pruning process takes effect–sounds and feels familiar to the current consensus of the moment.
While this process may initially seem wasteful due to the dismissal of numerous alternatives, the interplay between technical capabilities, market choices, and buyers' ultimate needs guides the selection of a product best suited to the prevailing environment. This pattern, reminiscent of synaptic development, has consistently manifested in what I believe to be the venture world, highlighting the cyclical and selective nature of innovation within industry segments and generational technology shifts (e.g., cloud, mobile).
Consider two historical titans within the American industry—the automobile and television. In both instances, Investors generously allocated capital during the early phases, given the significant yet uncertain growth potential of each industry. However, as these industries matured, there was a noticeable contraction in the number of competing companies, particularly once a dominant product and, therefore, company emerged.
The more relatively recent histories of the disk drive and personal computer industries echo this pattern, albeit with a pruning process occurring over a more condensed timeframe. What transpired over three decades in the automotive industry a century ago unfolded in approximately fifteen years for the disk drive sector and closer to a decade for PC manufacturers.
Of course, this wouldn’t be an analogous comparison of AI without having to mention the bubble/mania of the dot-com era – The Internet underwent a parallel evolution at the turn of the twenty-first century. While the Internet isn't a distinct industry, a period of vigorous experimentation marked the late 1990s. The subsequent pruning process manifested robustly in 2001, with 544 Internet companies closing, up from 223 in 2000. By the first half of 2002, shutdowns decreased by almost 75 percent from the previous year's total. Although the exuberance of the Internet and telecom booms in the late 1990s persisted into the twenty-first century, this boom-and-bust pattern is far from an isolated occurrence. Anticipating its recurrence in the future is a reasonable expectation and may be tightly coupled with the funding boom centered around AI by today’s standards.
As we transition from infancy to adulthood, there is a trade-off: we exchange vast mental flexibility for capabilities finely tuned to our environment. Skills and competence improve even as the number of synaptic connections declines. A comparable phenomenon unfolds concerning mature markets and investing.
As an industry grows, the number of competitors simultaneously decreases, driven by the selection of a dominant product, process, or both. Critics often disparage the boom-and-bust phenomenon, describing it as wasteful and speculative despite its role as a crucial catalyst for future growth.
Now that we have delved into the intricacies let's revisit the benefits for VCs and Investors. The first benefit involves cultivating a fundamental appreciation for the pervasive nature of the boom-and-bust pattern. In essence, when faced with an uncertain environment, it proves advantageous to initiate with a plethora of alternatives (akin to synaptic connections) and then strategically select the most fitting ones through a process of pruning (consistently investing over a long duration). Despite the evident costs in terms of energy and resources, this method stands out as the most effective.
The second benefit lies in understanding how this process contributes to market manias. Envisioning a baby's brain as a market, where each synapse represents a company or entrant, reveals the initial buzz and enthusiasm as the brain generates numerous synapses, anticipating some to be wildly successful. The introduction of valuations forms the foundation for a market mania, with VCs utilizing valuation as a crucial cue for a company's success. This triggers a positive feedback loop, propelling the mania forward. However, as observed, not all synapses or companies endure.
Failures and waste inevitably mark the journey to innovation. Participants (defined as anyone building or investing) in the market must acknowledge that, in periods of innovation, the business environment is inherently fluid, making it challenging to predict which businesses will ultimately thrive or falter. Yet, for the survivors, the potential payoff can be substantial, leading to the final point.
While markets, start-ups, and VCs are social constructs, they exhibit features with solid parallels in nature, exemplified by the intriguing similarity between brain development and industry innovation. As it relates to AI, it’s worth pondering if we’ve passed the state of mental and brain plasticity and are now entering the era of pruning, where most of the waste will be laid bare, and the survivors will continue to endure. Concerning investors, it’s crucially important to remember how most manias end yet how their impacts are rarely forgotten.
