Part Two: Innovation as a Complex System

For many reasons, innovation strategy is poorly understood and even more poorly executed.

This series of posts will discuss some of my thinking on innovation and creativity, and as such outlines a concept I have developed: the Combinatorial Theory of Innovation from first principles. Others may have written something like this in the past, and if so, super. My concepts draw on methodologies and frameworks in high-dimensional combinatorial spaces that I both created and used at places like NASA and for the DoD for complex systems. For me, these could (and should) be applied everywhere, but especially to how we think about innovation.

I would also like to acknowledge that there are a handful of superb scholars in the field of innovation theory, and I hope there is some overlap in our thinking. There are unfortunately also some less superb innovation scholars roaming the halls of academia and policy and I hope that by outlining my theory of combinatorial innovation, I can more adequately explain why some of their policies are bad, and maybe even at times immoral.

[NB: This set of posts in particular comes from a series of ongoing discussions between myself and Turlough Downes as we create an innovation policy working paper using combinatorial and chaos theory, focusing on Ireland's innovation ecosystem. Thanks Turlough!]

Art by Emma Kunz (inspiration taken from Pierre Levy's recent post on semantics)

This is a short post which outlines innovation as a complex process, and the innovation ecosystem as a complex system. This may be obvious to lots of people, but bear with me.

To create sound economic and social policy around innovation strategy, it is clear that the two core challenges of complexity and feedback loops identified in part 1 must be addressed. Those driving innovation policies need to be provided with maps and a clear sense of direction in which to move. This paper aims to reduce the complexity and to better understand the feedback mechanisms of innovation strategy, such that every policy maker can engage with industrial strategy, not just the bravest.

Towards Complexity Theory

There is a lack of understanding of the primary mechanisms of innovation strategy, for reasons already outlined in part 1. That innovation strategy is usually defined in terms of its adjacent economic areas is driven primarily by the field of economics itself overwhelmingly being defined in terms of achieving equilibrium: supply and demand, trade policies, fiscal policy. Each of the adjacent fields of industrial strategy can be, and are, governed by the neoclassical economic assumption that the system will tend towards a state of equilibrium. On the other hand, innovation cannot be an equilibrium-driven phenomena and therefore is nearly impossible to model using the same economic techniques.

Innovation is the outcome of interconnected processes of trial-and-error, learning, and knowledge creation and transmission in ways that are unpredictable, always changing and hyper-sensitive to ever changing environmental conditions.

For these reasons, innovation strategy cannot be modeled and understood through traditional economics as it is not reliant on an equilibrium-driven set of occurrences. In contrast to equilibrium-driven events, innovation is non-deterministic: there is no known end-goal, since being able to ‘predict’ future progress between chaos-driven interactions within the ecosystem is impossible. [This characteristic is going to be very important when I later evaluate existing innovation policy.]

Innovation is a domain which, unlike allocation problems such as trade and pricing mechanisms, faces fundamental uncertainty: the problem is not well-defined, therefore rationality and ‘optimal’ behavior is not well-defined, and as such the probably that the behavior of innovation system is in equilibrium is infinitesimally small.

Innovation is a dynamic system comprised of constituent components which interact with each other in highly non-linear ways, and whose interactions create often unpredictable outcomes. Consider the number of outcomes produced by a single entrepreneur alone, as she moves through the phases of ideation, collaboration, implementation and value creation. Additionally, the overall innovation system behavior outcome cannot be derived by understanding the behavior produced by each individual entrepreneur, investor or institution: the system driven by feedback loops which are built on interconnected interactions, shaping future outcomes.

In short, innovation and as such innovation strategy needs to be understood and modeled not by traditional economic means but instead by treating the ecosystem as a complex, adaptive system and employing new ways to understand the system behavior, as found within the field of complexity and chaos theory.

Further posts will introduce the properties of complex, adaptive systems, model the innovation ecosystem as a complex system and from modeling the behavior of the complex system, derive the concepts of system-wide industrial strategy from first principles.

On Complexity: Modeling Innovation as a Complex System

Complex systems are more than just complicated systems. There is no one-to-one mapping between complex and complicated problems; whilst complex systems are nearly always complicated, complicated systems are not always complex. For example, designing and building a SpaceX (or any) rocket is complicated- it requires understanding of thermodynamics, propulsion, aerodynamics and so forth. But with that knowledge, the interaction between the system components and overall system behavior is known. This is why SpaceX are able to perform the same mission, many times over with each rocket, with repeatability. On the other hand, designing and building a city not only complicated- the designer must understand zoning, urban design, architecture, civil engineering- but also complex; the interaction between the components such as traffic, mobility, planning, architecture, and human behavior leads to unpredictable outcomes that leaves the city in constant flux. There can be no repeatability within complex systems- infinitesimally small changes in the system will lead to drastically different outcomes.

A complex system is one which has many components, whose interactions drive the system’s behavior, not the components themselves. Complex systems have input from numerous sources and its overall system behavior is highly dependent on even microscopic changes to these inputs (such as adding a set of traffic lights). Popular examples of complex systems include the weather, the nervous system, cells and living things, telecommunication infrastructures and so on.

There are four main attributes of a complex system:

1. Emergence

The behavior of complex systems is ‘emergent’ in that the behavior of individual components of the system do not explain the overall behavior at the system level. For example, we cannot understand the behavior of a city by looking only at the behavior of an individual taxi or human. This higher-order, spontaneous behavior cannot be derived by aggregating the behavior of all of the system components. Said differently, the whole is more than the sum of its parts, which leads to emergent behaviors.

2. Non-linearity

Often referred to as ‘tipping points’ or ‘phase transitions’, complex systems show inherently non-linear behaviors. That is, doubling the input to a system does not double the output. For example, most people do not get the same enjoyment watching a great movie the second time immediately after watching it for the first time. The complex system may suddenly change behaviors or become unstable with microscopic changes to input that are difficult to predict or model. This can be seen with Covid-19 crisis in hospitals, where admitting 100 patients to hospital could allow the hospital to operate under optimal conditions, but admitting 105 patients may cause the hospital to collapse.

3. Non-predictability

The behavior of complex systems cannot be predicted, and this phenomenon is has two major consequences. Firstly, Given a state of a complex system today, it is impossible to determine what the initial conditions were of that system at an earlier point in time. Secondly, given the conditions of the system today it is impossible to accurately predict its future behavior. Take the example of the weather: if you look out of the window right now and take note of what the weather is doing, you will know nothing about what it did a week ago or what it will do in a week’s time based purely on what it is doing right now. Non-predictability is a second-order impact of emergent behavior of complex systems.

4. Self-organization

Complex adaptive systems initialize, adapt and grow without central control. These systems spontaneously organize themselves so as to better cope with various internal and external perturbations and conflicts, allowing them to evolve and adapt to a constantly changing environment. This self-organization occurs from local interactions eventually producing global coordination and synergy that are driven from the bottom-up. Such complex, self-organized “networks of interactions” typically exhibit the properties of clustering, being scale-free, and forming a small ecosystem. This can be seen with the emergence of ant colonies through the bottom-up interaction of individual ants leading to colony behaviors that naturally evolve over time.

The innovation ecosystem is a complex system that exhibits the same attributes and characteristics of the complex adaptive systems already discussed. If we consider that there are two great problems in economics: firstly, of allocation in the economy and secondly of formation in the economy, the first problem is clearly mathematizable, whereas the second is not. Innovation strategy is a problem belonging to the second set of economic problems, formation in the economy, and using a complex systems approach to solving these problems is key.

Next time I'm going to outline the constituent parts of the complex system that is the innovation ecosystem, and explain them relative to combinatorial theory.....