True Complexity and False Simplicity

By Jason Crabtree

Reprinted from the National Academy of Engineering 2020 Winter Issue of The Bridge on Complex Unifiable Systems

Today’s digital societies are more connected and interconnected than ever before. Nevertheless, subtle and substantive long-term changes have been afoot. Current thinking about risk management across organizations and practices has not kept pace with this emerging reality, in part because the changes are at the same time very fast and very slow.

The digitization of everything in our workplaces and our personal lives is accelerating with each successive shock to the economy. If the primary purpose of risk management is to improve outcomes for stake-holders, then the participants in risk-related functions need to account—and be accountable—for the structural, behavioral, and physical realities of pressing societal challenges on several spatial and temporal scales. This includes dealing with often ignored systems concepts such as ergodicity—a too often applied hypothesis that allows for replacing dynamical models with probabilistic ones in certain constrained cases—noting that in many nonequilibrium human systems of interest and importance, when does not equal if.

Fast vs. Slow Changes

Fast changes are easiest to see, comprehend, and explain. Covid-19’s threat to human health has rapidly driven hundreds of millions of people to the daily use of videoconferencing technologies at work and at home. The security implications of this newfound dependence on key providers of these services are profound.

Slow changes are more difficult to understand, both for casual observers and for the people charged with quantifying emergent risks. Simple cases of technology outages in today’s consumer-facing applications and services abound. More complicated cases, however, like the Travelex ransomware incident that disrupted numerous downstream banking services, lurk just beyond the public consciousness. Deeper trends, as exemplified by the 2003 essay “CyberInsecurity: The Cost of Monopoly” (Geer et al. 2003) on the security implications of modern computer operating systems, dominate the professional conversation below the popular discussions about nation-state threats and cyber norms.

Neither globalization nor corporate consolidation has by itself driven this migration of risk from individual entities into more systemically important institutions. Complexity and the falling cost of information technology have driven a rapid yet inefficient interest in the proliferation of software. To say that “software is eating the world” is a naïve formulation of a slow but emergent truth: universal computing has made complexity more economical than simplicity.

Complexities of Digital Dependence

Society’s dependence on all things digital is irreversible and inestimable. Since both digital dependence and interdependence continue to grow, predicting the exact effects of specific changes to the digital world is impossible.

It is possible, however, to approximate scenarios useful to both professional and public discourse about design choices and incentives. Exposure to systemic risk and the ongoing economic risk migration demands it. Even simple things like a dependence on managed IT providers can lead to a severe accumulation of risk by creating a limited number of unintentionally important counterparties whose successful operation is suddenly crucial to providing critical services ranging from education to health care.

The widespread use of complex but highly standardized and mass-produced contraptions with flexible and rapidly improving CPUs and favorable production economics at scale makes it possible to simulate simpler machines. The flexibility of modern operating systems and programming languages at their core has exacerbated the recent phenomenon of numerous machines whose functionality widely surpasses the tasks to which they are applied. Threats to digital society now have the advantage of low-cost complexity and can exploit the false simplicity that is so often foolishly implemented in systems.

Hacking is often merely the exploitation of system complexity, triggering an outcome not considered by an engineer. It is a repurposing of a system’s design, much as a parasite or disease organism repurposes an existing biological function. As yet, however, there is no digital immune system that will dynamically deploy a hierarchical and temporal set of response mechanisms as does a human immune system. Each integrated digital value chain will respond differently to the repurposing, based in no small part on how well designers have constrained the potential uses of software that can run on the flexible universal computers embedded throughout the connected world.

This minimalist perspective of security as the effective constraint of generalized capability will require hierarchical cooperation across the integrated value chain in the economy. It will be a dance between enabling and constraining action in diverse socio-technical contexts in which different actors will benefit disproportionately from each successive choice.

Security is functionally best thought of as a subset of reliability, that is, fitness for purpose as employed by users, not only as intended by engineers. Software is showing the effects of a decoupling between cost and consequence. Mass ransomware, major data breaches, and widespread information technology outages are all part of the emerging asymmetry between defenders of civil society and entities that seek to harm, disrupt, or coerce others.

Growing exposure to the transitive risks associated with digital interdependence demands the disclosure of breaches and sharing of metrics, and will require the difficult work of ontology specification within and across specialized knowledge domains. Encouraging accountability and economically rational actions in a complex multiagent decision-making environment requires semantically consistent approaches. This is not limited to cybersecurity. It is equally applicable to understanding the propagation of demand shock due to covid-19 or to crisis modeling for financial events like the 2008 market collapse.

Conclusion

To thrive in an increasingly volatile environment, and perhaps even to survive in it, more dynamic, continuous, forward-looking simulation-based exploration of possible future events is needed. Society cannot afford to be limited to historical experience or extrapolative prediction. Generative modeling, parametric studies, and the ongoing curation of previously considered scenarios must be sought out and, indeed, enabled. Scenario-based narratives from specific analyses must be communicated to individuals and organizations using familiar terminology.

The best approach to this solution remains the celebration of the struggle for clarity on important issues, especially in an era of disinformation coupled with shifting social contracts between people, organizations, and governments. Poor thinking and turgid dialogue will be our collective undoing. Reasoned argument and the formal cataloguing of knowledge offer a glimpse into a more hopeful, collaborative, and unifiable future.

Reference

Geer D, Pfleeger CP, Schneier B, Quarterman JS, Metzger P, Bace R, Gutmann P. 2003. CyberInsecurity: The cost of monopoly. Computer & Communications Industry -Association Report, Sep 24.