The Effects of Network Structure and Dynamics in Complex Systems

Your Name:
- How you would modify the problem:
- Your personal goals:
- The skills you would bring:

I don't have a lot of time to participate given that I've bitten off quite a chunk on another project. However, since I was asked for my thoughts, I have posted my thinking below...

Your characterization of the challenge that complex environments present to human organizations is an interesting one. I want to take a shot at reframing the problem in a somewhat novel characterization. To do it however, will require a short (for me, but long, for you) journey through some foundational concepts. Many readers may be familiar with some of this, but rather than make any assumptions about I’ll try and start from fist principles.

Your initial characterization of the situation and problem must be parsed carefully. Specifically, think carefully about differentiating between the environment and the company. These are two highly connected entities, and while they should be modeled and analyzed together as interacting and influencing one another, they have different fundamental goals. The challenge of the environment is one of proper characterization and dynamics – we need a good description of what it is and how it behaves, or more specifically, what selection criteria exist at the environmental level that differentiates successful organizations from unsuccessful ones. Of course, the environment itself is a bit of an ecological fallacy, it neither decides nor selects anything. It is really shorthand for the aggregated decisions and actions of agents outside of the organization we are modeling. Thus, from an agent-based perspective we have no environment per se, just a population of other organizations, investors, stakeholders, interest groups, suppliers, etc. Regardless, the crucial question for the situational challenge is an empirical one – “what is it, and how does it behave?”

Alternatively, understanding the organization’s options within the environment is a more complex question. Are you asking, “what can the organization do?” or “what should the organization do?” These subtle distinctions are important and will significantly affect the models and methods you consider. The first question, what can it do, is again an empirical question, a simple matter of considering all the possibilities of the entire menu of choices and getting a sense of their outcomes. From this perspective, one option is simply to disband and cash out. Implicit in your question, however, is really the matter of what should it do – i.e. a normative judgment about the opportunities facing the organization that preserve or enhance some set of values that its members, leadership, or stakeholders hold dear. This question constrains the search space, but complicates the analysis by forcing decision-makers to better understand the trade-off space, much of which will be composed of hidden (sometimes catastrophic, and sometimes serendipitous) relationships.

To provide you with a sense of what this is about consider the classical problem of “goal displacement.” Goal displacement occurs when one level within an organization, usually a lower level, alters its behavior to meet the goals of the smaller group, subverting the mission of the larger organization. For example, the YMCA started as a organization that intended to teach basic industrial employment skills to immigrants and people moving to the cities from rural America. However, because the market was not conducive to supporting an organization of that size with those goals, local offices and administrators shifted their services towards supporting suburban youth recreation. This shift in mission and structure allowed the organization to survive, i.e. everyone still had jobs, but its mission and purpose shifted in a fundamental fashion. Now, the question that must really be answered is the goal merely survival, or is it to retain a particular set of services and focus. If the question was what could the YMCA do to ensure continued existence as an organization, it succeeded. If the question was what could the YMCA do to ensure the continuation of its core mission, it failed. The success case is empirical – no value judgments are present in the analysis – the YMCA existed at time tn or it did not. The failure case is normative, it was unable to retain values that satisfied the goals of its founders.

I assume that you are really asking a normative question about organizational adaptation and survival – i.e. how can organization with a given mission stay in business while operating under uncertainty, rather than the simpler question of can it survive.

This question of empirical vs. normative science is an important one. Let me continue with it for a couple more paragraphs. If one examines the organization of the modern university, this question is never really posed. For example, if you were to start a new university, would you put the Engineering Department in the School of Sciences or in the School of Law? Overwhelming, we place engineers with scientists and social scientists with lawyers and policy analysts. This is because we are fooled by their tools, and lose sight of their dominant focus of inquiry. Social and physical scientists both seek to understand and explain the world’s natural state. They seek to formulate, characterize, or otherwise develop bodies of knowledge that explain some phenomenon – whether it is the origins of the universe, biological speciation, or the functioning of economic markets.

By contrast, engineers, lawyers, and policy analysts seek to exploit knowledge, guile, heuristics, and anything else that “works” in order to build things that accomplish some goal or set of goals. Whether building a car, designing a rocket, or drafting a constitution, each is concerning with the accomplishing of some normative value or objective. Thus, the persistent tension between “academic theorists” and “real-world practitioners.” (It is important to note that the “ideal” world never need come up to create this conflict, i.e. the ivory tower is not about studying the world in the abstract, but rather studying from a value free standpoint.)

I’ve harped on the distinction between empirical and normative science for a good reason – it has profound implications over the selection of tools and inspiration when confronting a complex problem. Remember, the natural world has no care whatsoever who lives or dies – what species land at what position in the food-chain (food-web now). Likewise, the global economy has no care if General Motors, IBM, Microsoft, Google exist or collapse. In both cases, these systems behave in some equilibrium seeking fashion (if the natural scientists are to be believed) that is simply following the gradient without regard or concern for the micro-level effects of the journey. Recall Liz saying that they “throw out the transient,” but from a social standpoint, if the attractor is the macroscopic distribution of wealth, the transient is whether you or I can pay our mortgage or retire. I don’t only care about the macroscopic distribution of wealth; I care about where I sit in that distribution. So, a great empirical solution (the economy has reached equilibrium) may instill tremendous instability because people are not happy with their lot in life – our values and goals may conflict the “natural” tendencies of the system – which of course complicates our analysis – natural process and human preferences conflict. If you want to really understand this, take the Sugarscape model (or the Tomonomics model) and find agent 7. Run it several times and see if you get the same result. The “scientist” might decide that each run produced a regularity in the structure of the outcomes, i.e. the distributions of wealth were similar and the time to arrive at them was consistent from experiment to experiment. Now, if you were agent 7 would think the results were the same each time?

Having beaten the problem of empirical vs. normative to death I think we need to revisit questions about what works and doesn’t work in complex systems or environments. We need to capitalize and exploit what we know about complex systems, but we also need to be mindful that any potential solution must preserve or enhance core organizational values – and these are not easily resolved.

What troubles me is that much or the organizational literature that deals with complexity makes one of three mistakes:

1) They fail to fully contextualize the environment;

2) They examine simple or trivial problems;

3) They analogize social problems to physical or biological ones.

The failure to contextualize what successful organizations do means that decision-makers never have a real understanding of what truly is successful or why. Read any case-study in military or business innovation and they provide wonderful insights of works based on generalizing off a sample size of one, of just slightly larger. The problem is that everyone in the system could have been trying the same thing, or a variant, but only a few survived. So, by only studying the winners we never get a full understanding of what was really happening within the system – how many firms were competing, what were they doing, etc. Did the winners win because they were really good, just lucky, or was their competition inept? Recall that during the 1980s the Israeli military was considered the best in the world based on its performance against conventional Arab armies. Then, the U.S. defeated the Iraqi military after 1,000 hours of ground combat. Afterwards, many began to wonder if the Israeli military was that good, or were the Arabs that bad? In one sense it doesn’t matter, but who wants to depend on the weaknesses of one’s adversary?

The challenge of trivial problems is another analytic pitfall. In my own domain, military and strategic analysis, several studies have been published on the benefits of new organizational structures, operational concepts, and weapon/sensor/communications technologies. In many cases, these innovations are shown in some analysis or field exercise to outperform old ways of doing things. The problem, however, is that much if these studies are composed of simple problems that never challenge values. For example, studies proving the effectiveness of automated targeted and strike systems against mobile targets are abundant, but in almost call cases they are engaging targets at sea, in the open desert, or some other socially and politically simple terrain. Do the weapons and processes work as well when the targets are individuals in an urban environment? Routinely, challenge problems are technical in nature – focusing on systems integration and communications, but never present scenarios that are contextually challenging, such as including the presence of friendly forces, civilians, or limited political goals. I suspect that the business literature is filled with equally flawed analysis.

Finally, physical, biological, and social systems overwhelm us with the successes of decentralization, local empowerment, and the aggregation of uncoordinated or self-organized entities. Indeed, many of these examples are fascinating and inspiring. However, they are also misleading and potentially dangerous. We know that systems will self-organize when put under sufficient stress, but not all solutions are ones that we desire. Consider the case of increasing returns and multiple equlibria in economics. We know that economic systems can experience increasing returns to scale – those cases where having what everyone else has, increases the value of having what everyone else has – such as gasoline combustion engines, QWERTY keyboards, Windows operating systems (until the Internet allowed from platform independent exchanges of information), etc. However, what is not well understood is how to ensure that the equlibria that is achieved via increasing returns is the most socially beneficial one (the fact that multiple equlibria exist means that invisible hand is incapable of making a value judgment because if one equlibria was empirically better than the others they would cease to be equlibria). I believe that we should embrace the concepts and lessons of complexity, but always return to the core challenge of ensuring that our organizations and institutions, no matter what their final form, retain their fundamental purpose as vehicles for achieving individually or collectively desired goals.

I think would return to the original notion of the organization and complexity. It is useful to recall that human beings have limited memory and computational power, which is why we need to act collectively in the first place. Indeed, it is precisely because we have inherent inabilities to deal with complexity that we have constructed organizations in the first place. Whether transitioning from small egalitarian nomadic groups, to settled collective agricultural farms, to highly diversified urban states social complexity, or transitioned from small family farms to multi-national corporations structures have co-evolved to meet the challenges posed by the environment, internal dissent, or external threats. If I were focusing on the social aspects of complex organizations and interactions I would start with researching the interactions between complexity, uncertainty, and hierarchy. They are the enablers of one another, so I believe the trick is understand what critical skills and processes need to exist within an organization, and then build a hierarchy around them to ensure they are properly nurtured and maintained.

What surprises me, more often than not, is not that organizations lack the necessary skills or good people, but that complex scenarios and environments often invert organizational priorities. As a result, what appears to be a need for decentralization is often a misdiagnosis. For example, the U.S. military has not fared well against the Iraqi insurgency. While many have argued for the continued need for decentralization, the track record of decentralized decision-making at low levels of command certainly should serve as cautionary tale. Instead, a fundamental shift in the proportion of services and capabilities would likely have sufficed, turning the support functions of civil affairs (engineering, policing, civil administration, public affairs, etc.) and intelligence into the dominant capabilities, while reducing the role and prominence of the combat forces to a supporting capability. I suspect that many of the organizational challenge problems facing organizations could be handled this way, but organizational cultures may compel leaders and managers who sit atop important mission areas to avoid searching for solutions that make their subordinate or support functions their new masters.

Some recommended readings:

Talk

Talk:The Effects of Network Structure and Dynamics in Complex Systems

From Santa Fe Institute Events Wiki