Categorization by Emergent Networks: The Distributed Sensemaking Simulation Model. Jarrett Spiro (INSEAD)* Joe Porac (NYU) Hayagreeva Rao (Stanford)

Transcription

1 Categorization by Emergent Networks: The Distributed Sensemaking Simulation Model Jarrett Spiro (INSEAD)* Joe Porac (NYU) Hayagreeva Rao (Stanford) In Preparation for Science September 28, 2012 Working Paper: Please do not cite or distribute without permission. Please contact Jarrett Spiro at for permission or if you have any questions concerning the draft.

2 Abstract Understanding complex situations, such as crises scenarios, often requires groups of actors to exchange and then interpret information, a process called distributed sensemaking. We created the Distributed Sensemaking Simulation Model to examine how the mechanisms by which actors interact determine three characteristics of a crisis scenario. First, the structure of the information-sharing network that emerges. Second, whether the distributed sensemaking event ends in a situation in which the actors agree on a categorization that would not have changed if they had gathered more information, i.e., high fidelity agreement. Third, how much time the actors needed to reach high fidelity agreement. The simulation model utilizes three basic behavior factors: how active actors are in reaching out to other actors, how much actors reciprocate ties, and how conservative the actors are when they categorize the information, i.e., the similarity threshold. The simulation model indicates that the level of outreach, reciprocity, similarity threshold, and network structural characteristics, especially network modularity, impact whether an event ends in high fidelity agreement and the amount of time necessary to reach high fidelity agreement. 2

3 Crises are characterized by low probability/high consequence events that threaten the most fundamental goals of an organization [1]. For crises such as terrorist attacks, disease outbreaks and natural disasters, the most fundamental goal of the organizations and individuals involved is ensuring the safety and security of others. In order to protect people, the relevant organizations and individuals must make sense of the situation quickly and accurately, or lives could be lost. Typically, no single organization or individual has the knowledge or resources i.e., cues needed to appropriately categorize the crisis scenario; the cues are distributed amongst a number of organizations and individuals [2], and in order to understand the situation, the cues need to be exchanged and interpreted [3-6]. Such distributed sensemaking can be viewed as a process of cue accumulation through the assembly of a network of actors, each with relevant, but only partial, information who collectively resolve a categorical dilemma and make sense of the crisis scenario. Consistent with Taylor and Van Every [7], distributed sensemaking is how groups composed of individuals with distributed segmented, partial images of a complex environment can, through interaction, synthetically construct a representation of it that works; one which, in its interactive complexity, outstrips the capacity of any single individual in the network to represent and discriminate (pg. 207). Crises scenarios often require distributed sensemaking [2, 4, 6, 8-11], but in many instances, the necessary cues were not exchanged, leading to disastrous results. A general lack of information exchange amongst government agencies prevented the government from stopping the 9/11 terrorist attacks [4, 6] and slowed the government s response to Hurricane Katrina [8]. NASA s Debris Assessment Team (DAT) was unable to directly interact with the Department of Defense to get permission to get pictures of the underside of the Columbia Space Shuttle, which could have provided the information necessary for the shuttle to be saved [2]. During the initial 3

4 outbreak of the West Nile virus in New York City in 1999, the CDC resisted interacting with the Bronx Zoo, which meant they lacked critical information about the disease and ended up misdiagnosing it as St. Louis Encephalitis [10]. And even though local and international agencies were able to quickly and effectively diagnose the Marburg Hemorrhagic Fever in Angola in 2005, the lack of information exchanged between those agencies and the general public actually produced behavior that increased the spread of the disease [11]. In fact, one of the most prevalent suggestions for reducing the severity of crises scenarios is increased information exchange [3-6]. Unfortunately, there has yet to be a systematic and scientific model of sensemaking in a crisis scenario, and so it is difficult to determine how even minor changes in the frequency of information exchange may or may not impact how a crisis scenario is resolved. One of our main goals in this paper is to provide such a model. Following Comfort [4], the components of how actors interact in a crisis situation comprise a complex system with recurring patterns of information search, exchange, and adaptation (pg. 101). Such systems are known as complex adaptive systems, because they emerge from numerous components interacting in complex and chaotic ways, which obscure the relationship between the components and the final outcome [4, 12]. The preferred methodology for studying complex adaptive systems are agent-based simulations, which are used to create theoretical models with empirically testable propositions for complex adaptive systems. These models link basic behaviors to complex outcomes through the analysis of network structures that emerge as actors interact [12]. In this paper, we utilize agent-based simulations to construct the Distributed Sensemaking Simulation Model, in order to provide a systematic way of demonstrating how various characteristics related to information exchange, including basic behavioral factors and the network structure that emerges as actors interact, impact the final 4

5 categorization of the sensemaking event and the number of periods necessary to reach that categorization. Simulation Mechanics The underlying mechanics of the simulation model will be based on three factors: outreach factor (o), reciprocity factor (r), and similarity threshold (t). The first factor, the outreach factor, dictates the likelihood with which actor A reaches out to another actor, B, in a specific period in the model. When actor A forms a connection with actor B, actor A shares his or her information with actor B. The likelihood with which actor B reciprocates and shares his or her information with actor A during some future period is dictated by the second factor, the reciprocity factor. The final factor, the similarity threshold, influences how the actors interpret the cues they possess. As actors accumulate cues, they must interpret those cues through a process of categorization. Cognitive categorization has been well studied by cognitive scientists, and a number of models exist that specify relationships between cues and categories [e.g., 13]. Our simulation makes minimal assumptions about these relationships and simply assumes that cues refer to category attributes, and that cue accumulation provides evidence about attributes that implicate one or more categories. Therefore, an actor categorizes merely by measuring the amount of overlap between the cues the actor possesses and the corresponding attributes associated with different categories. As in most settings, categories are not mutually exclusive. Categories are fuzzy sets with overlapping cue combinations [13]. The same sets of cues may implicate more than one category and so actors must rely on establishing a cutoff point that is good enough. In this model, the cutoff point is specified by the similarity threshold: the level 5

6 at which the amount of overlap between an actor s cues and a category s attributes are high enough for a category match to be defined. Therefore, the similarity threshold captures whether an actor chooses to be conservative or not in their interpretation of the event. Table 1 lists all of the parameters that comprise the simulation model and the values the parameters take in the specific simulation trials utilized in this paper. For more information, see Further Description of Simulation Mechanics. [Insert Table 1 Here] The basic mechanics of the simulation are illustrated in figure 1. Initially, each active actor in the simulation, i, attempts to interpret the situation by comparing their cue sets to the category space. If there is one match between the cue set and the category space that is both above the similarity threshold, t, and greater than the match between the cue set and any other category in the category space, then actor i selects that category. If this is not the case and no category is selected, then with probability o actor i forms a new connection this period and with probability 1-o actor i does not form a new connection this period. If actor i forms a new connection during this period, then actor i connects to an actor from her reciprocity set, actor j, with probability r. Actor i then shares all of her cues with actor j so that actor i s cue set is added to actor j s cue set. With probability 1-r, or if actor i s reciprocity set is empty, actor i forms a connection to an actor from her connect set, actor k, and actor i s cue set is added to actor k s cue set. Once the cue sets have been updated with new cues, the process repeats itself for another period. The simulation starts with only a single actor and more actors are added as actors active in the simulation connect to actors who are not yet active in the simulation. [Insert Figure 1 Here] 6

7 The simulation continues in this way until one of three stopping rules is satisfied: 75% of the actors agree on the same category, or less than 75% of the actors agree on a category but there has been no change in the agreed upon categories for at least 10 periods and at least 50 periods have already passed, or if neither of the other rules has taken effect by the 200 th period. We defined three possible outcomes of the simulation model: no agreement, high fidelity agreement, and low fidelity agreement. The actors can either converge on a category or not. Depending on the number of cues actors have accumulated and how the cues are distributed among the actors, the converged category can represent either high fidelity or low fidelity agreement. Within the simulation, a cue-seeding vector, a vector that includes every cue that can possibly be collected and exchanged by an actor, populates the actors cue sets (each actor starts with a subset of cues from the cue-seeding vector). The category from the category space that has the greatest overlap with the cue-seeding vector is the high fidelity category, since after the actors converge on that category they would not change their categorization even if they collected more cues. Low fidelity convergence occurs when actors agreed on a category but would change this category if more cues had been collected [14]. It is important for the actors to agree on a category because coordinated action is often necessary to effectively respond to a crisis scenario [4, 8, 15] and if the actors do not agree about what is happening during a crisis, coordinated action is not possible [16]. In the simulation, we use 5 possible values for the outreach (o) and reciprocity factors (r), 0.1, 0.3, 0.5, 0.7 and 0.9, and 4 possible values for the similarity threshold (t), 0.1, 0.3, 0.5 and 0.7 [17]. This meant that we needed to run the simulation 100 times in order to get a complete set of all possible combinations of the values of outreach factor, reciprocity factor and similarity threshold (5 x 5 x 4 = 100). We ran 145 trials for each of the 100 possible combinations of the 7

8 values of the outreach factor, reciprocity factor and similarity threshold, for a total of 14,500 runs. Of these 14,500 runs, we eliminated 6,032 runs as not being relevant for this paper. Some trials were dropped from our analysis because more than one category could constitute high fidelity agreement i.e., more than one category in the category space share the greatest amount of overlap with the cue-seeding vector and our focus is on cases where only a single category is considered a high fidelity categorization. Others were dropped because no more than one node became active during the trial, i.e., no network emerged. Of the remaining 8,468 trials, 6,376 trials resulted in high fidelity agreement, 666 trials resulted in low fidelity agreement, and the remaining 1,426 trials resulted in no agreement. We focus on two dependent variables: a) the likelihood of reaching high fidelity agreement as compared to low fidelity agreement and no agreement, and b) the number of periods required to reach high fidelity agreement given that high fidelity agreement occurs. High fidelity agreement can be viewed as an equilibrium condition [15]. The number of periods to reach high fidelity agreement is also an important dependent variable given that a crisis situation, by definition, demands a timely response. Simulation Results Figure 2 shows how varying the levels of outreach factor, reciprocity factor and similarity threshold impact the likelihood of high fidelity agreement and the number of periods necessary to reach high fidelity agreement. As the outreach factor rises from 0.1 to 0.9, the likelihood of high fidelity agreement increases from 58% to 83% and the number of periods needed to reach high fidelity agreement drops from 70 periods to 22 periods. However, there are decreasing returns to increasing the outreach factor. When the outreach factor grows from 0.5 to 0.9, the 8

9 difference in the likelihood of high fidelity agreement is only 3% and the difference in the number of periods necessary to reach high fidelity agreement is only 14 periods. [Insert Figure 2 Here] Higher similarity thresholds increase the likelihood of high fidelity agreement, but as the similarity threshold grows beyond a certain point, the likelihood of high fidelity agreement decreases. Since a higher similarity threshold requires more cues before a category match occurs the process of collecting additional cues, makes a high fidelity category match more likely. If the similarity threshold is too high, however, enough cues to agree on any category cannot be collected within the period limits of the simulation. The relationship between the reciprocity factor and both the likelihood of high fidelity agreement and the amount of time to high fidelity agreement is curvilinear. Middle values of the reciprocity factor increase the likelihood of high fidelity agreement and decrease the number of periods needed to reach high fidelity agreement. Middle levels of reciprocity indicate that actors are alternating between reciprocating and connecting with someone new. Emergent Networks The outreach factor, reciprocity factor, and similarity threshold all influence the likelihood and frequency of interaction among actors. As the likelihood and frequency of interaction varies, different network structures emerge [18, 19]. Guimera et al. found that by simply varying the number of incumbents and repeat ties that are added to teams during the process of team assembly, the network that emerged could either be fragmented or not and had very different small world characteristics [20]. Varying the outreach factor, reciprocity factor and similarity threshold also has a significant impact on the network structure that emerges in a 9

10 distributed sensemaking event (see Figure 3 and Emergent Network Analysis for a more detailed discussion of how the network emerges in the simulation model). [Insert Figure 3 Here] Network modularity refers to the presence of subgroups within a network, i.e., whether a network is cliquish. A subgroup or module within a network is clearly defined if there are relatively few connections between the modules as compared to the number of connections within the modules. We use Guimera and Amaral s method for calculating the modularity of a network [22]. (See Modularity for a detailed description of this method.) [Insert Figure 4 Here] Figure 4 shows the relationship between the maximum modularity value achieved during the course of each simulation trial and the likelihood of high fidelity agreement and number of periods to high fidelity agreement. There is an inverse U-shaped relationship between modularity and the likelihood of high fidelity agreement. When modularity approaches zero, there is only a 66% chance of high fidelity agreement, but at a modularity score of 0.3 the odds rise to almost 80%, before dropping to 69% as the modularity reaches 0.6. The relationship between moderately modular network structures and optimal outcomes has been well documented. Uzzi and Spiro showed that when the Broadway musical industry had a moderate level of clustering, another name for modularity, there was an increased likelihood that the productions were both financial and critical successful [22]. Lazer and Friedman and Fang et al. both demonstrated that a network with a moderate level of clustering provides the optimal mix of explorative and exploitative behavior [23, 24]. Even artificial neural networks benefit from moderate levels of modularity. Happel and Murre discovered that neural networks are best at 10

11 categorizing data when the network is modular, and Melin extended this work by showing how type-2 fuzzy systems could improve the process of categorization even more by providing a better way to integrate the modules, producing a more moderately modular network structure [25, 26]. From these past studies, we know that certain desired outcomes are likely to occur at moderate levels of modularity, but this work has said very little about what kind of outcomes are likely when the modularity is too high or too low. One of the benefits of the simulation model in this paper is that we can provide insight into the different types of outcomes that occur at the extreme values of modularity. Low modularity scores lead to actors sharing information too quickly. Actors then only gather enough cues to reach a low fidelity agreement. Once the actors have agreed on a low fidelity category, they no longer actively search for more cues, and so cannot appropriately revise their interpretation. The resulting network structure consists of a single dominant module populated by a set of elite actors who have agreed upon the low fidelity categorization. When the modularity is high, the different modules essentially represent different, unique subgroups within the network. The network structure thus represents a market in which each subgroup makes sense of partial information. Network-wide agreement can only be achieved if the partial solutions are exchanged for one another. Without the necessary between module links, this is unlikely to happen. The actors only share their cues within their own subgroups and so each subgroup converges on a different category. High fidelity agreement is most likely to occur when the appropriate balance between an elite network structure and a market network structure is achieved. (See Likelihood of Low Fidelity and No Agreement for supporting analyses.) 11

12 The relationship between modularity and the number of periods to high fidelity agreement is linear. When modularity is close to zero, high fidelity agreement occurs on average within 16 periods. This increases to 60 periods as modularity increases to 0.6. Low modularity indicates that there are more connections between modules, i.e., the module boundaries are more permeable, allowing greater amounts of information to flow through the network at a faster rate. This accelerates the process through which information is shared and a consensus categorization is reached. This happens in cases that end in either high fidelity or low fidelity agreement. Therefore, if speed is more important than fidelity for a specific distributed sensemaking situation, then having a moderate level of modularity is not desirable. The cost of attaining higher likelihoods of high fidelity agreement is time. The simulation model is theoretical in nature and so to provide empirical support for it we compare the results of the model to two real world crisis scenario cases: the initial outbreak of Sin Nombre and West Nile viruses in the United States [10]. For an accurate diagnosis to be made both cases required the cooperation of local, state and federal authorities, as well as some relatively atypical organizations, like the Bronx Zoo in the West Nile case and Navajo medicine men in the Sin Nombre case. Despite the similarities in both cases, one case was regarded as a success, Sin Nombre, while the other case, West Nile, was considered a failure. In the Sin Nombre case, high fidelity agreement was achieved relatively quickly, 28 days, and on the first try. In the West Nile case, the disease was initially misdiagnosed as St. Louis Encephalitis and correcting the diagnosis and reaching high fidelity agreement took 39 days, even though it was a simpler disease to diagnose than Sin Nombre. Figure 5 shows how the modularity progressed in the Sin Nombre and West Nile virus cases, respectively, as well as the average modularity progressions of the simulation runs that 12

13 resulted in high fidelity agreement over the same amount of time/periods as the two cases (see Modularity Progression for a discussion of this methodology and other, related results). We limited the similarity threshold to 0.7, since the organizations in these cases are often especially conservative before settling on a diagnosis [10]. The general pattern of modularity progression between the simulation and the case appear to be very similar. In both instances the modularity increases rapidly at first and then levels off. The modularity in the Sin Nombre case is relatively close to the simulated modularity throughout the entire case. The modularity in the West Nile virus case is also somewhat similar to the simulated modularity, following the simulated modularity plus one standard deviation, before converging on the simulated modularity at the end of the case. Since a simulation model is a simplified version of reality, it is more likely that it will be able to predict the simpler real-world events that went smoothly, i.e., the Sin Nombre case, rather than the more complex real-world events that did not go smoothly, i.e., the West Nile virus case. Therefore, by examining how closely the modularity of a specific distributed sensemaking event matches the simulation model, we are able to determine the likelihood that irregularities occurred during that event. [Insert Figure 5 Here] Discussion and Conclusion The simulation model introduced in this paper produces a wide variety of insights into how distributed sensemaking events operate, what leads to high fidelity agreement, and the number of periods required to reach high fidelity agreement. Increasing the level of outreach increases the odds of high fidelity agreement and decreases the number of periods to high fidelity agreement, but increasing amount of reaching out only produces modest returns after a certain point. Therefore, increasing the outreach factor leads to desirable results, but given the added 13

14 effort required for actors to form ties at almost double the frequency and the modest gains this produces, it is likely that merely increasing the amount of information exchanged is not the optimal use of the actors time and resources. A middle level of reciprocity within the event increases the likelihood of high fidelity agreement and decreases the time to high fidelity agreement. Middle levels of reciprocity indicate that actors are splitting their times between exploitative information seeking, i.e., reciprocating, and explorative information seeking, i.e., connecting with someone new, thus providing further evidence of the benefits associated with balancing explorative and exploitative behavior [23, 24, 27]. Having a moderate similarity threshold increases the likelihood of high fidelity agreement, but the relationship between the similarity threshold and time to high fidelity agreement is linear. Of even more importance is the network structure that emerges as the factors vary within the simulation, specifically the network modularity. High fidelity agreement is more likely to occur when the network is moderately modular, a finding supported by past research [22-26]. What has not been shown in past studies is that having too little modularity in the network produces an elite network structure, which increases the odds of low fidelity agreement, and having too much modularity produces a market network structure, which increases the odds of no agreement. The greatest chance of an event resulting in high fidelity agreement occurs when the elite network structure is balanced against the market network structure, but this balancing act slows the process down, increasing the number of periods to high fidelity agreement, which is at its lowest when the modularity is low. Finally, we were able to provide empirical support for the simulation model by comparing the progression of the simulated network modularity over 14

15 time to the progression of the network modularity in two real-world cases, the diagnosis of the initial outbreaks in the United States of the Sin Nombre and West Nile Viruses. One of the most significant contributions in this paper is the Distributed Sensemaking Simulation Model, itself, though. Many simplifying assumptions were made in order to build the simulation and as we begin to relax those assumptions, the model becomes a platform from which we are able to address several phenomena. For example, the initial connect sets in the simulation are randomly generated, which means the underlying network structure from which the distributed sensemaking event evolves is also random. Past work has shown that different network structures can radically change how networks operate, whether those are structures with skewed degree distributions [28], small worlds [22-24], or networks with varying numbers of structural holes [29, 30]. So by altering how the connect sets are constructed, we can gain insight into how different initial network structures impact distributed sensemaking events. This could help experts optimize their interactions and networks prior to a crisis, hopefully increasing the likelihood of high fidelity agreement and decreasing the number of periods to reach high fidelity agreement. This paper also utilized a relatively simple representation of a category space, but there are many different ways that such a space can be constructed. For example, if we embed the category space within a taxonomy and then provide thematic cues that span the taxonomy, we could address issues related to how taxonomic vs. thematic processes improve the fidelity of categorization in distributed sensemaking events [31]. This can help experts optimize how they classify information and streamline the sensemaking process. These are just a couple of examples of how even slight alterations to the model can produce new insights in the fields of social network analysis and sensemaking. 15

16 Notes & References: 1. Weick, K.E Enacted Sensemaking in Crisis Situations. Journal of Management Studies, 25: Dunbar, R. L. M., & Garud, R Distributed Knowledge and Indeterminate Meaning: The Case of the Columbia Shuttle Flight. Organization Studies, 30: Cupp, O.S., Walker II, D.E., & Hillison, J Agroterrorism in the U.S.: Key Security Challenge for the 21 st Century. Biosecurity and Bioterrorism: Biodefense Strategy, Practice, and Science, 2: Comfort, L.K Rethinking Security: Organizational Fragility in Extreme Events. Public Administrative Review, 62: Pelfry, W.V The Cycle of Preparedness: Establishing a Framework to Prepare for Terrorist Threats. Journal of Homeland Security and Emergency Management, 2: Article Comfort, L.K. & Kapucu, N Inter-organizational Coordination in Extreme Events: The World Trade Center Attacks, September 11, National Hazards, 39: Taylor, J.R., & Van Every, E.J The Emergent Organization: Communication as Its Site and Surface. Mahwah, N.J.: Erlbaum. 8. Comfort, L.K Crisis Management in Hindsight: Cognition, Communication, Coordinaton, and Control. Public Administrative Review, Special Issue: Rerup, C. & Vendelø, M Attention Coordination of Time-Sensitive Heterogeneous Information: The Pearl Jam Concert Accident. University of Western Ontario, Unpublished Working Paper. 10. Spiro, J., Porac, J., Rao, H., Weick, K., & Lawrence, K Garden Paths at the Edge of Life: The Interorganizing of Sensemaking in the West Nile and Sin Nombre Virus Outbreaks. New York University, Unpublished Working Paper. 11. Diedrich, A Imagination and Preparedness in Organising The Case of the Outbreak of Marburg Hemorrhagic Fever in Angola. LAEMOS Conference, Buenos Aires. 12. Miller, J.H. & Page, S.E Complex Adaptive Systems: An Introduction to Computational Models of Social Life. Princeton, NJ: Princeton University Press. 13. Murphy, G.L The Big Book of Concepts. Cambridge MA: MIT Press. 16

17 14. Hahn, F.H On Transaction Costs, Inessential Sequence Economies and Money. Review of Economic Studies, 40: On the Notion of Equilibrium in Economics, 1974, Cambridge University Press. 15. Butts, C.T., Petrescu-Prahova, M., & Cross, B.R Responder Communication Networks in the World Trade Center Disaster: Implications for Modeling of Communication Within Emergency Settings, Journal of Mathematical Sociology, 31: Maitlis, S. & Sonenshein S Sensemaking in Crisis and Change: Inspiration and Insights from Weick (1988), Journal of Management Studies, 47: When the similarity threshold was greater than 0.7, the simulation would almost never converge and so we chose to not include a value of 0.9. Essentially, when the actors were too conservative with their interpretation, they could almost never agree on a category. 18. Johnson, S Emergence: The Connected Lives of Ants, Brains, Cities, and Software. New York, NY: Touchstone. 19. Strogatz, S Sync: The Emerging Science of Spontaneous Order. New York, NY: Hyperion. 20. Guiméra, R., Uzzi, B., Spiro, J., & Amaral, L.A.N Team Assembly Mechanisms Determine Collaboration Network Structure and Team Performance. Science, 308: Guiméra, R., & Amaral, L.A.N Functional Cartography of Complex Metabolic Networks. Nature, 433: Uzzi, B. & Spiro, J Collaboration and Creativity: The Small World Problem. American Journal of Sociology, 111: Lazer, D. & Friedman, A The Network Structure of Exploration and Exploitation. Adminstrative Science Quarterly, 52: Fang, C., Lee, J., & Schilling, M.A Balancing Exploration and Exploitation Through Structural Design: The Isolation of Subgroups and Organizational Learning. Organization Science, 21: Happel, B.L.M., & Murre, J.M.J The Design and Evolution of Modular Neural Network Architectures. Neural Networks, 7: Melin, Patricia Modular Neural Networks and Type-2 Fuzzy Systems for Pattern Recognition. Berlin Heidelberg: Springer-Verlag. 17

18 27. March, J "Exploration and Exploitation in Organizational Learning." Organization Science, 2: Albert, R., & A. Barabási Statistical mechanics of complex networks. Reviews of Modern Physics, 74: Burt, R Structural Holes: The Social Structure of Competition. Boston, MA, Harvard University Press. 30. Burt, R Brokerage and Cohesion. Oxford, UK, Oxford University Press. 31. Lin, E. L., & Murphy, G. L Thematic Relations in Adults' Concepts. Journal of Experimental Psychology:General, 130,

19 Table 1: Simulation Parameters Name of Description of the Parameter Parameter Structure of Parameter Values taken in the Simulation o Outreach Factor Likelihood of forming an outward tie Percentage: 0-1 r Reciprocity Likelihood of reciprocating Percentage: Factor a prior inward tie 0-1 t Similarity Threshold at which an actor Percentage: Threshold settles on a category 0-1 C n Category Set of n categories defined Binary Matrix: Space by vectors of cues # of Categories (n) x # of Cues CS Cue-seeding Vector of cues that the Binary Vector: Vector actors can observe 1 x # of Cues X i Connect The other actors that actor i Binary Vector: Set* can potentially connect to 1 x # of Actors Y i Reciprocity The actors that have Binary Vector: Set* connected to actor i 1 x # of Actors Z i Cue Set* The cues actor i possesses Binary Vector: 1 x # of Cues * Unique sets constructed for every actor in the simulation 0.1, 0.3, 0.5, 0.7, , 0.3, 0.5, 0.7, , 0.3, 0.5, x 30, ~25% of cells = 1 1 x 30, ~25% of cells = 1 1 x 30, ~20% of cells = 1 1 x 30, initially all cells = 0 1 x 30, ~30% of cells = CS, the rest of the values are set to missing 19

20 Figure 1: Simulation Mechanics 20

21 Figure 2: How outreach factor, similarity threshold and reciprocity factor impact the likelihood of high fidelity agreement and length of time to high fidelity agreement. 21

22 Figure 3: How outreach factor, similarity threshold and reciprocity factor impact the maximum modularity attained during the course of a simulation run. 22

23 Likelihood of High Fidelity Agreement Modularity Number of Periods to High Fidelity Agreement Modularity Figure 4: The impact of modularity on the likelihood of high fidelity agreement and amount of time needed to reach high fidelity agreement. 23

24 Figure 5: Comparing the simulated modularity progression and the modularity progression in two disease diagnosing cases, Sin Nombre and West Nile viruses. 24

25 Supplementary Material Further Description of Simulation Mechanics In the simulation model, there is a category space (C n ), a randomly generated 30 x 10 binary matrix that represents 10 categories comprising 30 attributes, and a cue seeding vector (CS), a randomly generated 30 x 1 binary vector that represents the complete set of cues an actor can gather throughout the course of the simulated distributed sensemaking event [1]. There are also three vectors that are constructed for each of the actors who could potentially enter the event. First, the connect set (X i ) is a randomly generated 30 x 1 binary vector that indicates with whom an actor can form an outward tie. Second, the reciprocity set (Y i ) is the set of actors with whom an actor can reciprocate a tie, which is empty at the start of the simulation since none of the actors have any ties to reciprocate, yet. Third, the cue set (Z i ) represents a selection from the cue-seeding vector that comprises the initial cues each actor possesses at the start of the simulation. Regression Analysis The simulation utilizes several stochastic processes and each run is independent and identically distributed, so we can utilize multivariate statistical analysis techniques to gain greater insight into how the different factors influence whether a simulation run results in high fidelity agreement or not, as well as the amount of time to high fidelity agreement [2]. For the models with the likelihood of high fidelity agreement as the dependent variable, we used a logit analysis since the dependent variable is a binary variable, i.e., the simulation either resulted in high fidelity agreement or it did not [3]. The length of time to high fidelity agreement is a variable that measures the number of periods until high fidelity agreement is reached. This 25

26 dependent variable is a count variable, so we use a negative binomial regression for that model [3]. Also, since we are only using a selection of the data when the dependent variable is the time to high fidelity agreement, i.e., the simulation runs that resulted in high fidelity agreement, we include the inverse mills ratio based on the results of the logit analysis we used for predicting high fidelity agreement, thus making this regression the second stage in a Heckman selection model [4, 5]. [Insert Tables S1, S2 and S3 Here] Despite the high correlations that exist between the various measures of network structure, the results of the analysis are relatively unchanged even when bivariate analyses are substituted for the fully specified models. This means that the results are not being driven by multicollinearity. One of the most important insights gained by including the regression analyses is how much of an impact the variables measuring network structure have on the likelihood of high fidelity agreement and the amount of time to high fidelity agreement. Including variables for network size, network connectivity and network modularity increases the pseudo R-square of the model determining of the likelihood of high fidelity agreement by a factor of five, i.e., an increase from 0.05 to Including the network structure variables in the model of time to high fidelity agreement almost triples the pseudo R-square, i.e., an increase from 0.05 to This highlights how important it is to examine not only the impact of the basic factors underlying the simulation model, but the impact of the emergent network structure, as well. 26

27 Modularity Through a process of simulated annealing, the Guimera and Amaral method of calculating modularity seeks to place different actors in different groups until the maximum modularity score is reached [6]. The modularity score is calculated using the following formula: Modular N M s=1 l s L d 2 s 2L Where N M is the number of modules in the network, L is the number of links in network, l s is the number of links between the actors in module s, and d s is the sum of the degree of the actors in module s. The first term of the modularity index, l s /L, measures the ratio of connections within the module to the total number of connections in the network, i.e., how many connections exist within rather than between modules. The second term of the modularity score for a module, (d s /2L) 2, is needed because without it, any complete component of the network would be considered a module and a network comprised of a single component, as many networks are, would only ever have a single module. The modularity score varies from 0 to 1, with 0 being no modularity, i.e., there are no subgroups within the network, and 1 being complete modularity, i.e., all of the subgroups are completely isolated from one another. Emergent Network Analyses There are many ways to measure the structure of a network, and in the main body of the paper we focused on only one of these, modularity. Two other measures are the final network size, i.e., the number of nodes, and final level of network connectivity, i.e., the number of ties per node. Just as they did with modularity, varying the levels of the outreach factor, reciprocity 27

28 factor and similarity threshold also impact the size and level of connectivity within the network structure. The relationship between the three basic behavioral factors and network size and connectivity are illustrated in figure S1. The same basic pattern of relationships between the outreach factor, reciprocity factor and similarity threshold and network modularity appear to be present for these other measures of network structure, as well. Given the high correlation that exists between different measures of network structure (see Table S1), it is not surprising that the measures emerge from similar circumstances. [Insert Figure S1 Here] We also include regression analyses with the final size of the network, final network connectivity and network modularity as the dependent variables and the basic behavior factors, outreach, reciprocity and similarity threshold, as the independent variables in table S4. The relationships dictating the emergence of the network structure shown in figures 3 and S1 are statistically significant and exist even when controlling for other factors. Likelihood of Low Fidelity and No Agreement The paper primarily focuses on the relationship between the outreach factor, reciprocity factor, similarity threshold and network modularity and the likelihood of high fidelity agreement, since that is likely be the desired outcome of a distributed sensemaking event. It might also be of interest whether specific levels of the outreach factor, reciprocity factor, similarity threshold or network modularity, influence the likelihood of either low fidelity agreement or no agreement, though. Being able to determine not just that there is a decreased likelihood of high fidelity agreement, but that the decrease is the result of an increased likelihood of either low fidelity or 28

29 no agreement could be extremely useful in anticipating what might happen in a specific, realworld scenario. Figure S2 shows how varying the levels of outreach, reciprocity and similarity threshold impacts the likelihood of low fidelity and the likelihood of no agreement. As the outreach factor increases, the likelihood of both low fidelity agreement and no agreement decreases. This is likely a result of the lack of information being exchanged between actors when the outreach factor is low, leading actors to rely on the small amount of cues they can accumulate locally. This limits the actors to low fidelity categorization or being unable to reach a categorization, at all. As the reciprocity factor increases, the likelihood of low fidelity agreement increases and the likelihood of no agreement decreases. Increasing the reciprocity factor increases the likelihood of low fidelity agreement because when reciprocity is too high, too much time is spent exchanging information with the same small group of actors, not allowing the actors to seek out cues from more socially distant sources and, thus, move beyond low fidelity categorizations. There is a negative relationship between reciprocity and the likelihood of no agreement; if there is low reciprocity, the actors residing in the immediate social vicinity will not share enough of the cues amongst themselves to be able to agree on the same categorization. [Insert Figure S2 Here] As similarity threshold increases, the likelihood of low fidelity agreement decreases. A greater similarity threshold indicates that the actors are more conservative in their interpretation of the cues and so will be more likely to wait for more information before being choosing a category. A higher similarity threshold typically means that the actors are too conservative and 29

30 may require more information than is available before making a decision about a categorization, thus increasing the likelihood of no agreement. The theorized relationships between modularity and both the likelihood of low agreement and no agreement were discussed earlier in the paper. When modularity is low, the network structure resembles an elite structure with one dominant module. The actors in the dominant module will not spend enough time getting cues from more socially distant places and will be much more likely to agree to a low fidelity categorization. When modularity is high, the network structure resembles a market in which each module has a subset of the cues, but not enough effort is made to spread those cues across modules. This leads to each individual module categorizing the situation differently, if they settle on a categorization, at all, and so no networkwide agreement is possible. Figure S3 shows that this is, in fact, the case. As modularity increases, the likelihood of low fidelity agreement decreases from 30% to almost 0%, and the likelihood of no agreement increases from 5% to 32%. [Insert Figure S3 Here] Modularity Progression One of the benefits of the Distributed Sensemaking Simulation Model is that it not only allows us to analyze the end result of a distributed sensemaking event, but to analyze how that event evolved over time. This can be captured in many ways, but we focus on how the modularity changes over the course of the distributed sensemaking event, an analysis we utilize to relate the simulation model to two real world cases, the initial diagnoses of the Sin Nombre and West Nile viruses in North America. 30

31 Modularity progression is calculated by finding the network modularity consisting of all of the ties that were formed up to a given period. As more ties are added to the network, the network structure changes and the network modularity evolves over time. The shape of the progression of the modularity over time in the simulation is shown for the three possible outcomes in figure S4. From the shape of the modularity progression for the different results of the model, there are insights into what influences whether a specific run of the simulation will result in high fidelity agreement, low fidelity agreement of no agreement. If the modularity increases too quickly, the network remains relatively fragmented and there are not enough connections between modules for the modules to agree on a categorization. If the modularity increases too slowly, there is not enough recruitment of new nodes with unique cues into the network. This allows the network to be dominated by a single module that will focus only on the cues contained within that module, leading to a greater likelihood of low fidelity agreement. Only when there is a balance between the rate at which connections are formed with new nodes, bringing new modules and cues into the network, and the rate at which connections are formed between modules to integrate the cues, will the distributed sensemaking event lead to high fidelity agreement. [Insert Figure S4 Here] We can also link how the network modularity evolves over time more directly to the outcome of the distributed sensemaking event. Figure S5 is a graph of both the change in modularity over time for a distributed sensemaking event that resulted in high fidelity agreement and the percentage of actors within the network that have settled on the high fidelity categorization. The percentage of actors that have settled on a high fidelity categorization increases at an increasing rate, which mirrors the decreasing rate of the modularity of the 31

32 network. Therefore, as the network structure stabilizes and new actors are being integrated into the structure consistently, thus keeping the change in modularity low, more actors are able to gain the necessary cues to determine the high fidelity categorization. [Insert Figure S5 Here] 32

33 Supplemental References: 1. We ran several simulation runs varying the number of categories, attributes and number of possible actors, and this had a minimal impact on the results discussed in this paper. 2. Kelton, W. D Statistical Analysis of Simulation Output. Proceedings of the 1997 Winter Simulation Conference. Eds. Andreadottir, S., Healy, K. J., Withers, D.H. & Nelson, B.L. 3. Greene, William Econometric Analysis. Upper Saddle River, NJ: Prentice Hall. 4. Heckman, J. J Sample Selection Bias as a Specification Error. Econometrica, 47: The regressions are primarily used to supplement the figures in the main body of the paper, verifying that the relationships shown in the figures hold even when controlling for other factors, and that there are enough observations (i.e., there are enough runs of the simulations) to support our assertions. 6. Guiméra, R., & Amaral, L.A.N Functional Cartography of Complex Metabolic Networks. Nature, 433:

34 Table S1: Descriptive Statistics and Correlation Matrix Correlations Variables Mean Std. Dev. Min Max High Fidelity Agreement 2 Length of Time to High Fidelity Agreement Outreach Factor Similarity Threshold 5 Reciprocity Factor Size of Network Avg. # of Connections Per Actor Modularity

35 Table S2: Logit Analysis: The likelihood of high fidelity agreement Variables Model 1 Model 2 Outreach Factor *** *** Outreach Factor (squared) *** *** Similarity Threshold *** Similarity Threshold (squared) *** *** Reciprocity Factor *** *** Reciprocity Factor (squared) *** *** Size of Network *** Avg. # of Connections Per Actor *** Modularity *** Modularity (squared) *** Constant *** *** Number of Observations Pseudo R-Sq Note: ***=p<0.01, **=p<0.05, *=p<

36 Table S3: Negative Binomial Regression: The length of time to high fidelity agreement Variables Model 1 Model 2 Outreach Factor *** *** Outreach Factor (squared) *** Similarity Threshold *** *** Reciprocity Factor *** *** Reciprocity Factor (squared) *** *** Size of Network *** Avg. # of Connections Per Actor *** Modularity *** Inverse Mills Ratio *** *** Constant *** *** Number of Observations Pseudo R-Sq Note: ***=p<0.01, **=p<0.05, *=p<

37 Figure S1: Outreach factor, similarity threshold and reciprocity factor all have a statistically significant impact on the final network size and level of network connectivity (p<0.01). As the outreach factor increases, the network size and network connectivity increases, but at a decreasing rate. As the reciprocity factor increases, the network size and network connectivity decreases. And there is a curvilinear, U-shaped relationship between similarity threshold and both network size and network connectivity, with the minimum value of both occurring when the similarity threshold has a value of