Computational Social Science and the Problem With Hakkar’s Blood

Everyone remembers the Ironforge Plague, back when Zul’Gurub was new and some enterprising raiders brought the Curse of Corrupted Blood back from fighting Hakkar, and in doing so turned the Undercity and Ironforge into a scene from a charnel house. You remember it: Skeletons lining the hallways of every urban locale, the plaintive cries of the newbies as they died matched in intensity only by the laughter of the high-end hunters who brought the curse back with them by stashing their infected pets in ZG and then releasing them in IF. At the time I recall hearing that the Centers for Disease Control were excited by this story as a way of using virtual worlds as a way of understanding the spread of disease. And a couple of months ago Elizabeth alerted me to the article in Nature in which a couple of epidemiologists discussed exactly this idea in some detail.[fn1]

Now the problem for me is that Lofgren and Fefferman’s Nature article is really nice. It’s smart and informed and actually demonstrates that the authors actually know their Jin'dos the Hexxers from their Bloodlords Mandokir. The thing is that I’ve used the “Problem with Hakkar’s Blood” as one of my standard pitches about the dangers with drawing real world conclusions from virtual world behaviors.[fn2] And the article has once again made me ask what are appropriate questions for the field of what might be labeled “computational social science.” So after the fold, I want to discuss the conclusions of the article, and ask the Terra Nova hivemind for some help with understanding what computational social science might look like.

Ok, so let’s begin with the Problem with Hakkar’s Blood. This is just another way of saying that we have to be careful about draw real world conclusions from the actions of players in virtual worlds. The milieu of all virtual worlds is different from the real world, the stakes are different, the player base is unrepresentative of real world societies, etc etc. So it’s a mistake to conclude from the Ironforge Plague that people find it amusing in the real world to infect others with a fatal disease. Coz, you know, they don’t.

The nature of the conclusions we can draw from any study of the virtual world are radically circumscribed. We can certainly conclude that, e.g. “Some people playing WoW find it funny to watch noobies of their own faction die in the hallways as a result of an overly-strong debuff.” But this is a conclusion about gameplaying, about the nature of the virtual world itself. It’s not seeking to use the study of virtual worlds to make a conclusion about human behavior more generally. So, the Problem with Hakkar’s Blood means that it’s not clear to me that we can conclude much (anything?) about transmission rates of disease based on what happened one night among the Gurubashi.

Lofgren and Fefferman’s Nature article about the plague is actually really measured about this problem. Their conclusions are essentially little more than a call for more study on how virtual worlds can be used in epidemiology. They suggest, for example, that we could reduce the reproductive value of the curse to study differential propagation rates for the disease; and to study the different rates of infection in different zones that have different population densities; and so on. The article is worthwhile because it doesn’t make any conclusions from the Ironforge Plague, but simply suggests what we could do if we could tweak the state conditions of the world and from this see what happened.

Which leads onto the issue of “computational social science,” and the questions I have of you. I think that Dmitri “Catass” Williams was the first person to suggest that we should create two virtual worlds where all-but-one of the starting conditions are the same. This would mean that we could examine player behavior under controlled conditions, and could generate useful conclusions about the significance of the “dependent variable”, i.e. the only one which differs between the two worlds. So we might have two worlds, one where certain types of communication were possible, and the other one lacking it. We could start both worlds, and then see how they differed as a result of this one difference.

I think I was the first person to call this “computational social science” [fn3] because it seemed to me to be roughly analogous to the development of computational physics, which added a brand new methodology to the previously exclusive duo of theoretical and experimental physics. Computational social science would allow us to generate all manner of amazingly powerful insights, but there are lots of problems with the development of it. Most obviously is the difficulty of building and populating virtual worlds that are intended to answer social science questions, rather than provide a fun experience for the user. And then there is the issue of cost.[fn4] But let’s assume arguendo that we can solve these problems.

My question for the Terra Nova community is how we can develop computational social science and manage the Problem with Hakkar’s Blood? That is, what sort of issues could meaningfully be examined using the computational social science of two all-but-identical virtual worlds?

One obvious—but unsatisfactory answer—is only to ask questions where the significance of the answer is confined to the virtual world (or even more constained, only the specific virtual worlds under study). So we could create two virtual worlds where, say, the only difference was the presence or absence of an auctionhouse, or real estate, or soulbound items, and then examine relative user enjoyment of each world, or retention rates, user churn, etc etc. While the answers here might be of interest to the developers, it doesn’t tell us anything interesting about human behavior in general (i.e. in the “real world”).

The more difficult approach is to do what Lofgren and Fefferman are on about, and what I take Robert to be investigating in his work. This posting isn’t really the place to engage in questions about the design of any individual experiment, but I am interested to know what people think might be appropriate areas to investigate, and how.

Epidemiology and finance have been proposed. What other areas might work for computational social science? What questions might usefully be answered? And why?

---
[fn1] And Elizabeth’s mom provided me with access to the article, which I’ve posted here. I’m indebted to them both.

[fn2] I’m going to persist with the “real world” vs “virtual world” dichotomy which I know is problematic. Life is short, and I’m gonna ask you to indulge me this time because the alternative language is so clunky; and besides, if I’m being honest, I’d rather be pithy than accurate. Therein lies the story of my life and my downfall, but that (as Hammy the Hamster would say) is a story for another day.

[fn3] Although inevitably it was a journalist who interviewed me, used the expression, and then failed to even mention that he had spoken with me. I love the hours and hours that I’ve spent giving “deep background” to journalists in this area, only to fail to be quoted. Truly, I don’t miss those wasted hours at all.

[fn4] Although the recent announcements of Metaspace, Croquet, Sun, etc etc do provide some indication that this might become easier over time.

Comments

[Mike Sellers]

1.

"Computational social science" is a great phrase. Though I think this necessarily stands on the shoulders of Epstein and Axtell's seminal Sugarscape (from 1996) and their phrasing of "generative social science" (2000) -- in their words, building "social science from the bottom up." Sugarscape built models of reproduction, trade, renewable and non-renewable resources, wealth accumulation, disease, war, and other computational social aspects. Now, the agents in these (by our standards) early simulations were all autonomous -- simple NPCs in other words -- but the models are nonetheless compelling.

These models also show, sadly, how much the drive for simple entertainment ("kill monster, get gold") has thus far limited the development of virtual worlds as simulators and NPCs as anything other than vending machines. Still, I think even existing virtual worlds are a step above using a classroom full of college freshmen as experimental media for economic or other social science models.

This brings me to the first point you raised above: yes, the conclusions from any virtual world model/experiment are necessarily circumscribed -- the glee with which people passed on a deadly disease in WoW, for example, is clearly bounded by the cultural context of a world in which "death" and "deadly" mean something entirely different than they do in the physical world. But, back to the classroom of college students as an appropriate basis for any social science model, is a virtual world really any less tenuous as a basis for a model?

In terms of what could be effectively studied in a virtual world, certainly trade, aspects of disease/knowledge/meme spread, and other first-order effects can be studied. Personally I'm looking forward to being able to seeing what different civic models emerge once we support those beyond the static "guild" system, and of being able to study a wide variety of phenomena under the little-known umbrella of ekistics in ways that simply cannot be approached in the physical world. Of course this will require more of a simulationist bent to a virtual world (computational or online ekistics?), something that is almost anathema to most current virtual world designers. It gets in the way of "kill monster, get gold," and thus the first attempts at simulation in a virtual world -- necessary for most significant computational social science -- tended to annoy early customers of online games. This too is a difficult hurdle that will have to be overcome.

[Tom Boellstorff]

2.

I'll be interested to see where the discussion goes on this topic in the next few years. I can see the attraction of the idea of a "computational social science," but there are fundamental problems with the idea and reasons why it's always been quite unsuccessful. The debate goes way back in the history of all social sciences. For instance, the Boas-Mason debates of the late 19th century-and Boas's "The Limitations of the Comparative Method of Anthropology" is a great read in this regard.

The problems with the idea of a computational social science as it's described here are far more fundamental than the practical challenges of coding virtual worlds and then getting people to participate in them. Perhaps the key issue is that unlike a controlled experiment in, say, chemistry, where you alter one element and see what changes, human sociality is embedded in a kind of history that makes the identification of discrete elements usually spurious. We heuristically draw lines around domains of human experience and call them "gender" or "disease" or whatever, but "gender" in Indonesia in 2003 isn't the same thing as "gender" in Britain in 1940. Nor is it the same between the actual world and virtual worlds, or even in two virtual worlds created at the same time differing by one rule or facet. This is why the notion of "memes" seems like such an unhelpful metaphorical extension from genetics to culture, and why the idea of computational social science just seems like the replaying of that false metaphor on a larger scale.

Comparison is certainly a doable and often useful thing. I've been doing a lot of it lately in my other research life this year (the final chapter of my "A Coincidence of Desires" book, entitled "Comparatively Queer" that looks at sexuality in Southeast Asia, and my Annual Review of Anthropology piece on sexuality worldwide). But I'm still not convinced that the idea of a computational social science is useful. I'll be interested to read more examples of work that people find to be successful in this regard.

Can someone point to an example in the last 10-20 years where something that looks like computational social science has produced a convincing result? And importantly, result about what? Sellers pointing to Joshua Epstein and Robert Axtell seems interesting, but I would hardly describe this as "social science from the bottom up" because the "bottom" is the model itself and thus tautologous. You can't ontologize a model that purports to explain something outside the model itself. What do these outsider models tell us about experiential dynamics and individual motivations?

[John Beety]

3.

Cultural interaction is one topic that could be explored in greater depth. How does the server culture of, say, a European World of Warcraft server differ from that of a North American server? What would have happened had Final Fantasy XI chosen to separate servers by geography instead of letting Japanese and North American customers play together? Do the denizens of Teen Second Life play nicely relative to their Main Grid counterparts, or is the Teen Grid more comparable to Lord of the Flies? Those are just a few questions off the top of my head.

I think the Hakkar's Blood incident was useful for introducing new agents to the potential for virtual worlds, even if it does not work well as a real-world "plague." There have been thousands of abstracted computer disease simulations in the past (carrier status and so on), and grafting a slightly more sophisticated disease behavior onto a virtual world seems doable. In some ways, the attention paid to the Hakkar's Blood incident reminds me of the early days of virtual-world economics; something cool (a functioning economy) appears in front of an expert capable of communicating it to others (Dr. Castronova) and it snowballs from there. Where this will lead, I have no way of knowing, but it ought to be good.

[Mike Sellers]

4.

Tom Boellstorff: Can someone point to an example in the last 10-20 years where something that looks like computational social science has produced a convincing result? And importantly, result about what? Sellers pointing to Joshua Epstein and Robert Axtell seems interesting, but I would hardly describe this as "social science from the bottom up" because the "bottom" is the model itself and thus tautologous.

Convincing may well be in the eye of the beholder (or peer reviewer). And looking backward over the past 10-20 years maybe looking the wrong direction for an area that has had little real traction until recently. But "from the bottom up" is not tautologous -- what agent-based models in particular show is the often surprising epiphenomena that would not be predicted from the model itself, which focuses on the actions of individual agents rather than overarching downward-focused principles.

As one example (bearing in mind that it's been years since I read Epstein and Axtell's work), they showed that in populations where trade was not prevented but in which wealth accumulation was ('forced redistribution' would be one way of describing it), there was, as expected, far less disparity between rich and poor agents. OTOH when trade and wealth accumulation were allowed, the disparity increased greatly, following (as we'd expect now) a Pareto curve to the wealthiest agents. However, what was unexpected was that in the latter case, the economic carrying capacity of the population increased. The counter-intuitive result is that the seeming "unfairness" of "the rich get richer" system leads to a greater supportable population overall.

Now, is this significant or convincing? Can it be validated in the physical world? Very good and important questions. Whether it can be experimentally validated may be beside the point however; understanding the model's strengths and weaknesses (rather than simply dismissing it as tautologous) is vital for being able to place such results in context and to assess their applicability to other work and the physical world. The same is true, for example, with attempting to draw epidemiological conclusions from how a virulent disease is treated in an online game.

If nothing else, generative or computational social science will likely lead us to re-examine non-experimentally derived assumptions about a range of behaviors and social phenomena. As a colleague at DARPA used to tell us about our "artificial psychology" work -- 'all models are wrong; some models are useful.' The task for any form of computational modeling and simulation is to show utility in the form of uncovering previous assumptions and providing new ways to view existing data, not necessarily to show high-fidelity verisimilitude.

[Nic Ducheneaut]

5.

I think organizational problems in VWs are almost a 1-to-1 match to organizational problems in the physical world. The kinds of difficulties encountered by guilds when managing and planning raids are not that different from those you see at the office on a daily basis... So understanding the dynamics of groups online, and then manipulating a few variables to see how they affect the life of a group, seems like a good domain for computational social science to me.

Of course I'm only preaching for what I'm doing research on here :)

[Andy Havens]

6.

The problem w/ the blood, for me, comes down to one word: roleplay. The things I do through my character are not, in most cases, things I'd do myself. In many cases, they are *insanely* not things I'd do. Even within the realm of the metaphor, the behavior of me-through-my-character can't in any way be said to speak to how I would/might behave in RL. Why? Because I'm specifically playing in an MMO/VW in order to do things that I won't/can't do in RL. I'm roleplaying, which is going to yield very odd results when measuring anything I do for any circumstance other than the particular character I'm playing in a particular world.

In RL, I am incredibly mellow, generally. Very laid back, patient and hard to piss off. I have played characters (a dwarf hunter) that have similarities to my RL personality in those ways. And I have played characters (an undead warlock) that are nervous, mouthy, angry, vindictive and petulant. Different personalities and actions in different skins.

Now, even for those who don't "seriously" roleplay, I think you've got the same issue across much of what could be studied this way, from social, economic, political, etc. standpoints.

What might be useful, though, is to concentrate on things that exist in MMOs/VWs, but have direct application elsewhere; computer mediated communication, group project processes, virtual meetings, longterm goal management, etc.

Trying to measure stuff like economics and social adhesion is hard enough when done directly in RL. Doing it with people who are playing parts through toons and then trying to make conclusions... bad blood.

[Syntheticist]

7.

@Andy: The majority of people don't RP, so while I see your point, I don't think it's something computational social science would have to worry about. The concern, as Dan said, is that people do silly things for kicks, and yeah, watching newbies get pounded by a wild Infernal in the Auction House is always good for a laugh!

I like the ideas about organisational techniques, building on that, I'm interested in communication forms. Do people interact more in Guild Chat or open chat? Private tells or open says? Do they write letters? I think these sort of areas seem to communicate how people form social groups, and how they view "outsiders". There are many guilds that play the game for themselves, and see the rest of the server population as something that gets in the way. Does guild chat encourage that world view? Are people bitchier when they can whisper to a friend that the tank is absolutely clueless, knowing that the tank can't hear it?

[Andy Havens]

8.

@Synthesist: I know that most people don't RP in the "acting within the role" sense. But, to some degree or another, don't you have to roleplay at least inasmuch as your character has things to do that you don't? And doesn't the choices of character, class, race, trade, weapons, buddies, etc. infer a level of roleplay?

The desire to play the game at all, I believe, makes a statement about your desire to be involved in a world that is utterly (in most cases) unlike the one you inhabit.

Some choices, yes... they will reflect upon issues that bear on the real world. But what one does and loves today as a Paladin, that same individual may hate and shun tomorrow as a druid.

9.

Thanks for the thoughts, all.

My only comment at this point would be to note that the exchange between Syntheticist and Andy comes closest to my view about the PoHB. We know that RP exists, we know that people act differently in a game, we know that people act differently in situations where resurrecting a char is a matter of a few minutes lost rather than a Miracle That Surpasses Understanding. Knowing this, is there any type of question that one could ask with these two different servers that would generate meaningful information about human behavior anyway? What questions can be answered that are not influenced by the nature of the game, nature of VW-interaction, etc?

[Thomas Malaby]

10.

The idea that all models are wrong and some are useful is one to live by, at least research-wise.

@Andy: Absolutely correct, in my view. And the same holds true in our everyday lives -- the different domains in which we act prompt surprisingly different actions, even in analogous situations. Goffman ftw.

So, I'm guessing that the imagined research advantage here is not really about the difference between something that is our everyday (natural?) state and an *other world* (one without consequences, or external effects, etc.). It's really about the difference in the degree of *control* (and kinds of control) that we imagine a researcher could have over this particular domain as opposed to others.

Those *are* good grounds to be excited. That doesn't change the fact that, while it's exciting and worth pursuing, sometimes this work strikes me as something like trying to tweak the rules of a courtroom in order to understand, say, behavior in a nightclub (or is it vice versa? ;) ).

[Thomas Malaby]

11.

@? (Dan?): [Missed your comment while writing.] I think that the organizational angle is a good one -- institutions are easier to compare across circumstances than, say, individual experiences or emergent cultural meanings and stakes. This is probably because they tend to employ similar techniques of governance.

[Dan Hunter]

12.

Sorry, yes Thomas, that was me earlier.

I love your analogy of the nightclub/courtroom. And I was struck by Tom's really thoughtful comments about seeking to find commonalities when so much of our behavior and our construction of what amounts to behavior (or culture, or gender, etc) is socially contingent.

And yet, and yet...I'm still drawn to the notion that there must be some features of virtual world behavior that map onto other types of behavior. People still weat clothes in nightclubs and in courtrooms, even if the nature of the speech acts that they perform are different.

The reason I like the Hakkar's Blood example is that it's so extreme. Which is why I find the idea so intriguing that there might be other types of questions that do not have this problem.

And indeed I would find it as intriguing to discover that there is simply no question that VWs could answer. That would be a finding of itself.

[Tom Boellstorff]

13.

@Dan and Mike:

I’ll consciousness-stream some reflections on the excellent points you both raise because I’m in a rush (not to lump you together at all, I just don't have time or space to respond to other excellent comments made that others have already addressed very effectively).

The key methodological and theoretical issue here as I see it is that folks like Bartle and Castronova are absolutely correct that virtual worlds (regardless of the degree or form of roleplay) are places. And I would claim that places can’t be models.

This is the problem with the “bottom up” claims linked to modeling: this isn’t dismissing modeling, but pointing to the need for specifying how the referential claims work. Note for instance the following paragraph from Mike’s post, which to my mind is an excellent summary statement:

"As one example (bearing in mind that it's been years since I read Epstein and Axtell's work), they showed that in populations where trade was not prevented but in which wealth accumulation was ('forced redistribution' would be one way of describing it), there was, as expected, far less disparity between rich and poor agents."

The issue is: when you speak of "populations where trade was not prevented..." and "rich and poor agents," of whom are you speaking? Who are these populations and these agents? All human beings in all times and places? All human populations throughout time and history “where trade was not prevented but in which wealth accumulation was?” Is this a claim that the model definitively proves that “in populations where trade was not prevented but in which wealth accumulation was,” it will always be the case, forever and in all places, that there will be “far less disparity between rich and poor agents?” I doubt it.

So what did the model “show?”

This is the issue of tautology: a model needs to specify what it’s trying to model or it becomes tautologous. From my HIV work in Indonesia I’ve seen great uses of epidemiological modeling, but this is about behaviors understood etically—not about, say, the meanings attached to particular practices as “sexual” or not.

I love Dan’s statement that “I'm still drawn to the notion that there must be some features of virtual world behavior that map onto other types of behavior. People still wear clothes in nightclubs and in courtrooms, even if the nature of the speech acts that they perform are different.” I agree that there are many forms of referential and indexical relationships between virtual and actual, and also (importantly) between differing virtual worlds, but I wouldn’t see it in terms of mapping. This is again a parallel with actual-world cultures. Gay Indonesians reterritorialize and transform ostensibly “Western” notions of gay identity, but they are not “mapping” or “modeling” said notions of identity. This isn’t how folks I know who do social scientific modeling would think of modeling. And in all this I don’t see the “roleplaying” issue as all that crucial, since it could be taken to imply that persons aren’t playing all kinds of roles in actual world socialities, in which there can also be forms of anonymity and displacement.

There are some very important and interesting transformations of human sociality taking place in virtual worlds, and these transformations work upon and reference in myriad ways cultural logics from the actual world and other virtual worlds, but in terms of understanding these processes, I don’t think modeling will do the kind of methodological work that some hope it can. Since virtual worlds are places of human culture, using a virtual world to model other virtual worlds or the actual world makes as much sense to me as saying one could use Indonesia to model the United States or vice versa.

I still worry that the idea you could take two virtual worlds that differ by one “rule” and use that to discover something about human nature says more about an assumption that social scientific inquiry must mimic natural scientific inquiry to be legitimate. And it’s this that leads me to think of the history of this kind of endeavor, which goes at least back to Spencer. It’s not sufficient to say that the most interesting work has taken place recently and so looking backward is the wrong direction. Particularly because so much about virtual worlds is new, it’s crucial to be cognizant of the historicity of the methodological and theoretical frameworks we employ when studying them.

[Travis Ross]

14.

I think that even if we consider virtual worlds to be “places” they could still be used to provide insight into various institutional arrangements. If we have a framework that provides us with a way of modeling and understanding institutions then we should be able to use virtual worlds to test these institutions. For example, if you have a world that has been built with a highly subtractable public resource, you could examine behavior to see if a tragedy of the commons does occur in regards to the use of the resource. It might even be possible with the appropriate theory to test causation in these situations if the framework provides an accurate enough map of institutions. Even if you argue that there is no a way to rule out the possibility of a 3rd influential variable then at least you still can at least see correlation, and even that should be useful. Finally, evaluating institutions in this fashion could provide information for policy makers, even if we do not have perfectly simulated “real” conditions.

[dmx]

15.

Anyone know if those old Sugarscape models are still around on the net? I remember reading about them in some pop-sci book, and found the whole thing fascinating.

[Richard Bartle]

16.

So because WoW labelled it a "plague", that makes it a plague?

Thank goodness they didn't call buffs "highs"...

Richard

[Mike Sellers]

17.

Tom,

I have a feeling the direction I was going is different from where Dan and Dmitri are going with this, so I'll make this brief and try not to hijack things.

You wrote: The issue is: when you speak of "populations where trade was not prevented..." [in virtual worlds or agent-based models] and "rich and poor agents," of whom are you speaking? Who are these populations and these agents? All human beings in all times and places? All human populations throughout time and history “where trade was not prevented but in which wealth accumulation was?” Is this a claim that the model definitively proves that “in populations where trade was not prevented but in which wealth accumulation was,” it will always be the case, forever and in all places, that there will be “far less disparity between rich and poor agents?” I doubt it.

I'm not sure to what degree these questions are rhetorical or to what degree you're unfamiliar with agent-based modeling. The "populations" in Sugarscape are, as I said before, fairly simple software agents. They can be made more or less complex, more or less contextualized, but still represent an advance over more anecdotal methods of discerning patterns and systems in social phenomena.

As to the latter questions, would any existing social science or economic experiment make any such definitive claims about applying to all humans everywhere? It's vital to understand how model-making in agent-based simulations works before dismissing it (as vital, say, as having read Levi-Strauss, as came up in another discussion recently), and to understand both the strength of the modeled systems and the bounds of the contexts in which those models are applicable. Systems modeling is a lot like experimental design: it can be done well or sloppily, and reported well or poorly, but it's a skill that when applied well can clarify insights that otherwise remain opaque.

"Computational social science" faces the same hurdles of validation and utility that any experimentation or modeling does -- but in this way it at least has the possibility of validation and falsification, unlike much of the rest of social science, which may make only the most tenuous of highly contextualized claims, and without any possibility of validation beyond the author's skills at argumentation.

dmx, google "Sugarscape" - there's a lot of information, including as I recall a couple of sites with animations showing the populational changes over time.

[Andy Havens]

18.

@Thomas, who said: "...the different domains in which we act prompt surprisingly different actions, even in analogous situations."

Da. I know from years of work with marketing research that how people reply to a survey in a mall will be different to how they reply by phone, email, etc. And how they reply after a free sample--even on issues not related to the sample--will differ from how they reply if offered nothing.

The different problem with RP, though, is that I'm not always going to answer/behave *as me.* Or, more specifically, I will behave as different me-s from time-to-time.

It seems to me you could test lots of things related to the overall experience, but fewer things that are RL takeaways. Have two worlds with identical gameplay, but make one using stark, dark, gothic-y art and the other with happy, bright, cheery art. All the gameplay elements identical, but what looks like a gloomy ice-castle on Server A looks like a Disney special on Server B. Find out if people's behaviors, time-spent, etc. differ from one to the other.

Have one where NPCs will randomly kick your ass if you don't pay them some vig, vs. one where NPCs randomly dole out superior rewards if you tip them.

Stuff like that. But, regardless of whether I put on my hard-core RP robes, what you'll find out is how much I value Option A as it applies to this game, this character, this class, etc. Dark-n-Dank World might not have much of an effect on MMO playing... but does that mean we should goth-up our fast food joints? Probably not.

[dmx]

19.

'''
dmx, google "Sugarscape" - there's a lot of information, including as I recall a couple of sites with animations showing the populational changes over time. '''

@mike ;- Cheers brosef. Found it. (I kinda realised about 5 minutes after that, that it was online).

I'd like to see a a broader modelling framework thats somewhat more accessible than most to the casual coder to experiment with this sort of stuff. Like a prebuilt larger scale sugarscape with the ability to go

class myguy(baseguy){
def infect(target){
target.infected= true
}
}
(Obviously that wont work, but the idea of a simple modeling frame work you just subclass prebuilt agents, add behaviors and run them without needing a deep understanding of com-sci esoteria and advanced maths would be a great tool for the casual enquirer.

[dmx]

20.

.....that said ideal modeling framework would probably be written in scheme and feature some sort of aspect oriented coding setup. I guess beard-tech has its place lol.

[Dmitri Williams]

21.

I think this area has promise, but there's a piece missing from the discussion. There's obviously potential here, and there's obviously an easy mistake to be made by assuming that the rules and behaviors inside these spaces will map outside of them automatically. Clearly they will in some situations and won't in others. For example, Nic D.'s organizational example strikes me as mostly plausible while of course the plague is rubbish.

The key, then, will be to run validating experiments to see which kinds of behaviors, rule sets, incentives and social contexts will map and which won't. We can't really make the jump from virtual experimental platforms to RL until that step's been tackled. I envision parallel online/offline tests to see where and in what situations the outcomes remain the same and in which they don't. Once we do some of those we'll start to get a sense of when it's an appropriate tool and when it isn't. So Andy's shopping mall/phone survey analogy is pretty apt. You've got to have external validity before charging off on things like this.

BTW, when we ask people how much they role play on surveys and give them a range, it's a small number who say they lean toward actual RP and a much much smaller slice who say they are dedicated RPers. I just started a combined stat/ethnographic study of this and it's immediately obvious that while it's a rich practice, it's very much a small minority of players.

[jeremy snyder]

22.

i really like the idea, and i agree with nic's comment earlier that you can find a reasonable 1-to-1 match for anything you would want to study in at least some virtual world out there. the real problem is getting the parallel populations to do the tests. it's easy enough for us on the service provider side to take a snapshot of a world and put a second world online so that variables can be manipulated and so on, but then you need the players, and for them to have the same motivations so you can make consistent assumptions about behaviors and measure meaningful responses.

for instance, if you take a clinical trial of a new drug as a parallel situation, all the participants are motivated to get better, so you can assume what behavior they will exhibit during the trial. even if you got enough new players or migrating players, you're starting a new world, and you don't know what players might do with a fresh chance like that.

[Thomas Malaby]

23.

Just catching up on this thread. I appreciate what you were saying, Tom (Boellstorff, up a bit). Of course, anyone would be daft to believe that model-based testing is not already neck-deep in the assumptions/interpretations it must make in order to "tether" the model to the reality -- I hope no one is saying otherwise. After all, in the end, no matter how science-y it appears to non-scientists, it never escapes its utter reliance on metaphor (wherein one algorthim seems to be a pretty good model for a messy real-world process *because it captures the parts of that process in which one is interested in asking questions*).

This does *not* mean that it's any less science -- on the contrary, all science relies heavily on interpretation. In fact, and as I'm sure you know, all *inquiry* relies heavily on interpretation. Hypothesis-testing is not of a different order than the rest -- there really is no bright line that separates hypothesis-testing from exploratory research (and from critical and supported judgment, for that matter -- geologists use all three, regularly).

So I guess the biggest danger here is in treating this kind of approach -- which is a worthy one, I'd say -- as somehow intrinsically *more* worthy, or more real, than any of a number of other ways to get at what interests us about the connections between virtual worlds and other aspects of our lives. Instead, we should tools appropriate to the questions at hand. We'd be in a really sorry state if we started to believe that the only thing that counted as scientific were things with a transcendent promise of certainty. That more properly should be left to religious ideology -- I'd rather my bridges were built by pragmatists.

[Dmitri Williams]

24.

Jeremy, the solution comes from assembling a group of potential players and then assigning them randomly to one of the two groups. In high ed we get these people from classes, so their usefulness is limited. But with resources, we can gather samples of any population and offer them the right incentives. The hardest thing to do, though, is finding people who fit the profile, but haven't yet done something. But in the case of novel content or experiments for outside use, that is less of an issue because we aren't testing the impact of current games.

[Timothy Dang]

25.

Late response and long post. I'm preparing a talk on this very topic for a conference next week.

As far as Hakkar's Blood goes, it just clarifies that anyone wanting to interpret behavior--experimentally or observationally--needs to be simultaneously aware of the game metaphors and the underlying reality. Metaphor is important because it's an important motivator to the players. If I saw a stream of content-free numbers rolling down my screen and could work to maximize the first number on any row by pressing the F1-F12 keys at appropriate times, well... I'd rather be a superhero. The mapping to reality is important because "death" just plain isn't death. Usually "death" isn't even a bad day. One could possibly say that the Corrupted Blood disease was equivalent to a contagious case of 10-minute bouts of really loud hiccups.

Andy: Regarding RP, is that a problem? Lots of the research which seems interesting is about "emergent" kinds of results, things that happen on a grand scale as a result of lots of individual decisions. Unless I think that a large proportion of the players role play in a biased way, I don't have a problem with taking that role-played behavior as part of what happens. If there's something radical like 60% of people don't role-play at all and the 40% who do role-play choose to role-play hermits with vows of poverty, then that could cause problems.

Some significant economic ideas which (I believe) MMO's would be good to study:

- Specialization and the extent of the market. Going back to Adam Smith, this is the idea that if you're in a little village you might be a part time tailor, dairyman, carpenter, and doctor. As there's more and more people around, it makes more sense to specialize. And that specialization can lead to greater wealth (and possibly other problems). MMO rulesets create some control over the possible degree of specialization, and server population limits control the extent of the market.

- Transactions costs and the theory of the firm. This is one of Coase's famous ideas (interestingly, Coase has about 3 famous ideas, and different economists have differing strong taste for their favorite). As an economist, liking the market as a way of organizing production, Coase asked "OK, if the market's so great, why do we have these giant islands of command-and-control within the market that we call companies? Why isn't everyone an independent contractor?" The obvious answer is transactions costs--it would be a major nuisance to contract for everything, and the incomplete contract of employment, "You'll do anything I ask you to do, within reason, and I'll pay you," can actually work very well. So, if you increase transactions costs or change contracting rules, what happens to adventuring parties / guilds within an MMO?

- Travis is right on that common-pool resource studies could be good in MMO's, because they place the resource in a "real" ongoing social context which doesn't exist in lab experiments.

There's a bunch of others. Here's the way I think about it (this is from an economist's point of view, of course). Lots of economic theory which isn't captured by game theory comes down to: People won't leave money on the table. If there's some kind of potential benefit out there, someone will find a way to get it, which may well involve sharing it with others. A laboratory environment with highly restricted predefined action possibilities is great for some things, but lousy for that (not all lab experiments are so confined). MMOs, regardless of specific mappings of elements to RL, are open-ended enough, stable enough, and real enough that they could really get at such questions. They're also confined enough (hard-coded rules) and fake enough (controllable and with more limited ethical implications) that they can be practically manipulated for experiments.

From the academic culture I come from, there's an enormous drawback to MMO research, which is the inevitable lack of statistical independence. Even if, as Dmitri suggests, you could randomly sort people onto different treatment servers ("what do you mean I can't play with my spouse?"), you're going to have self-selection as people play more or less or quit entirely based on whether they like the treatment they got assigned to. Players in one treatment will probably be fully aware from web boards what's happening in the other treatment, and are likely to be aware of the research question. RMT could provide another undesired connection between the treatments.

So, if in an ideal world I had 30 servers in each of two treatments and I wanted to compare, say, guild size between the treatments, any statistical test I perform is really going to be bogus.

Statistical validity or not, I think it's worthwhile.

I'm in full agreement with Dmitri that an important (and challenging) part of developing the discipline is to experiment with experiments--what results can be taken from an MMO to RL? This has been part of the development of experimental economics as well. For a while many economists rejected experiments out of hand, since how can you expect a student playing a game for a few bucks to a professional making RL decisions worth serious money? So, part of the discipline is cross-testing results: students, professionals, different cultures, high stakes, etc. Lots of results have been found to be very robust, and it can be interesting to see which aren't.

On the other hand, for MMO research, I think that it will be harder to do these validating experiments. The very point (from my perspective) of doing MMO research is there's no other tool available which can practically address the same questions. So what form would these validating experiments take? They could be micro--testing small assumptions one at a time in an MMO and a traditional environment. This can have interpretation problems as well though. Lots of experiments can show contradictions of individual-level economic assumptions, but demand curves generally slope down regardless. Experiments could be validated across MMO's with different characteristics and/or player bases, but that begs the question. And experiments could be compared to simple RL non-experimental studies. I think that's the hardest but most valuable approach.

Regarding the term "computational social science"--I'm afraid that might bump into existing terminology and cause confusion. "Computational economics" is already a recognized term, for instance, and that doesn't include real humans as part of the model--Sugarscape is more like this category.

For that matter, does the "computational" aspect really matter? In principle, one could do the same kind of research by taking two groups of 1000 people and putting them on retreat-games and a computerless wilderness camp with different rules. I think serious military wargames (as in those practiced by the actual military with lots of people, rather than Advance Squad Leader) capture much of what researchers would find interesting about MMO research. The computational part of things helps in several ways: It makes the rules more concrete (though the rules can't control out-ofworld communication channels); it makes data collection (for many things) very easy; and most importantly via games it's created a situation in which lots of people seem willing to participate just for the fun of it (and pay for the privilege).

So, heck if I know, "ludic social science"?

[Kent's Imperative]

26.

http://kentsimperative.blogspot.com/2007/10/epidemiological-intelligence.html

Epidemiological intelligence reconsidered

....The question of the potential utility of virtual worlds for examining epidemiological effects in simulation remains a fascinating area of study. It is a natural extension of other research attempting to track in a similar fashion the effects of viral ideas – memes – within simulated virtual populations for information operations / psychological operations studies. This is the sort of analysis which may dramatically alter the manner in which medical intelligence professionals approach their craft in the future....