Economist John Duffy of University of Pittsburgh went to an experimental lab in Second Life and came away unconvinced. Some thoughts:
- The virtual lab he visited was a replication of a real-world lab. We've learned, however, that using virtual world technology to simply replicate a real-world thing is usually counter-productive. The utility of virtual worlds does not lie in their ability to re-do, in virtual form, what we do in the real world.
- He carries with him some honored norms about what an economics experiment must be. Participants must not know what the experiment is about, for example, so you ought to ask every participant "Are you an experimental economist?" and exclude them if they are. Otherwise they'll recognize that they are in a test of backwards induction or whatever. Duffy did not face that kind of question, nor anything that would allow the screening of participants based on expertise. There were other issues, other ways in which an attempt to replicate a real-world econ experiment, in traditional form, simply breaks when placed in a virtual environment.
- He properly noted that the incentives (couple hundred Lindens) are very low in terms of cash value. Why would anyone think hard in return for a quarter? However, this raises a debate about the utility of the Linden dollar. Some people seem highly invested in the state of their virtual things, despite low real-world values. This in turn raises big questions in economics about how things, utility, and happiness relate. Punt.
- He was able to lie about his age, a problem for the internet at large.
I come away convinced that there's little point in trying to replicate the standard micro-level lab experiments in virtual worlds. While you gain in terms of distance communication, you lose in terms of body signals and eye contact and such. In that kind of hands-on experiment, you can't automate much.
I would note, however, that Duffy's criticisms are cast too widely when he concludes that these problems also condemn truly macro-level experiments such as those proposed by Bloomfield and Castronova.
Macro level experiments are different breeds of fish. In moving to a truly macro scale in virtual worlds, we're gaining not just distance communication but large increases in scale (from two dozen participants to hundreds of thousands), length (from two hours to thousands of hours), and emotional investment (from a couple of bucks to things over which people have commited murder). These advantages, it seems to me, far outweigh the costs in terms of less control over participants, selection effects, and errors in demographic data. Moreover, at such scales, there are ample ways of dealing with these issues: you can't do a Heckman selection correction in a lab experiment of 27 people, but you certainly can do it when you have 27,000.
My sense is that Duffy thinks the goal, my goal, is to place traditional economic experiments inside virtual worlds. I agree with him that that would rather not be worth the time. And that's why the goal, in my view, is much more expansive: I want to create a huge economy in a virtual world and then experiment on it. As this is a new and different beast, it cannot and should not conform to the older norms of micro-level experimental research. It needs new norms, and we have our top men working on them right now. Top. Men.
(When I saw that Professor Duffy had misspelled my name, I was tempted to refer to him as "Professor Dufey." But that would have been childish!)
Recent Comments