Some of you may have been following the Arden project (reported in Nature and the Chronicle of Higher Education). I'm pleased to announce that the project has come to fruition. With generous support from the MacArthur Foundation, we have created a fun game environment and used it to conduct a month-long experiment. Our experimental question (kept secret up to now) was: Are fantasy game players economically "normal"? Or on the contrary, when they make themselves into elves and dwarves and hobbits, do they stop taking economic decisions seriously? We created two virtual worlds, one an exact copy of the other, except that in the experimental world the price of a simple healing potion was twice as high as in the control. If people are taking prices seriously in this fantasy environment, they should buy fewer of the potions when potions are more expensive.
At stake here is the entire idea of using virtual worlds as a Petri dish. If fantasy gamers behave in ways that violate our most basic assumptions of economic normalcy, then it makes no sense to use virtual worlds to study large-scale economic behavior. If, conversely, fantasy gamers seem to be normal economic agents, then perhaps some of the behavior in virtual worlds does indeed generalize to the real world. If so, then we can consider using virtual worlds to conduct controlled experiments at the macro scale of society, where our most pressing problems seem to live (natural resource management, intercultural mistrust, information security, disease).
The initial findings of the Arden experiment will be released during the International Communications Association meetings in Montreal next weekend. The session we're part of is this one:
"High Density Session: The Web 1.0, 2.0, and Beyond"
Time: Sat May 24, 3:00 - 4:15pm
Place: Le Centre Sheraton / Drummond West
Get more information about the meeting here.
In this format, eight presenters will each have 5 minutes to describe her or his work. Then we will go to our posters, pasted to nearby walls, where each of us will answer questions about the findings. I'll be standing at my poster; thus if you have a particular interest in the Arden project and its findings, please feel free to attend this session and see them first hand. Fresh off the internets, as it were.
I will collect comments from the ICA meeting and use them to revise the paper we're writing, before sending it off to a journal. This will occur sometime in June. At that time we will also release the paper as a Working Paper. An announcement about the paper will be made here.
Just so I understand, you've been looking explicitly at fantasy worlds correct? Not just choosing fantasy as representative of virtual worlds in general?
Posted by: Tim | May 16, 2008 at 16:20
Healing potions don't strike me as very elastic... I look forward to reading the paper.
Posted by: Craig | May 16, 2008 at 16:24
I guess this line really bugs me...
To take the leap from "If it's so in fantasy then it is so in all virtual worlds" just seems pretty... er... brave. Fantasy just implies, well, fantasy - as in "not real". I guess I'd rather see something done in a world that was highly depended on economic issues. Something like EVE Online for example, or a game where vital (and in my mind health potions aren't vital) elements could only be bought.
Dunno. I guess I will wait until I see the study.
Posted by: tim | May 16, 2008 at 16:47
I would also have studied the amount of time spent farming up the gold in order to buy those potions, because if certain tasks in-game demand a particular number of potions... then the players are going to buy that many, unless you are studying the cumulative potions bought by the entire population.
The time is real though, and gamers, particularly MMORPG gamers, are notoriously efficient. So they economise their time, in order to get the greatest return for their investment. That is to say, how much time grinding gold gives the opportunity for time playing the fun encounters?
Posted by: Adam Ruch | May 16, 2008 at 20:49
Good point, though I'd suggest that at times MMORGP gamers are also notoriously obsessive and addicted. Doing an economic study based on slot machine usage in Reno for example might not be such a good thing :^)
I guess (tiptoe tiptoe) I am both enthusiastic (tiptoe tiptoe) and also unenthusiastic (tiptoe tipoe) about this whole post. To put it another way, "All this hype for all this time, and this is all we get?!?"
Arden and the NASA MMO were two very very interesting and very, very high profile MMO projects that at least in terms of my expectations (and I think those of others), flopped. And badly. They had such promise. They got people thinking, talking and enthused about the idea of an MMO for education and research.
I have great sympathy with the realities of academia, budget realities and the pains in developing a project like this. But what I see is that clearly a lot of effort was spent on PR, but not much on results. I understand this and have done it with my own projects in the past. What I don't understand is how such powerful, well respected and high profile groups could do this.
There's a real meta-story here that I wish Terra Nova or someone would have the nerve to take on - and that is why have two VERY well publicized educational MMO projects gone from media darlings to the stuff of WTF?! It's harsh, but isn't it what some people are thinking?
Posted by: tim | May 16, 2008 at 23:39
Ted>At stake here is the entire idea of using virtual worlds as a Petri dish.
I'm guessing, from the observation that this is raising the stakes rather than lowering them, that the result is not going to turn out to be entirely disappointing.
Richard
Posted by: Richard Bartle | May 17, 2008 at 04:24
Though I'll be surprised if you don't address this in your paper: Does a negative result in the Arden experiment actually preclude using an MMO as a petri dish? To me, even if a the result is counter to what you guys are obviously hoping for, an MMO already represents a kind of society, a sub-society, which is all we can hope to study anyway, isn't it? I don't think (in a casual sort of blog-comment off-the-cuff way) that Society (capital S) is reducible to being represented by any particular analogue or another.
However, more directly addressing your issue here Tim, I think that what Arden is meant to establish is the value of all the other insights that many people are already deriving from MMO's. In order to observe and analyse trends, social patterns, etc etc within an MMO, one has to establish that there IS a dialogue between the meat-space mundane world and the virtual one. This Arden project is the hinge on which future statements about what MMOs tell us about society, people, culture or whatever, turns. It is the point of relativity, a lens through which actions/events in other MMO situations can be compared.
Posted by: Adam Ruch | May 17, 2008 at 05:53
Interesting project!
One question that pops to mind is that of sample size. How big of a playerbase do you need to get such that you're really measuring some sort of legitimately averaged human behavior and such that you're minimizing the impact that performing this experiment on a highly connected community (which is thus not representative of how normal consumers act when buying, say, Pepsi) has?
--matt
Posted by: Matt Mihaly | May 17, 2008 at 20:20
Seems everybody here overlooks the real problem:
Why should players of virtual worlds CARE if they blow all their virtual money on something they don't need?
There's not the same consequence in a game as in our real life, where lack of money might mean you and your family goes hungry, loses the house, etc.
In a virtual world, allthere is at stake is missing out on "fun content", which might not really be a big reason to economize.
Posted by: Thomas | May 18, 2008 at 05:07
@Thomas: Testing your hypothesis is the whole point.
@Matt: The requirements for sample size depend on the amount of information you're collecting. We're examining one piece of information. Groups of college students drawn from classes are a very typical protocol in media research.
@Tim: I didn't do a very good job of understanding my own limitations, of grasping how hard this is to do, or of managing the expectations of people out there. All I can say is, sorry if you're disappointed this isn't WoW with Hamlet; we now have another project going; it should be better on all these dimensions.
@Richard: It's not just rhetoric! I bet the ranch!!
Posted by: Edward Castronova | May 18, 2008 at 07:10
I understand what the point of your experiment was, but... it seems to me that some fundamental assumptions of economic theories are that
1) money will be hard to come by
2) money can be converted into something intrinsically valuable
Now, in your game (which I havent had the chance to try out) you could reasonably replicate point 1. But still, people would know it's just monopoly cash.
Number (2) is more difficult on a tight budget. WoW accomplishes it in various ways, one of the ways being that the top gear can help you reach content that you paid real world money for, or gives you something close to real respect and envy from peers. Even admiration for virtual gear is still admiration, it seems.
So, where am I going with this?
You wrote that
'If fantasy gamers behave in ways that violate our most basic assumptions of economic normalcy...'
Well, those assumptions are based on fundamentals that may not be replicated in your game world.
So what do you hope to prove or disprove, other than "in a world where nothing is worth buying, people don't care how they spend their money"?
No offense intended towards you or your project of course, it just baffles me.
Posted by: Thomas again | May 18, 2008 at 09:22
@Ted: What were your ethical considerations of the deception involved in the project? (That could include both Arden and Arden 2.)
Posted by: Tripp | May 18, 2008 at 12:13
The players economic choices are probably only telling if they are both interested in their current play and have drunk the cool-aid enough to be looking ahead toward advancing their avatars situation. If they think they might quit in a few days or even a few weeks, their economic decisions are going to be less disciplined and more whimsical.
I know my son, who played runescape for 6 months, had me suspend his account but still played and had a new goal to amass a huge amount of one commodity for the heck of it even though he had played the game more economically sensibly in the past.
Long and short of it is, I wouldn't trust the data from players who aren't really interested in the game. With an only moderately engrossing game, perhaps a whimsical one where many players are there for reasons other than advancement, your data will probably suffer.
Posted by: shander | May 18, 2008 at 13:49
Tripp, I don't see any deception in this design. Subjects knew they were in an experiment and signed a standard release form. What are you thinking of?
Posted by: Edward Castronova | May 18, 2008 at 16:40
Ted>It's not just rhetoric! I bet the ranch!!
Oh, I know this was a high-risk venture! I'm just saying it looks as if your bet came off.
If it hadn't come off, it would have still been OK from my point of view as it would mean we'd need a different theory of economics for virtual worlds than for the real world, which isn't exactly fatal. However, it would have badly hurt any attempts to use virtual worlds as petri dishes for real-world economic decision-making.
If it has come off, that's extraordinarily good news for you. Assuming that other economists pick up on the idea and get solid results from it, it could be amazing - and there's a Nobel Prize for economics..!
Richard
Posted by: Richard Bartle | May 19, 2008 at 05:02
whats the real problem?
Posted by: Financial | May 19, 2008 at 09:19
Ed,
Interesting experiment. Did you track player "farming" behavior as well? For example, if you treated the health potion like food/water/commuter gasoline in an economic equation - something the person considers a "need" rather than a "want" - they might work twice as long to buy the same number of potions rather than buy half as many.
I'm sure there's a clever economist name for this kind of behavior, but I'm not an economist.
Posted by: Axecleaver | May 19, 2008 at 15:11
Maybe I'm confused (it's happened before!). The deception I was referring was no so much for the subjects but "the rest of us." That is, you presented the project as being centered on teaching about Shakespeare, his work and his world. But in fact, that had nothing to do with your real research, if I understand correctly.
Maybe part of the confusion for me is the distinction between Arden and Arden II. Was the first one just a prototype to get to the point of the second which was then the real one?
I guess in looking at your post here, I got the impression that the project was not as successful as you would have liked. About the future of the project, after the fall, you said, "my stock response now is "don't hold your breath; nothing worth noting is going to happen for a long time." The Bard has left the building for now, and his return date is unknown."
So am I wrong in thinking that this was a bit of a ruse for the "real" work of the project? Or did I miss a chapter in the project's story? -Am I making sense?
Posted by: Tripp | May 19, 2008 at 16:25
Whether it economically makes sense to buy fewer healing potions when they are twice as expensive depends on whether or not "twice as much" is still trivial. In the real world, a piece of bubblegum at 1 cent is no different to me than a piece of bubblegum at 2 cents. So part of the question is, what percentage of my current cash available does the potion cost, and is that percentage trivial to me?
Also, if I buy all the healing potions I possibly can, using up all my cash, in both versions of the game (at "regular" price and at "twice" the price) then assuming I make the same amount of money in both versions, I'm still buying half the number of potions in the more expensive version but that doesn't mean I'm taking my money any more seriously. It's just that I run out of money faster.
Another important factor is how fast I heal up in between fights without the potions and whether I can survive the fights without the potions. How inconvenient I find it to have to wait to heal between fights and how necessary I consider the potions to my survival are other factors the may be somewhat independent of price.
Posted by: Susan | May 19, 2008 at 20:21
Because again, they spent real time to get that money. I think you'd have an argument on your hands suggesting that money could be traded for something 'intrinsically' valuable. There isn't anything intrinsically valuable about the piece of bubble gum Susan has used as an example, but it is 'worth' something in that people are willing to trade their money for it. Money, being the representation of time spent working, serves as a conduit between time and item.
Virtual gold is no different. Though it looks different, is represented differently, comes in different denominations--that is all superficial. It serves as the same conduit between time spent doing something not immediately useful to the player (work/farming), in exchange for something that does have a use which the player appreciates, wants, needs, and therefore values.
@Susan:
If a particular boss fight requires no less than 4 healing potions per attempt, in the 'regular' version we'd see X amount of time spent to gain money, in order to by Y number of healing potions, to then do 1 boss fight (B), right? (1X = 1Y; 4Y = 1B, so 4X = 1B) In the Double Price version, we'd see a similar relationship, though it should look more like 2X = 1Y instead, and 8X = 1B. So if we know that it takes 10 minutes to farm enough for 1 healing pot, then we'd see 40 minutes spent farming for the potions, to do 1 boss fight. Where in the double price version, we'd have to see 80 minutes. OR we'd see half the number of boss fights done. OR we'd have to see players getting creative to avoid using their potions in that fight.
There's lots of ways to extrapolate the data from an experiment like this. How many boss fights are attempted? On average, how many monsters are killed per session, before a player goes back to stock up on Gold and Potions again? Does that cumulatively affect how fast the server population levels up? Did the cheap server accomplish things the expensive server did not? etc etc.
/anticipation
I want to know your results.
Posted by: Adam Ruch | May 19, 2008 at 21:29
Matt and Susan raised the two questions that occurred to me.
The number of players matters because it affects the likelihood that one's economic enounters will be one-shot deals with strangers, rather than repeated exchanges with familiar characters. (Evolution of cooperation stuff.) Prices in a game with a few players who know each other (and know they need to depend on each other over the long term) are likely to be lower than prices in a game with so many players that most economic events are unique exchanges with strangers. (This might not be a large factor if goods are purchased only from NPC vendors, but it's significant in a player-run economy.)
The average amount of money a player possesses -- relative to the typical price of the goods being watched -- matters, too. As Susan said, if most players have (say) 100,000 in cash, they're unlikely to make a distinction between one item that costs 20 units of currency and the same item that costs 40 units. That's probably less true if players need to purchase the item repeatedly, as might be the case with healing potions. But I think it still stands that players in a cash-rich game are less likely to alter their economic behavior with respect to relatively inexpensive goods than players in a game where money is hard to come by.
Finally, I'd like to know: what are the "basic assumptions of economic normalcy" that Arden was intended to explore? Those were defined before the tests for them (such as healing potion prices) were devised, yes?
Like the other folks here, I'm very interested in this whole effort. I'm looking forward to hearing about the results and conclusions!
Posted by: Bart Stewart | May 20, 2008 at 12:54
Bart nicely summed up my reaction as well. The first point is particularly important. We know that people see the various resources in their lives (connections, competencies, and -- yes -- cash, among others) as, in practice, convertible Thus, even the degree to which folks are "economically normal" (is this a smuggling in of RCT under another phrasing?) is something that is heavily dependent on contextual features, including the ability to accumulate the moral obligations of social capital.
Posted by: Thomas Malaby | May 20, 2008 at 14:06
It seems quite a reach to consider a single study of a single virtual world space, no matter who did it or what their credentials are, to be a "hinge on which future statements about what MMOs tell us about society, people, culture or whatever", or to result in a "At stake here is the entire idea of using virtual worlds as a Petri dish" perspective.
It's only ONE study of ONE virtual world space, at a time when virtual worlds are *only just appearing* on the radar. Any credentials of the experimenters can only be of pre-existing technology or societal realities... we're still only just beginning to discover how virtual worlds will manifest themselves.
If somebody had predicted that the future of the World Wide Web as a viable business model for sales of music was at stake during the first couple of weeks that Amazon.com started to sell printed books online.... what would have been your reaction?
With all due respect to the past accomplishements of Dr. Castronova.... isn't it really much too early to be making such statements?
Perhaps much more valuable are the insights gained from the experience in which the original platform, Metaverse, had to be abandoned after intense efforts revealed that there was far more complexity involved in building a virtual world than had been previously thought, even to someone who had spent hundreds of hours studying virtual worlds.
At the very least, regardless of the content of the study, everyone should realize that we are in the situation of trying to identify that metaphorical elephant while peeping through a small hole in a fence as the elephant walks by on the other side....
Posted by: MMORPer | May 20, 2008 at 18:55
MMORPer>It seems quite a reach to consider a single study of a single virtual world space ... to result in a "At stake here is the entire idea of using virtual worlds as a Petri dish" perspective.
Would this argument wash if the study failed? It's easy to say that a positive result means nothing because it's only one study, but not quite so easy to say that a negative result can safely be ignored on the same grounds. Could Ted realistically ask for money to create a new study if the results of his first study were bad? It would be very difficult to do so. We'd have to wait quite a while before anyone could try something similar again.
This isn't to say it could never be done again, but it would be something of a setback.
>It's only ONE study of ONE virtual world space
I agree, but that isn't an argument that Ted could have used in his defence if the whole thing had failed to deliver.
Richard
Posted by: Richard Bartle | May 21, 2008 at 02:48
Chewing gum will always have an intrinsic value, because of the pleasure kick from the sugar and chewing.
However, this pleasure comes from real-world sensations that are not (yet) possible in an online world.
I don't buy the argument that something is valuable just because of the time spent aquiring it.
If you think so, I'd like to sell you some prime beach front property - which is to say the nice sand castle I just spent four hours building :)
When kids build those sand castles, half the fun come from tearing it down again. But does that mean they violate our most basic assumptions of economic normalcy?
I hope you don't mind me playing the devil's advocate here.
Posted by: Thomas^3 | May 21, 2008 at 04:14
I was discussing this with a teacher of mine, who reminded me that paying for things will make them be percieved as more valuable. Instead of seeing less potions in combat if they are more expensive, one might actually see more of them, because they are being percieved as more valuable. If they are this expensive, they must be really good.
How can you reject a hypothesis if any outcome could support it, one way or another? What you are trying to test is a very complex subject, so whatever the outcome you cannot draw any real conclusions about using VW's as petri dishes in general. No really, there is no way you can do that.
Posted by: kevin | May 21, 2008 at 04:51
@thomas:
Another aspect is the death penalty and the competition involved. Compare this to the real world: if my healthcare premiums go up 100%, will I still need insurance? I might not get sick; I might not die.
A better test might have been an item that gave a good buff -- something optional that gave a better chance of success, vs. something that prevented a loss.
But there are games I play in when I could not bother to care if I died or not (LOTRO), and there are games where the death penalty is pretty severe (say, Eve Online).
Also, if I die, am I grouped? If I am the only healer in a group I would care more about keeping myself healthy to prevent a wipe. If I am a tank in a group, if I stop getting healed, we're facing a wipe anyway.
I'll be interested in seeing the factors around the experiment, and how they were isolated...:)
Posted by: Shava Nerad/Shava Suntzu in SL | May 21, 2008 at 11:16
What Ted's doing here is establishing a necessary, but not sufficient condition for testing the petri dish model. If it works on this most basic test, then the idea itself is viable and can be expanded into other domains, contexts, etc.
Let's not get too ahead of ourselves. This is an important step, but it's also not attempting to be an endgame. Therefore, let's not hold it to that standard. We'll get there.
Posted by: Dmitri Williams | May 21, 2008 at 19:41
Thomas^3 -> The argument isn't so much that time = value, but that if someone is willing to part with their own time in order to obtain an item or build a castle, INSTEAD of using that time to do some other thing, then the gum/castle/healing potion must have SOME value to that person. Whether that value is quantifiable in monetary terms is totally open on a case-by-case basis. The value of a sandcastle isn't so much the thing itself, as the accomplishment of having built it, I suppose. Sand is just sand in any arrangement, but so are bits and bytes in a computer game. Perhaps the value comes from the accolades and Hurrah! from your family/friends/guild for having achieved the epic sandcastle/sparkly weapon. No, you can't actually DO anything much with a sandcastle, other than knock it down, but you can claim it as an accomplishment. Another problem with value is the assumption that value = $. Markets are not fixed a priori, and just because YOU wouldn't pay money for a sandcastle, doesn't mean that no one would. Or perhaps pay for a skilled sand sculptor to build you a sand-car for a TV ad? Or for sandcastle building competitions? Market value vs Personal value is something I think we should tease out a little more, because I don't buy the argument that you have to be able to SELL something to consider it valuable.
The other tack is this: Someone mentioned before that spending virtual gold is never going to make your family go hungry... unless you choose to play WoW instead of go to work. In which case you've decided (based on addiction or other slightly more rational terms) that for you, today at least, the guild's raid is more important than the day's salary. So its not an exact exchange rate, but there is some kind of value trade-off happening, to me anyway.
Posted by: Adam Ruch | May 21, 2008 at 21:00
It would be an interesting additional step to interview participants and hear their thoughts on this...what did they think about the price of healing potions? Did they seem high? low? How did they decide how many to buy? What alternatives were there (e.g., finding a healer instead of using potions?), etc.
I know "hard data" is nice to deal with, but subjective experience is helpful in gaining an understanding too.
Posted by: Tripp | May 23, 2008 at 10:49
I had a lot of fun with shander's post, substituting "real life" for "game", and "individual" for "player." One does not have to be on-line to act in an irrational manner. Remember this the next time you read a newspaper story about yet another person hoarding cats.
The point of the experiment, as I understand it, is to see if behavior in VWs can be correlated with behavior in actual life. The only way to do this is with experimental method of limiting variables, like this test does.
Posted by: CherryBomb | May 26, 2008 at 17:13
Why has anyone not considered the heuristic validity of this? The population being studied is North American adults who play fantasy "on-line" computer games? The effect of the manipulated variable is pointless in such a niche group. Someone clarify what the sample population is here...
Posted by: Derek | May 27, 2008 at 04:33
I challenge you to a game of trivia! Click here to battle against me online at ConQUIZtador. Let's see who's the winner... https://www.conquiztador.com/?a=26041
Posted by: p005 | May 29, 2008 at 08:31
Derek -> The 'niche' you describe is representative of tens of millions of people. Secondly, its not really applicable to throw people who otherwise would not play online games into the study because they aren't making decisions in 'natural' online virtual worlds anyway, rational or irrational. Thirdly, this is an initial ground-work study as has been mentioned before. These things have to start somewhere.
Posted by: Adam RUch | May 30, 2008 at 19:31
IF you use a well balanced diku style MMO, the players will buy more healing potions if you rise the price. Until dying becomes cheaper than buying potions.
There is even a name for this effect, but I forgot it, I'm afraid. If I remember well, the scenario used in economics to explain the effect is England at times of war. People are suffering and live at the bare minimum of existence.
If you have $10 a week and need to eat, you can either buy bread for $1 or meat for $2 a day. Best case, you can eat 5 days bread and 2 days meat = $9 per week.
If the price for bread would rise, you have to buy more bread, as you can't afford the two days of meat anymore, yet you still need to eat.
So if, in such an extreme economy, the price for bread rises, you will sell more bread.
As this extreme economy at war is exactly the goal you have in mind when balancing the economy for an MMO (players at the bare minimum they need for a virtual living), I would expect the players to buy more healing potions if the price for healing potions goes up. Until there is another alternative or they don't need healing potions anymore.
Posted by: Wilfried Henseler | Jun 03, 2008 at 21:17
Tripp said a bunch of things:
Maybe I'm confused (it's happened before!). The deception I was referring was no so much for the subjects but "the rest of us." That is, you presented the project as being centered on teaching about Shakespeare, his work and his world. But in fact, that had nothing to do with your real research, if I understand correctly.
It's been a consistent frustration of mine that I can't seem to explain the core concept of the project. The project's intent was to develop virtual worlds as a social science research tool. I also thought it would be good to have a Shakespeare virtual world. So I decided to use Shakespeare as the lore. Other people decided that all I was doing was virtualizing Hamlet. I viewed that as a fairly harmless misunderstanding. But it turned out not to be harmless. Tripp's accusation of deception is something I wouldn't have to live with if I had chosen hobbits instead of Shakespeare.
Maybe part of the confusion for me is the distinction between Arden and Arden II. Was the first one just a prototype to get to the point of the second which was then the real one?
No. Arden 1 failed. It was a failed attempt to make a fun game. So I abandoned it and made Arden II. Arden II has not failed.
I guess in looking at your post here, I got the impression that the project was not as successful as you would have liked. About the future of the project, after the fall, you said, "my stock response now is "don't hold your breath; nothing worth noting is going to happen for a long time." The Bard has left the building for now, and his return date is unknown."
The project was not as successful as I liked. We tried to make a Shakespeare game, and the game we made was no fun. Shakespeare was a mistake. In the fall, I realized I had to start defusing the expectations that had arisen about a World of Warcraft-sized Hamlet world. Face it: the Bard was boring. As a result, Arden II has almost no Shakespeare in it. It's just D&D. And it's a lot more fun.
So am I wrong in thinking that this was a bit of a ruse for the "real" work of the project? Or did I miss a chapter in the project's story? -Am I making sense?
I will leave it to others to judge whether you're making sense, and to yourself to decide whether you've grasped everything I have been trying to say. All I can say is, yes, you're totally and completely wrong in saying that I have executed a ruse.
I tried to build a Shakespeare game. It stunk. You can't run social science experiments in a game that stinks. So, in pursuit of the goal I have had all along, I decided to make a game that does not suck complete ass. That required getting rid of Shakespeare. "The Bard has left the building" was not a ruse, it was an announcement of a policy change.
Posted by: Edward Castronova | Jun 10, 2008 at 08:27
Derek said:
Why has anyone not considered the heuristic validity of this? The population being studied is North American adults who play fantasy "on-line" computer games? The effect of the manipulated variable is pointless in such a niche group. Someone clarify what the sample population is here...
You are stating a hypothesis: "The effect of the manipulated variable is pointless in such a niche group." The experiment is meant to test precisely that hypothesis.
As Columbus sails westward, you're saying "This trip is stupid. Everyone knows the world is flat."
Posted by: Edward Castronova | Jun 10, 2008 at 08:32
"The effect of the manipulated variable is pointless in such a niche group. Someone clarify what the sample population is here..."
Moreover, it turns out that this is not quite as niche a group as you might think. We have work coming out soon that will take a sample of MMO gamers and compare it to US Census data. I'll post it here when it goes live, but in the mean time, suffice it to say that it's a broader group than expected.
Experiments and sampling all have to go through these validity steps. Arden II is just the first step, and it's not trying to do and prove everything all at once. It's going to take a sustained and systematic series of experiments and samples to write out the full validity play book. We know this.
Posted by: Dmitri Williams | Jun 10, 2008 at 12:53
cool idea. i recently read a book called "Super Crunchers" by Ayers [or 'the death of intuition']. one method discussed is very similar to your project. for example, a credit card company will send out a million envelopes, half with a picture on it, half without the pic. which one generates more interest? it'd be interesting, for example, to see WOW alter some simple things on two similar servers. i expect, like life, the results won't be different because it is a game ... how could they be?
Posted by: bill | Jul 02, 2008 at 13:54