Virtual economy: As real as real

Our first paper on the economics of EQII is now out in the current issue of the journal New Media & Society. There's a link to the paper on my research page.

We think the paper is notable because it is the first instance (as far as we know) of published, peer-reviewed, basic economic tests using actual large-scale data from a virtual world. No estimates, no samples, no bootstrapping--just all of the data, period. These data were anonymous and privacy protected, and what we analyzed were the macroeconomic trends and patterns. We have three main findings, below the fold:

First, the virtual world we studied appears to behave in the way a real economy does. The people there are as rational (or irrational) as we are offline. As a result, there are price indexes, an inflation rate, etc. This suggests some at least rough mapping is taking place for the world's economics, and that maybe, just maybe, these worlds might serve as testbeds for economic research and policy tests.

Second, the results were not particular to one server. A natural experiment occurred in which a new server came online, and it's economic indicators quickly approached and matched those of the existing ones. This suggests the powerful role of code in shaping and directing human behaviors in the aggregate. Another point scored for Lessig.

Third, the data give a much more accurate picture of the real-world value of the assets generated and translatable via RMT markets. Updating Ted's foundational work in the space, we now find that for this world at least, the numbers are lower than previously thought--about $130-160/year, or on par with Liberia and Congo.

The paper is the result of fantastic work by the PhD students at USC and Northwestern: Cuihua Shen, Yun Huang, Brian Keegan, Robby Ratan and Li Xiong. This crew is really something.

Citation:

Castronova, E., Williams, D., Huang, Y., Shen, C., Keegan, B., Ratan, R. (2009). As real as real? Macroeconomic behavior in a large-scale virtual world. New Media & Society. 11(5) p. 685-707.

Comments

[Noizy]

1.

I'm just wondering what effect of the Kingdom of Sky expansion that came out in February 2006 had on the economic data.

[Ailon]

2.

I wonder if you can predict real life events like like credit crisis using this economy simulation.

[Mitch Evans]

3.

I wonder if the QTM analysis would be different if the authors compared virtual currency with modern currency instead of pre-1971 dollars.

Also can economic models be accurate based on merely 7.17% of in-world transactions? If the remaining 93% of EQ2's transactions involve NPC merchants, then the game holds significant control over the money supply and the means of production. Does that kind of managed economy map well to the real world after the fall of the Soviet Union?

[Isaac Knowles]

4.

@Mitch:

The question would not be so much what percentage of the total transactions are made through consignment trade, but rather what the monetary value of that trade is relative to the value of all other forms of trade (i.e. with NPC merchants). All this controlling, of course, for things like barter (if at all possible).

The 93% is not terribly farfetched. Arguably, at least 50% of all transactions I make are with the government, because I pay taxes in various forms. Each transaction on which I have to pay sales tax is actually two transactions - one with the government and one with the merchant (notwithstanding the fact that, technically, the tax is on the merchant - not on me). An income tax assures that I have made a plethora of transactions with a central planner for a variety of goods and services - whether I want them or not. Yet government spending accounts for a (relatively) small amount of US GDP compared to consumption.

Trade with other players is 'taxed' once during consignment trade, but there is no real fiscal policy in EQ2, for obvious reasons. The player is simply paying for the service of having a centrally located intermediary to facilitate exchange. That is pure play, IMHO, plain and simple, and no more necessary than any other way in which the game might regulate the flow of money. In any event, the kind of monetary policy an MMO operator is capable of deploying will far more nimble.

[Mitch Evans]

5.

The sales tax/consignment analogy makes sense, but I disagree with the notion that government income taxes are the equivalent of NPC transactions in an MMO.

First, income taxes are not paired to any quid pro quo like NPC transactions. They relate to public benefits, but the state does not issue food, clothing, etc. in a direct exchange for your income tax payments. Nor do income tax payments remove money from the economy absent a government that retains a budget surplus.

By contrast, the 93% of EQ2's transactions are direct exchanges of goods to and from NPCs. Players either buy goods from NPC vendors or sell them back for gold. Moreover, the game operator never needs the gold captured through NPC vendors and effectively "prints" gold flowing out from NPC vendors into the game economy. Thus, unlike taxation, NPC trading is the faucet and sink for MMO gold.

Again, if 93% of the goods are exchanged with NPCs who can set prices arbitrarily and print as much currency as they wish, how does that model any real world economy?

[Isaac Knowles]

6.

The government is fully capable of setting prices arbitrarily and printing as much currency as it wishes, and many governments do both. I believe the authors even suggest that the data they collected may be best suited to look at extreme economic shocks, such as those brought on by extensive government intervention. And while one could argue that just because a government can print all the currency and intervene as much as it wants doesn't mean it will, the same must also be true - if not more so - for a game operator, whose customers can be quite fickle.

But that's not really the point. The point is that 93% is not so high relative to real-life, conditioned on the _hunch_ that the actual contribution to GDP made by transactions with NPCs is much less than 93%. Though transactions with government are ubiquitous, actual contributions of the government to the economy range from 19% in the US to 50% or more in Scandinavia, to practically zero in the third world.

[Someone Who Can Read]

7.

Let's be clear. This article purports to be written by 7 economists. On its second page it says that the US dollar is "assumed to have a value in gold backed by the US treasury". Since the Bretton Woods agreement in 1971, the US dollar has not been backed by gold. It has been backed by the full faith and credit of the US government.

So on the second page of their article these 7 people make a basic mistake illustrating a lack of fundamental knowledge of their purported subject matter.

I'd be curious why anyone thinks it's worth paying attention to anything else they say after such a demonstration of non-knowledge.

[Isaac Knowles]

8.

I have another question for the authors, though: If I sell junk to an NPC vendor, isn't that production that would be recognized as part of GDP? Does your data record income and expenditures by 'the government' on finshed goods and services?

[Ned]

9.

You know about EVE Online right, and the staff of economists and researchers that continuously evaluate their economy and put out Quarterly Economic Newsletters. Nearly the entirety of the EVE economy is completely player run over 66 regional markets...

[Isaac Knowles]

10.

@Someone Who Can Read (the first two pages):

There is nothing in their analysis or model to suggest that they are making that statement as an underlying assumption. I could easily read that line as making light of that fact that most people do assume (incorrectly) that their dollars are backed by gold. Chalk that one up to an observation on the ignorance of the masses, but not of the authors. (Also, they are not all economists).

The question at hand is whether or not economic aggregates in a virtual world behave similarly to economic aggregates in the real world in the response to policy shocks. While by no means definitive, the paper is a step forward.

[Mitch Evans]

11.

The sentence in question is:

"By examining transactions from a large commercial virtual world with
hundreds of thousands of players, the current mapping test concerns whether
the items and economic behaviors within a virtual world function in the
same way that they would in the real world – where, it is noted, currency is
also largely representational (a dollar is only a piece of paper, but is assumed
to have a value in gold backed by the US treasury)."

It suffers two flaws. First it "assumes" that real currency is backed by a fixed amount of hard assets (i.e., gold) which has not been true for almost 40 years.

Second, it sets up an analysis of economic activity of "hundreds of thousands of players" but then relies primarily on 7% of the total transactions from 2 out of 25 servers. If the population is evenly distributed, that means 7% of the activity of about 24,000 people.

As to apologies for the authors' non-economic backgrounds, they are authors of an economic study. Surely they can get a fundamental aspect of monetary analysis (no gold standard) right.

[Isaac K.]

12.

I really don't understand the fixation on what is clearly not an analytically relevant sentence in the paper, and I'm not apologizing for anyone's background. The fact is that the sentence you quote is orthogonal to the paper's findings.

No real-life measure of GDP includes transactions with oneself, but the number of these transactions may very well dwarf the number actually counted towards GDP. The mere fact that the authors can actually count those transactions gives their data an edge over that collected by, say, the BEA. But if they could, we'd probably be shocked by how few transactions made in the US are not actually counted towards GDP.

The paper is assailable on the grounds that the measure does not appear to exclude unfinished goods, and so their calculations may be guilty of "double counting". I also disagree with the exclusion of expenditures on merchant goods and services. But it has merits and it shouldn't be discounted just because, you know, you want it to be.

[Nice try blocking my email]

13.

@Isaac:

Let's try this for an analogy. You go to the doctor because you feel sick. The doctor says you have the flu. You ask how the doctor knows, and the doctor replies that you're nauseous, have a fever, and look like you have insufficient red bile. Someone asks you if you think your doctor's diagnosis was right. How are you supposed to know whether this doctor knows anything if they believe in the 4 humors?

If a comment about how the US dollar is based on gold was really orthogonal to the article, it would have been removed by one of the seven authors (assuming any of them even knows it's not true). Obviously all seven of them thought it was true and relevant, probably because they were relying on it to set forth their bigger point that just like virtual currency is supposedly virtual but actually backed up by something real, so too the US dollar is supposedly virtual but backed up by real gold. (And so much for the peer review process at this "journal" for not catching it.)

So I can see why you want to excuse a basic error of substance in an article whose substance you have an emotional commitment in finding to be a valid subject of research and whose conclusions you want to be noticed. You should be furious instead: if people keep making bush league errors like this in virtual worlds studies, the field will never get taken seriously enough to advance.

Now, since you're asking for someone to engage the substance of the article, here you go. There may have been real academic questions exposed by this study that the authors have completely missed. They missed them because they were looking to make another point that they couldn't support.

The article posits that because there are people in the world willing to pay a certain amount on IGE and eBay for virtual currency, that means the virtual currency has a mappable value to real-world currency. The article does not consider that the relative lack of knowledge people have of the ability to purchase virtual currency, combined with the active messaging done by publishers such as SOE discouraging players from purchasing it, may be affecting the purchase price. If SOE permitted direct real-world currency purchases then would that positively affect pricing? Negatively affect it? Be... dare I say it... orthogonal to the pricing? There are real-world underground economies analogies to this such as the markets for the prices for illegal drugs, where pricing might be affected when a product is moved from illegal to legal. How much does a college student pay for beer vs. someone over 21. The illicit nature of the transaction inside the virtual society might be affecting prices. No researcher will ever get the straight data on that though because SOE, like most publishers, is never going to let them know how much currency is actually trading hands outside approved channels. So these authors need to do some reading in related fields and develop hypotheses, then test them against the data they do have. I see a reference to Lankenau 2001 but for the limited purpose of differentiating this study from the prison economy studied there. Perhaps focus some time on comparative work. And then consider to the fact that EVE Online sells time cards for US dollars and also for ISK (in-game currency) which would allow a far more direct exchange rate to be established, and compare whether the EQ2 economy moves in the same way as the EVE one to determine whether the illicit nature of the currency purchase in the former game makes things different than the latter.

It's a shame to have been so close and to have missed it.

The authors missed it because they are looking to demonstrate that virtual worlds are perfect analogies for the real one. They may be, they may not. But they may show something more interesting: they may be useful to serve as a large-scale model for studying underground economies. And developing the first large-scale model to study those would be, well, worth gold.

Or maybe look at this: the economic model in a game like EQ2 might be skewed by the fact that the purchasable items are such a small proportion of the ecosystem and perhaps players opt out. Is economic activity perhaps a function of the type and nature of products available for commerce? To use a real-world analogy again, would consumption patterns in the Soviet Union have been the same or different if people could buy a house and car without permission from the government? Would Canadians pay more for health insurance if buying and selling private insurance for matters governed by the public system weren't a criminal offence? Or maybe players don't purchase things at consignment because they don't feel they need to, because they can get their own loot drops. An analogy could be drawn with the expansion across America in the 1800s, when settlers didn't need to purchase land if they didn't want to because they could keep going West, but purchasing land brought benefits like higher comfort and safety and proximity to others.

In an economy where things can be both found and bought, what effect does that have on pricing? And do virtual economies match this pattern? This would actually be an interesting study too.

Astonishingly and EVEN WITH ACCESS TO THE RAW DATA the article doesn't posit how many players on the server may have opted out of the market. The assertion that the "GDP" of this server was $130-164 may not be true if only 10% of the population is actively trading. Castronova might have been right the first time. And why wouldn't they be trading? Is it because they don't want to pay for the items? And if so is that because they're not desirable, or because they might drop sufficiently often as loot that they're not worth paying for? Or because there's a hedonic factor to trying to obtain the item as loot that wouldn't be met by a purchase?

Okay, I've made my point. Belabored it, maybe. But I guess what I'm saying is that where the authors make basic errors of knowledge in their field, seek to draw mysterious conclusions from their data, and miss more obvious and interesting questions that were right in front of their faces, I think I get to say that they did their field of study more harm than good here.

[Nice try blocking my email]

14.

Seriously. Blocking a domain because you don't like getting criticism? Weak.

[Dmitri Williams]

15.

FWIW, I'm not blocking any emails, and I'm reading all of this with interest.

[ecastronova]

16.

Let me try to address some of the complaints.

On the issue of "using pre-1971 dollars." Many people assume that the dollar is backed by gold. It is not. We write that the dollar is assumed to be backed by gold. This is true: Many people believe this. We did not write that WE assume the dollar is backed by gold. Mr. Knowles is correct in his interpretation of what we said.

Beyond that, if someone's entire criticism of a piece of writing is based on an assumed mental attitude of the authors, with the assumption drawn from one or two words in one sentence, the criticism is unlikely to be accurate.

Moving on.

On the issue of "using only 7% of transactions." The game's NPC merchants buy and sell things. When they buy things, they create money. When they sell things, they create loot. In this regard, there is no difference between NPC transactions and monster drops. Both systems make the developers the ultimate creators of goods, both money and items.

On a strict definition of GDP, sales to and purchases from NPC merchants are not relevant to aggregate production. Production relevant to GDP occurs only when a real individual creates something and then sells it to another real individual. One way of creating something is to buy it from an NPC merchant. Another way is to craft it. A third way is to generate it as a monster loot drop. These are all ways to creat an item. But the item doesn't count for GDP (in a strict real-world definition of how GDP is actually used) unless and until it is sold to another player.

Thus, the NPC "economy" has no direct bearing whatsoever on player markets. Player markets are unregulated by the owners and move regularly to free-market price equilibria. The equilibrium prices depend on the game's policies (such as drop rates, crafting systems, NPC prices, and so on) but they also depend on the interests and resources of real people. This is just like the real world, where the interests and resources of real people intersect with the "state of the world" - the availibility of resources and the technologies for producing goods - to produce real markets and real prices. The free markets of the real world operate exactly as the free markets of EQII. These markets are the subject of our study.

In general, and YMMV, but I find the line of criticism in this thread unpersuasive. As stated, the criticism suggests that we, as a group, have grotesquely erred in our approach to these data. It is said that we have made mistakes that no college sophomore would make. Perhaps, but I do not think so.

[Mitch Evans]

17.

"Yes. The fact that WoW's developers control money supply does not affect the movement of prices."

How can that possibly be true?

Does the QTM theory simply not apply in WoW?

Did you also study WoW and its markets? Or are you extrapolating that from your study of 7% of 2 servers in EQ2? The paper justifies studying EQ2 because, as a fantasy MMO, it represents 85% of the MMO space. Did you compare the economic mechanics of those games, or is the common presence of swords, elves and magic enough to bridge that gap?

[ecastronova]

18.

I am sorry - I posted a brief rejoinder before I realize how lengthy Mr. Evans' comments had become. In the above comment he is quoting my brief rejoinder. I was writing my longer response, which you may all read.

I will not, however, be reacting further to Mr. Evan's comments.

[Robert Bloomfield]

19.

As someone who studies laboratory economies for a living, I find the commentary on this paper almost surreal. I think these results are simply a lot less surprising to economists than to people in other fields, and that economists are likely to use game worlds to address very different questions than the ones being discussed here (and use very different worlds to address economic issues).

The standard way to do an economics experiment is to create a gamelike setting, but have the ultimate reward take the form of a laboratory currency that is converted into the local currency. You manipulate the setting or rules of the game (I am often altering the information available to participants or the regulations governing exchange), to test the relevant economic theory.

At this point, no fewer than 17 gazillion experiments have shown that the laws of supply and demand have darn good predictive power in the laboratory, and that arbitrage opportunities don't last long without good reason. So the question I ask is: why would they NOT hold in an MMO? Because of the fancy swords and animals? Irrelevant! Because of the large size? That should increase the power of economics.

And if these basic results seemed not to obtain, what would you conclude?

I see a couple of possibilities. The biggest one is that experimental economists work hard to make sure that lab rewards are convertible into good hard cash--whether or not it is backed by a gold standard. So it is helpful (though not shocking) to see that gamers view non-monetary rewards pretty much viewed like money.

The second open question is whether the researchers can actually understand the structure of the overall economy. In particular, if it seemed like the law of supply and demand wasn't working, or there was some persistent arbitrage opportunity, the researchers probably don't understand something basic about how the economy actually works. Again, this isn't a huge surprise that Ted, Dmitri et al. can do this (given that they are as charming, handsome and virile as they are knowledgeable about game environments), but it is still a noteworthy contribution, and I am glad to see this paper published.

I think the future of this type of research (taking a setting created for entertainment and testing economic theory) tells us much more about game settings than about economics, so I wouldn't expect to see a paper like this published in American Economic Review or another top econ academic journal. Instead, it will be pursued by people who will be surprised that economics can be a powerful tool for understanding behavior in settings that are typically studies with a different set of tools and theories.

If you want to test economic theory, you are going to need to create settings expressly designed to do that, or else get very lucky in natural experiments that arise (like the creation of a server, but with more surprising results).

[Isaac Knowles]

20.

Ted: I can see in general why transactions with NPCs are not included in the measure. Do you then count equipment repair as some sort of purchase for oneself rather than (say) a part of gross investment?

Robert: I had been writing a response that sounds an awful lot like what you wrote. However, I'm not sure I agree that only a world set up for the express purpose of testing theory is one that will be valuable to economics. Even simpler economies like WoW or EQ are capable of producing reams of data (at very low cost) that are of potential value to behavioral, labor, or IO economists. The main issue has always been, and remains, getting your hands on the information. Hopefully, more work with this data is in the pipeline that addresses these concerns. If not, it seems to me like there are significant arbitrage opportunities for the recent econ PhD to take advantage of.

[Abramth Asp]

21.

Why do you waste your time on the virtual economies, of online worlds that dont really deal with real money?

Any one here heard of Secondlife?

I make 50-100 US a month or more if I do side jobs.

[Andras Lukacs]

22.

While attempting to parallel game environment and the "real" economy, I cannot stop to think that exploitation and alienation is missing in virtual realms, while it is the backbone of any capitalist economic system.

[Robert Bloomfield/Beyers Sellers]

23.

Andras makes an interesting point. A key difference between real life and game worlds is that the player in a game has a relatively high-value alternative to not playing. (They can go to another game or go offline entirely). The alternative to not living is rather less attractive, allowing exploitation.

This provides a partial answer to Isaac's doubts about how important it is to look at worlds designed specifically to examine economics. Andras' point shows it is more specific: we need to think about exactly what we want to study, and then create a setting in which to study it (standard experimental practice). It is possible to go the other way, but you have to be really careful about what questions you can and can't answer... and I am afraid the list of 'can' is pretty short for topics of pressing interest to real-world policy makers.

p.s. @ Ambramth, congrats on your financial success in Second Life. SL allows some nice profit opportunities, but is a challenging environment for research. However, I would not that there is a LOT of exploitation there ... maybe there is a connection!

[Isaac Knowles]

24.

Robert said: "...I am afraid the list of 'can' is pretty short for topics of pressing interest to real-world policy makers."

Should this be the standard by which the value of VW economics research is judged? Certainly no other social science that has ventured into VWs has directed its research down such a singular path. One can pursue economics research for its own sake, or (what is less idealistic) for purposes other than those that are intended to directly benefit society (academic reputation, investment opportunities, etc.)

Despite the contributions by Ted, Dimitri, and a couple others, there is as yet no 'breakthrough' research in virtual economies. This would seem more a function of the luck of the researcher than of some intrinsic quality lacking in entertainment-geared VWs. What VWs offer is a clear and unambiguous data generating process; it is left only to find some way to make use of it in an interesting way. Researchers need the creativity and the resources - private, public, non-profit, whatever - to collect that data, as well as the means to identify all possible treatments.

Robert seeks a natural experiment; well what about an unannounced patch or hotfix on one shard/realm/server that is, for whatever reason, not necessary on another? This happens a lot at x-pac time.

Another use of resources is to purchase a large amount of RMT for any VW and find a way to flood the (internal) market with currency. This is a relatively cost-effective - and not (too terribly) unethical - way to create treatment effects. Directed appropriately, some novel discovery may be made.

Even if it ends up being a niche field (sport economics comes to mind), it doesn't take a policy-maker's utility function to make the research worthy of pursuit. The first paper to enter economics journals need not be a grand slam article in AER... but I could stand to see something JEBO :-p

[Robert Fuller]

25.

Excuse me if this has already been posted.

This is a great piece. One that will eventually lead to a better understanding of human social behavior. Point said this particular piece is extremely valuable in finding the ever so difficult percentage and overall affect that Black Markets have on an economic system.

[robusticus]

26.

Yeah well the data so far looks like those Black Markets have no effect whatsoever.

Unless you want to completely ignore the NPC economy, which, barring some sort of designer specification would be a better interpretation of the designer intent for value per item, if what an NPC pays is a fixed floor, as it is in EQ2.

It's absurd that one would interpret the intent to provide for arbitrage on vendor trash common items. This assumes a very low population. And it is even more absurd this would be accepted by "policymakers" as some sort of bizarre virtual tort on a non-existant player type. Though this is not as absurd as comparing a cross between a ski ball arcade and a bingo hall to a true market economy. That's what test 4 in the paper indicates. Do you suppose each server in SL would trace to the exact same economic footprint like EQ2 (and probably WoW) does?

The ego needs someone to blame for perpetuating a really bad design and farmers make a convenient target as they are reviled by players, unlike the designers themselves.

[Frank Lamancusa]

27.

Congratulations on your publication.

I would like to add that CCP, the maker of the MMO EVE Online, has an economist on staff who analyzes data for the entire game. He publishes quarterly reports on various economic topics.

His blog and links to his reports are at http://www.eveonline.com/devblog.asp?a=author&p=CCP%20Dr.EyjoG.

Start with "Move over, Greenspan" under Devblogs and progress chronologically.

[Nicholas A. Karlovich]

28.

Great paper guys, some very interesting stuff. If I may quote from the conclusion:

"In future research, it might be possible to modify real-world economic data-gathering so as to better measure the creation of value inside virtual worlds."

To me, this "creation of value" is the most important factor that was left out of the study, as the game does not allow the most valuable items to be traded or sold to other players, and these items are almost never sold to an NPC (with the exception of a rare bank cleaning).

However, if the data shows how many of these items (high-end raid items of the "no-drop", "soulbound", or similar variety)are created, estimations can be made about the true amount of value moving into the player population. The unfortunate thing about games like EQ, EQ2, and WoW is that the economy only exists as a stepping stone for players to advance to the items that are no longer a part of the that economy.

A large amount of the real money transactions purchase items in this "untradeble" category, precisely because in game currency cannot be used in transactions for characters. I understand this is beyond the scope of this paper however.

[Morgan Hardy]

29.

As far as the comments about, "Why not WoW or SL?" go, research is always constrained by available data. And SOE has been unusually giving, as far as the MMO developer space goes, in providing the data used for the study. It's the fruit of a relationship the various authors have been cultivating for years.

I'm sure if Blizzard turned over terabytes worth of data to people for research, you'd see results out of it.

[Peter]

30.

This sort of thing will be even more interesting to study in eve online after the Stock Market goes live. http://www.eveonline.com/news.asp?a=single&nid=3299&tid=7&sp_rid=MjgxNTYxMjEzMgS2&sp_mid=33968141