Standard Metrics of Use

I'm AFK for a couple weeks and I miss a crazy series of threads on SL. Well, as a sociologist/statistician, I'm going to avoid the politics here and simply state what I would like to see for universal metrics of virtual world use--something to get us in the ballpark of apples to apples. The intent, after all, is to have agreed-upon baseline numbers which you then argue about being indicative of something.

I'd like to have measures of these basic things for any given world:
Duration
Intensity
Economic activity
Multiplicity
Active users

Here are my potshots on what those would look like:

The first three are cummulative for all accounts/avatars owned/used.

Duration (could also be called "stickyness"): The amount of time the person has been a user of the world, expressed as a population mean with a standard deviation. This would be retroactive time from today, telescoping backwards. The user need not be on every day. Gaps between logins are allowable, with the maximum allowable time gap between logins being 90 days. Use example: "The average user of MegaWorld has a duration of 83 days." The standard deviation of the mean would help explain whether the typical person logged in once and quit or actually hugs the mean.

Intensity: Average hours per day per user. Sure, subject to all kinds of distortions for people who leave accounts logged on for various purposes (selling goods in Lineage 2, e.g.). Nevertheless, pretty useful as an indicator of bandwidth and interest levels and roughly comparable across worlds.

Economic activity A: A binary variable reporting whether the user does or does not pay for access to the virtual world.

Economic activity B: The average amount paid per user per week across the world population.

Multiplicity: Another population mean, an indicator of the number of accounts or avatars maintained by the user. This number would then be used for the previous three to obtain per-character averages. Note that the number could be either more than 1 or a fraction of 1 if the account is shared.

Lastly, I would like to see the term "active users" simply be anyone who has logged in to the world within the past 30 days. Not perfect, but simple and pretty good for an apples-to-apples measure across VWs.

With these numbers, you could calculate a range of use statistics for different users' needs. And, of course, having the raw numbers themselves (rather than means & SDs) to plot distributions would be even better.

I understand the incentives to supply, distort or hide these figures and I just don't care. I want the raw numbers based on these criteria (even if it makes Thomas think less of me ;) ). And sure, we're all subject to using what companies actually decide to disclose, but that's no reason to not spell out a better standard.

No doubt the excellent collection of minds perusing this site can improve on my starter list. The goal is to come up with some series of metrics that make sense across virtual worlds.

And if you want to pile on Clay, LL or anyone else, there are three other threads open that would love to hear from you.

Comments

[Prokofy Neva]

1.

Sounds good, but I'm not liking the concept of "economic activity" meaning only the payment to access the world. The economic activity in the world is important, too. I think those are actually the numbers to watch -- the "GDP and GNP" if you will of SL, how much economic activity within the world, how much is repatriated to real life, etc.

It seems LL does count transactions twice, and there is some funny stuff going on with their numbers on the statistics page. I've also always protested against their removal of land sales from "business" and "profitable" avatar lists. People selling land count as a business too. It's insane to cut out the main business of SL from those figures. They do that for ideological reasons, wishing to privilege what they think of as "compelling content".

[Matt Mihaly]

2.

How about picking a different phase from 'active users' to denote all logins within the last 30 days? It's pretty misleading to categorize someone who logged in for a few minutes and left, never to return, an 'active user.'

--matt

[Raph]

3.

So a typical MMORPG would show something like

Duration: 9mos (or whatever -- I have seen anything from 6 to 18 mos here, and I don't know that anyone is measuring this well right now)
Intensity: 2 1/2 hours
A: Yes
B: $15
Multiplicity: 2+ (yes, that's the average)
Active users: 90,000

So the thing this doesn't capture is that there's 100,000 paying. So I wouldn't toss that metric yet. :)

[Andy Havens]

4.

Some good thoughts previously here:

http://terranova.blogs.com/terra_nova/2006/01/100k_group_list.html

[csven]

5.

First off, thank you.

Second, with regards to Economic Activity B ("The average amount paid per user per week across the world population."), I'm wondering if you wouldn't mind explaining that a bit more.

[Thomas Malaby]

6.

Hehe, they all look like good starting points to me, Dmitri. After all, I said here that "more data (of all kinds) is helpful". Of course, the devil will be in understanding what these numbers mean (as always), but that's the fun part ;-).

[Dmitri Williams]

7.

@Prokofy. We can come up with another label, but my intent is a measure of money given by users to creators, not actual activity within the world. That'd be a separate metric, and one I wasn't trying to tackle (yet).

@Raph. Quite right, and I wasn't clear in stating that I'd want not just the binary paying/not paying, but the resulting % from the population. This then lets you calculate the gross income as well as the number of payers vs. non-payers (plus some other fun stuff).

It'd be great to see a side-by-side with those kind of numbers, wouldn't it?

@csven. Just some time-based metric of money paid to the game maker/hoster. Given smaller units of time, you can extrapolate upwards, but you can't gain granularity going in the other direction.

@Andy. Right also. I just thought it was time to goose the subject given the, err, intense interest in the SL/Shirky threads. And the recent activity in the dreaded back channel.

@Thomas. If I get numbers, you are one of the guys I ask about for their meaning. Triangulation FTW.

@Matt. OK. If you don't like "active users," provide some alternatives. I think it's fine as it doesn't imply *how* active, but if you can do better, great.

[Prokofy Neva]

8.

We can come up with another label, but my intent is a measure of money given by users to creators,

Well, here's where you are coming at it with the game/MMORPG bias, assuming all creation and content is in the hands of game gods, and payments to them in RL outside the arch of the game are the only metric that counts.

But the creators are inside the world in SL too because it's user-content. The sales in world are largely of user-made content; the game-gods only sell the land.

[Allen Sligar]

9.

This is a good start, I'll come back to this later in the weekend but it's important to understand that taken alone, those data points could be rendered useless. Meaning regression to the mean, and/or sampling issues dependant/indepandant etc.

What it takes in actuality is multiple data points trended over a period, this is hard to expalin with brevity. However, what I'm getting at is using your base data points, and using associted data points, and capturing data related to an initial, and growing population over time would give you the ability to trend your analysis of these particular metrics.

The eventual goal or rather outcome would enable predictive modeling of the population under consideration as it related to the particular data points youve specified.

You could, in fact, predictively model many things related to your subject depending on the granularity of your data points as well as historical data.

I'll revisit this thread later, and get into more depth related to what you've proposed Dimitri, mainly because I'm looking for input on standardization of dimensions, measures and data points from the Academic's and Dev's but I need to get sleep right now....7 days until we start beta, times currently at a premium for me :)

[Clay Shirky]

10.

@Dmitri,

How about "Visitors and users" for people who've logged in at least once in the last 30 days, and "Active Users" for people who have logged in in the last 30 days who also logged in the previous 30 days? (This is close to what IAmEric was suggesting in a previous thread, minus the hourly count per user.)

[[email protected]]

11.

I think a big problem here is that people are focusing far too much on people numbers.

Nobody will provide accurate published numbers, simply because it's in no one's best interest to do so.

Not SL, WOW, EQ .. anyone.

If I wanted to, I could tell you all the lies that are being told in the drilled down numbers (how islands built are including void islands, how the transactions are meaningless, how I've ran across herds of automated libsecondlife bots on at one time, etc etc).

... but then I can also tell you that I make more than 8000 USD seling virtual goods.

But then the same could be said in WoW .. how much of that concurrency is people farming gold or plat or whatever it is?

The fact is, only the highly technical among us really understand what's going on, and it ain't pretty.

What makes SL or WoW or whatever compelling will never be the numbers, because the numbers will always lie.

What makes them compelling is what they provide. How I can go in world and collaboratively study and build complex systems with friends, students, and customers. How we can interact in a deeply immersive way, in a way that is going to only get more immersive every day.

There was a great post awhile back on Terra Nova. It was about group IM chat. This was very insightful and in many ways, quite brilliants, because groups in SL are the government .. are the over arching social structure .. especially now that forums are gone.

Study this and I think you'll get a lot more insight into the sociology of virtual worlds.

Study the numbers and you'll just be counting the number of angles dancing on the head of a pin.

[Clay Shirky]

12.

If I wanted to, I could tell you all the lies that are being told in the drilled down numbers (how islands built are including void islands, how the transactions are meaningless, how I've ran across herds of automated libsecondlife bots on at one time, etc etc).

Yes please -- tell us that, or point us to other places telling us that.

Study the numbers and you'll just be counting the number of angles dancing on the head of a pin.

Blaze, the 'angels on the head of a pin' reference is inapt, because, unlike the number of unique users in Second Life, no one knows the answer about angels. Second Life calculates the number of unique users daily for it's economic stats, but only publishes the number with positive balances -- it could easily answer the more general question if it wanted to.

And the idea of doing research on sophisticated sociological problems without knowing how many people you are looking at is simply daft. Size matters -- villages are different kinds of things than towns are different kinds of things than cities. You have to understand the prevalence and density of social options, like "how many people are there to talk to or collaborate with or trade with?", in order to even begin thinking about second order effects. Economies or governance structures can't be meaningfully looked at without answering the basic demographic questions, which is what Dmitri is trying to do.

[Thomas Malaby]

13.

Clay wrote:

the idea of doing research on sophisticated sociological problems without knowing how many people you are looking at is simply daft

And it is just as daft to believe that you can confidently interpret those numbers without understanding things that don't always show up within them, Clay. It's a two-way street, as Dmitri and I understand it. One of the things that means is that micro-level qualitative (and quantitative) studies are just as essential to the long-term effort to develop our understanding of these spaces.

Yes, these numbers would be cool to have, and it would be a lot of fun to how to adjust (in terms of the formal characteristics) and interpret the stats Dmitri suggests in order to *begin* to attempt meaningful comparisons across virtual worlds. As I said in the other thread, the numbers we would need would have to be new ones, not the old categories that have been worked from (and worked over) in the past. Your dismissal of Blaze's comments and your other comments bespeak to me a certain lack of interest in the broad and open-ended inquiry in which we are engaged.

I'm not sure you will respond to my comments here, since you did not deign to in the other thread, but I continue to wonder why we should be interested in your point of view if it is so uninterested in nuance and a balance between the qualitative and quantitative, between meaning and materiality. The ongoing effort to understand SL and other VWs is a process that requires a lot of hard and new conceptual work from people on all sides. Part of what this means is that banging the cheap drum of "Linden knows but won't tell us" starts to sound like a message that is meant to play well (especially since your criticism was always more powerful, imho, as a critique of the MSM). This is especially true given the fact that it's pretty clear to most that are familiar with SL and LL that Linden Lab quite pointedly is *not* in a position of total understanding of what it has wrought, and therefore even if we knew everything they know (which would be nice), we still would be a long way from the whole story.

[IAmEric]

14.

Hi Dmitri,

Thanks for taking the question seriously and getting some real concrete suggestions started. If the tasks was to come up with some apples to apples numbers for comparison across virtual worlds, this is a great start.

I'm still relatively new to this area, but I am more interested in Prokofy's thoughts. I'm not so much interested in the economics of the game from the perspective of an outside observer trying to compute say the impact of virtual worlds on the GDP of the host country or countries. I am more interested in things along the lines of Castronova's original papers regarding the economies within the game.

You said you haven't tried to tackle this problem (yet), so I am very much looking forward to hearing what you have to say when you do.

Perhaps jumping the gun a little bit, to study the economies within these virtual worlds, it makes sense to me to treat them as if they truly were newly discovered economies. What would a virtual census burough do to construct a meaningful number for the population of the virtual world?

For example, I wish more people who are better at economics than I am could reproduce this type of work but for various virtual economies. The following was taking from an article in Walrus Magazine a while back. I can't find the original article because the link I have is no longer active, but I saved this quote:

"Things got even more interesting when Castronova learned about the "player auctions." EverQuest players would sometimes tire of the game, and decide to sell off their characters or virtual possessions at an on-line auction site such as eBay. When Castronova checked the auction sites, he saw that a Belt of the Great Turtle or a Robe of Primordial Waters might fetch forty dollars; powerful characters would go for several hundred or more. And sometimes people would sell off 500,000-fold bags of platinum pieces for as much as $1,000.

As Castronova stared at the auction listings, he recognized with a shock what he was looking at. It was a form of currency trading. Each item had a value in virtual "platinum pieces"; when it was sold on eBay, someone was paying cold hard American cash for it. That meant the platinum piece was worth something in real currency. EverQuest's economy actually had real-world value.

He began calculating frantically. He gathered data on 616 auctions, observing how much each item sold for in U.S. dollars. When he averaged the results, he was stunned to discover that the EverQuest platinum piece was worth about one cent U.S. — higher than the Japanese yen or the Italian lira. With that information, he could figure out how fast the EverQuest economy was growing. Since players were killing monsters or skinning bunnies every day, they were, in effect, creating wealth. Crunching more numbers, Castronova found that the average player was generating 319 platinum pieces each hour he or she was in the game — the equivalent of $3.42 (U.S.) per hour. "That's higher than the minimum wage in most countries," he marvelled.

Then he performed one final analysis: The Gross National Product of EverQuest, measured by how much wealth all the players together created in a single year inside the game. It turned out to be $2,266 U.S. per capita. By World Bank rankings, that made EverQuest richer than India, Bulgaria, or China, and nearly as wealthy as Russia.

It was the seventy-seventh richest country in the world. And it didn't even exist.
Castronova sat back in his chair in his cramped home office, and the weird enormity of his findings dawned on him. Many economists define their careers by studying a country. He had discovered one."

I consider Castronova's work to be ground breaking, but I am a little perplexed why he hasn't become more involved with Second Life. Are you around Professor Castronova? :)

The economy of Second Life doesn't need discovering. It was there front and center all along.

In the case of EverQuest, it seems like the economy emerged as a byproduct. Is that true? How is it now? Are there other virtual worlds where the virtual economy is an important aspect of the "game"?

Dmitri, your ideas represent a great start, but in the end, I think to compare virtual economies, we should start taking the economies seriously. How do you compare real economies? Perhaps we should just try to compute GDP, inflation, unemployment rate, wages, etc.

Is it farfetched to view Second Life and other virtual worlds with thriving virtual economies as "emerging markets"?

Best regards,
Eric

[Clay Shirky]

15.

@Thomas,

No dis intended in the other thread, I stopped reading it in favor of later ones, and only just now noticed that it was still ongoing.

You say "And it is just as daft to believe that you can confidently interpret those numbers without understanding things that don't always show up within them, Clay."

I don't disagree. I'm certainly not saying that once we know the population, all our questions will be answered. I am however saying that none of the nuanced questions will be terribly well grounded if we don't have population figures as a start.

[Prokofy Neva]

16.

Eric,

Having read the work of Castronova over the years, as you cite, I've constantly been perplexed why he never tackled the economy of Second Life; I really wish he had. It seemed like a perfect match. I've asked that question publicly in forums a dozen times, including here. Curiously, he's never answered. Oh, I know he's been cited as a talking head on this or that general article on Second Life where he provides the requisite sound-byte about how lots of people are online these days and some of them monetarize their time on line.

But he's never visible inworld giving public lectures, etc. (though his university's program has various activities in the Kurrian Expedition) and he's never posting or commenting on this amazing synthetic economy -- you think he would.

Now, how to explain that? Traditionally around things SL, one can explain these silences by possibly 1) an NDA 2) working on a competitor for SL still under wraps 3) work on a non-public alt or 4) simply having concluded that SL isn't a game, and even its synthetic world status is questionable (i.e. as was stated above there may only be tokens standing in for a real-world gambling economy). I have no idea if any of these apply, but I have to conclude that something about SL is either very compelling, or not compelling enough for Castronova, and that's an indicator all its own.

Clay, I can only keep repeating as others have repeated: 36,000 paid subscriptions, which support some 100,000 regular log-ons. That's never enough for you, but that's what it is.

The news of breaking the barrier to reach 20,000 concommitant log-ons were announced yesterday mainly to disdain from all the SL-related blogs, who remarked that if SL was down half a dozen times in that day due to asset server problems, these were mainly 20,000 cranky customers.

blaze, I've seen the herds of bots, too, but since only 40 or 50 can be logged on a mainland or island sim at a time, I don't think this is a significant dent in the numbers -- alts are way more significant. Libsecondlife has had a program for months, related to Copybot, which can log on multiple bots and run them through IMs. These clone drones are flooding camp-chair establishments and training them of thousands of Linden dollars by ordering the bots to sit and occupy all the chairs to get the camp pay-out, but then not play the casinos. The camp-chair kings are howling, and wondering why LL doesn't shut this down as an exploit -- though casinos with camping are hated by a vocal minority of tekkies and oldbies, still, botting the owners is theft, if they are drained of all their money.

There are also groups like ESC logging on herds of bots to test them for various things, and of course I at the Herald have asked the pointed question whether the non-copyable sheep avatars on all these bots were paid for, and gotten the not-completely-satisfying answer that yes, they are (but then, the avatar's creator is under an NDA).

NDAs explain a lot of the mysteries of our world, as I've commented:
http://secondthoughts.typepad.com/second_thoughts/2006/11/nondisclosure.html

I don't *at all* buy this concept of the scientists at the Lab having created a monster they can't track or understand. They just aren't talking. And you wouldn't either, if you had invented a thingie that was going to make you and your friends a bundle.

LL tracks how many objects are created constantly; remember the ticker they used to run on the front page "last 10 things purchased in world" that they finally removed because it was endlessly showing gambling objects like "Skunk Money"? Run that program, and you could tell lots and lots about the nature of the economy and whether the figure of the US $ spent is just washing in and out of casino objects.

LL now has the ability to remove an object from the world at one fell swoop on multiple sims when a griefer puts it out (or maybe they always had that but didn't use it or couldn't use it). That means they have (or had) the ability to track any object inworld across all sims whenever they wish.

When I asked Philip once -- to get past the gamed traffic numbers -- to indicate what the parcels were that people actually picked and put on their "Picks," he had the database query and numbers within a day or two, showing a different "top 20" than the gamed "top 20". I know that all the Lindens are fascinated with statistics, and put up as many as they can and still remain a private corporation -- or so it seems. It's never enough, however,

If we're going to make up wish lists of numbers, I'd really like to have a number that shows people who log-off *but come back after longer periods*. This is the most interesting number to me. As noted, I have a list of tenants each month who return after long absences from SL. It's a handful, but a growing handful because they must read about SL and decide that their tryme or even lapsed premium account they never fully cancelled can be reactivated. While some percent log-off due to poor performance, I would say most leave because of a bad relationship or a move in RL to a country or situation where they can't get DSL or log-on. So a figure that lit up with the emigrants who return would be interesting to me to see -- log-ons after 6 months or 12 months. And there's no reason that couldn't be tracked to see if it offsets any of that 90 percent or whatever the large figure is that are looky-loos who then leave.

The Herald and Tateru have reported on the waves that occur -- ebbing and flowing. These waves are tied to SL's performance, the patch days, news stories, etc. Some days you feel as if the entire place is empty. Others you put out an ad and 36 people arrive to look at a rental within an hour, and you can't possibly wait on them all. There is definitely always a huge ebb and flow, and I guess what is important is that each wave does leave behind more starfish on the beach.

[Thomas Malaby]

17.

@Clay: That's fine put that way, of course, but you put things more strongly frequently. Such as when you say,

You have to understand the prevalence and density of social options, like "how many people are there to talk to or collaborate with or trade with?", in order to even begin thinking about second order effects.

This is not really true, if by this you're suggesting that inquiry can't even begin until the broad-level demographic questions are answered. The point I'm trying to make sure you're on board with (and frankly, I've seen little indication of it) is that the numbers do not precede other kinds of understanding, such as what Blaze was pointing to. These are all ingredients of an ongoing inquiry, so I would ask that you shy away from the rhetoric which suggests that we can't say anything until we understand everything from a numerical point of view. That's impossible.

[Dmitri Williams]

18.

@Clay. There are a lot of levels of nuance and granularity we could suggest, but there's always a tradeoff. At one end of the spectrum you get something pretty abstract and broad and of minimal use. At the other end you get a perfect detailed replica. Or, as Steven Wright used to intone: At home I have a map of the United States. . . Actual size . . . At the bottom it says 'One mile equals one mile.' So I'd stick with the 30 days and "active" since that's a billing standard and might actually see the light of day. Plus, if you have the duration numbers (mean and SD) and intensity, you can make some qualitative inferences about activity stretching past that 30 days.

Alan is right to suggest repeated measures, and anyone can be right to suggest more depth to any one of the measures. Personally, I'd just like a massive data dump of everything ever, which I'd then use to calculate trends via time series analysis and use hierarchical linear models to look at groups compared to individuals and multi level models over time. But that'd be a bit much for the quasi-experts who need a level of understanding that is broad, but still a bit detailed enough the get some nuance. And it'd be unrealistic to ask press or even most investors to understand most of that. They'd just ask statisticians to process the numbers and spit out some coarser stat anyway.

@blaze. I disagree. Numbers matter for scope and scale. A census is a platform upon which everything else gets built. Once you establish it, you can then drill down and know how the people or situations you are looking at are or aren't representative of the whole or some portion of the whole. If Thomas or you or I identify some interesting new phenomenon (Gor groups, RMT, cybering, VoIP, guild dynamics, etc., etc., etc.), it can then be studied both in depth and context, but also with an understanding of how common it is. Meaning is of course important, but when it's only .01% of a population it's well, less meaningful to me. And media research tells us that this sense of proportion is one of the things the press is the worst about in most news formats.

For the researcher or analyst, this is also about allocating research resources. If you want to understand as much as possible, you devote a large chunk toward the largest pools of phenomena and then keep several moderate chunks for rare and emerging phenomena. Imagine a corporation investing in movies. They'd want to diversify by going both the big-studio route, but also investing in some Miramax-level and Sundance stuff. You spread, but spread based on baseline information.

@Prokofy. I'm after game-exogenous money, not internal. Do I think that internal money transactions and user creations are important and sociologically interesting? Yes, and I am actively engaged in research projects measuring them. But it's simply a different topic for a different post. A base statistic of who pays how much for access is worthwhile for a range of people. If you want to propose some universal measure for in-game economic generation, trade or arbitrage, go for it. You're probably in a good position to take your own starter pot shot.

@Eric. We (Ted, Dan, Nick and myself) have this kind of very detailed data on a popular MMO and are working toward getting it usable. The good news and bad news about modeling an entire economy is that the data volume is massive. We're talking about terrabyte of data on transactions, and that is tough to instrument and to query. It's taken a full-time programmer and the assistance of the National Center for Supercomputing Applications here at Illinois about 5 months to even get the data into an analyzable format. We're about a month and half away from having it ready to use, and then some time after that to crunch numbers and make sense of what we find. But again, that is a topic for a different post. Maybe you should be emailing Prokofy and working up a system and a post for us to host . . .

[IAmEric]

19.

Hi Dmitri,

It's great to know that people at my alma mater are thinking about this stuff. Go Illini! I got my doctorate on the other side of Green St though :)

Will you make that data available once it is digestable? One of the reasons I am so interested in virtual economies is that they provide the kind of experiments whose absence constitutes what von Neumann claimed made the study of economics so difficult. I would love to have that kind of data for SL. If you are interested in collaborating, there is a chance I may be able to make some things happen on my end.

Best regards,
Eric

[Dmitri Williams]

20.

It isn't SL data, and the idea is that long-term, it'd be made available to a broader research community once we take a good whack at it and make sure we comply with an NDA. It's going to be a little while, though. You're welcome to ping me off-TN.

And no worries, I do work on both sides of Green St. ;)

[Clay Shirky]

21.

@Prok:

Clay, I can only keep repeating as others have repeated: 36,000 paid subscriptions, which support some 100,000 regular log-ons. That's never enough for you, but that's what it is.

It's not enough for me, because neither metric counts people. I wonder why you are willing to accept such unrepresentative data.

[Clay Shirky]

22.

@Thomas, since my tone seems to grate, I'll let Dmitri post for me:

Numbers matter for scope and scale. A census is a platform upon which everything else gets built. Once you establish it, you can then drill down and know how the people or situations you are looking at are or aren't representative of the whole or some portion of the whole.

This is what I believe.

@Dmitri, re: "So I'd stick with the 30 days and "active" since that's a billing standard and might actually see the light of day."

Point taken. As with Matt and Eric, I am interested in numbers that discount the 'storm surge' effect, but since that can't last long, the 30 day number will settle down over time.

[Dmitri Williams]

23.

Right, and if you were actually getting that 30-day number regularly, you'd get the surge numbers and trends. In my fantasy world of data, we see these numbers monthly.

[Thomas Malaby]

24.

@Clay: Again, that's fine, but if that's all you believe then you haven't been reading Dmitri carefully. You seem to take this idea about numbers as grounds for generalization as a reason to stop the presses until they are determined once and for all. This suggestion that this is what needs to be done first saturates your claims here, but not Dmitri's. He knows that all of this work needs to be done in tandem; that's what process means. To say otherwise to score rhetorical points is disingenuous.

What is more (and here Dmitri and I sometimes end up quibblinga little over language), claims about "some portion of the whole" can be made without extensive statistics. If this interests you, then I invite you to see my post to TN to come.

[csven]

25.

@csven. Just some time-based metric of money paid to the game maker/hoster. Given smaller units of time, you can extrapolate upwards, but you can't gain granularity going in the other direction.

And I'd argue that one can't adequately discuss virtual worlds on an "apples-to-apples" basis without looking beyond the standard game developer model. Eric's quote from Castronova's study should remind us all that what was so startling to many people *wasn't* what the developer was pulling in, but what was happening among the users outside the space. Consequently, I agree with others here that the economic numbers need to reflect something other than simply what the developer is collecting. If it doesn't, I don't see how this will ever begin to approach an apples-to-apples system.

My suggestion for economic activity would be something more like this:

Economic activity A (Access Gatekeeper):

1) A binary variable reporting whether the user does or does not pay for access to the virtual world.

2) The average amount paid to the Access Gatekeeper per user per week across the world's population.

Economic activity B (World Provider):

1) The average amount paid to the World Provider per user per week across the world's population.

Economic activity C (Real World Developer Centric):

1) The average amount paid to a Developer outside the virtual world per user per week across the world population.

Economic activity D (In-world Developer Centric):

1) The average amount paid to a Developer inside the virtual world per user per week across the world population.

Economic activity E (User Centric):

1) The average amount paid to anyone for anything related to the virtual space per user per week across the world population.

-

{Note 1: The Access Gatekeeper, World Provider and Developer may all the be same entity. I'm considering here that a virtual world might be accessed through a large umbrella gatekeeper (Multiverse 2 or AT&T Worlds or something), provided to users by a separate World Provider who both contracts with the Access Gatekeeper to be a part of the network and with a Real World Developer to build the underlying world content.}

{Note 2: A Developer can be a high-level entity contracted to build the underlying world assets for the World Provider, a low-level entity creating content specifically for users inside the virtual space (e.g. There Developer Program), or anything in-between. The differentiator is in which "space" the economic exchange takes place: in the Real World using traditional currencies or in the Virtual World using the embedded currency system.}

-

Tear it up.

[randolfe_]

26.

There is no legitimate notion of GDP, GNP, NIA, current accounts, deficits, or even real rates or PPP within any of the current generation of VWs/MMOGs.

How does one measure monetary metrics? I've read most but not all the various academic notes on VW economies (as of summer 06), and find little of it convincing.

These "economies" aren't bona fide economies at all. They are, at best, quasi economic abstractions. They more resemble parasites (some might say symbiotic ones) upon the real-world economy, not unlike the "drug economy". The drug economy is not an economy. It contributes nothing to the Solow model (or whatever your particular choice of doctrine) independent of the host economy. It merely distorts the host economy's national income accounting. The same is true of SecondLife et. al.

In fact, all that really matters is the definition(s) offered by Dmitri.

I also don't care to know about the "GDP" of Harrah's. The fact that people are exchanging some kind of money-for-service-entertainment therein doesn't mean that they are generating economic activity deserving to be studied as an economy. All it means is that Harrah's has a P&L.

Last I checked the SL "virtual world" still falls squarely under the sovereignty of the US for both it's corporate operations and a vast majority of it's customers activities. States have a SNP because they enjoy some form of limited sovereignty. They represent legitimate economies, even if they are sub-economies. But SL enjoys no sovereignty, and is not a sub-economy any more than the drug market.

Arguments about VWs like SL somehow representing transnational entities, therefore granting them some kind of special consideration are also not convincing. Modern transnational corporations have existed for over 40 years now. Although some sovereignty/jurisdiction issues remain, by and large these things are old hat. The only reason disciples of VWs think they are different is because the "powers that be" largely do not yet understand what VWs are. Once they do -- probably fairly soon -- they'll be treated no different than any other multi jurisdictional business. It took them a while to catch onto internet poker too.

Better that we just call these things games, measure them as games, and quit pretending they are something greater.

[Clay Shirky]

27.

@Thomas,

You seem to take this idea about numbers as grounds for generalization as a reason to stop the presses until they are determined once and for all.

Not at all. I'd settle for them being determined once.

[Thomas Malaby]

28.

Well, at least now we understand each other.

[Andy Havens]

29.

@Clay, who said: "I am however saying that none of the nuanced questions will be terribly well grounded if we don't have population figures as a start."

I must strongly disagree.

I've been involved in all kinds of marketing research where we weren't able to know raw or absolute "population" metrics (especially for competitors), and you can do insane amounts of good work that doesn't require an understanding of the total size of your audience. And you certainly don't need to know the size of all the audiences in all the competitors in a space in order to compare and analyze all kinds of qualitative AND quantitative aspects of their offerings.

There's brand and name recognition testing. There's "decision process" testing to determine how/when customers move through the pipeline. There's test marketing of any game/world feature (current or planned) using split-run tests and small customer sample groups. There's concept testing and mystery shopping of both operational and customer service capabilities. There's user acceptance testing, UI testing, customer lifecycle research. All kinds of price testing including elasticity, sell-through, coop possibilities, reseller acceptance, affiliate tests, win-back programs, etc. There's segmentation research to see the demo/psychographics of different service's customers, regardless of total numbers (which is another issue in-and-of itself).

None of this -- none of it -- requires gross numbers. Unless all you want to do is make a specific judgment about, "If X number of people use a service, it must be linearly X important. If Y service has 1/100th the users of X, it must be 1% as important."

Which, again, I must disagree with.

You've said a couple times now that you aren't interested in the "guts" of SL. The experience. We know you want to know about the raw pop numbers of SL. Lots of this other stuff we're talking about here -- even many of these numerics -- are nuance, and will require an understanding of the guts.

Because the differences between WoW and SL involve those "inside experiential thingamabobs" that you don't want to deal with. And that includes the fact that WoW doesn't allow for a monetarily free experience that does, however, add value to the overall platform.

So, if you get an "apples-to-apples" paid account comparison in WoW and SL, what you end up with is one of two things:

1. Either a complete dismissal of SL (compared to WoW), because 36,000 paid subscriptions is so much bloody lower than 7 million paid subscriptions. Or;

2. A need to now "ask the nuanced questions" (what I've been calling "context" for the last week or so) that will explain why the press, and many of the foine folks here at TN are so interested in SL, despite the low (compared to WoW) numbers of (paid) accounts.

I pick #2.

[IAmEric]

30.

Hi Randolf,

If you take a snapshot of the world today, it is difficult to disagree with you. No one could debate your statement: "There is no legitimate notion of GDP, GNP, NIA, current accounts, deficits, or even real rates or PPP within any of the current generation of VWs/MMOGs."

Does that render the concepts meaningless? If we extrapolate (on a log scale) the past 5-10 years into the next 5-10 years, is it really so inconceivable to think that virtual economics as a legitimate field of study will emerge?

Maybe I've drunk the virtual koolaid, but I can see no outcome other than massive growth of virtual worlds in the coming years. Virtual economic transactions will soon rival that of some countries. The central bank of China is concerned that virtual currencies will undermine the value of the Yuan. That is a pretty powerful thought.

I'm suggesting that we study the economies of virtual worlds as true legitimate economies. It is certainly fair to criticize the idea, but the field seems very fertile to do some really good work. If I were still in academics, I'd be all over it.

The comparison of virtual economies to black economies is a fair one, but I don't think it negates the idea. The drug economy is an economy that distorts the real economy in ways similar to the way virtual economies will distort real economies. Economists will eventually love to have a handle on both.

It is not necessarily the right thing to read the literature when we should be producing it.

Eric

[Allen Sligar]

31.

I'm back, no longer sleep deprived and well caffeinated. I'm even going to use my spell checker today because I'm feeling frisky.

"Numbers matter for scope and scale. A census is a platform upon which everything else gets built. Once you establish it, you can then drill down and know how the people or situations you are looking at are or aren't representative of the whole or some portion of the whole."

This in essence represents our goal in building a benchmark data set for the video game industry. I haven't gone public with what we're doing in its entirety, although a few developers we're sent an email recently, although I only got 2 responses, :( that had more detail.

@Dmitri: In reference to this:

"Alan is right to suggest repeated measures, and anyone can be right to suggest more depth to any one of the measures... And it'd be unrealistic to ask press or even most investors to understand most of that. They'd just ask statisticians to process the numbers and spit out some coarser stat anyway."

I'll go into how to apply what you’re after, in various levels of utility and granularity depending on the end user needs, later but let’s deal with your proposal now:
-------------------------------------------------------
"I would like to see for universal metrics of virtual world use--something to get us in the ballpark of apples to apples. The intent, after all, is to have agreed-upon baseline numbers which you then argue about being indicative of something."

I'd like to have measures of these basic things for any given world:
Duration
Intensity
Economic activity
Multiplicity
Active users
-------------------------------------------------------

From a less academic and more utilitarian data mining perspective (because that’s what I do) when beginning to tackle this issue I realized the industry lacks standard measures and dimensions. The underlying reason is related to a fundamental lack of a descriptive well defined language. Various academics lurking around here have made some great progress, but let us compare the ability to data mine other industries for actionable data sets for a moment:

Health Care, (descriptive, definitive language, allows for the extrapolation of things like incidence of disease, when cross referenced with demographic information, disease management when cross referenced against financial cost metrics, and predictive modeling for increased ROI on disease management when coupled with programmatic interventions)

Petroleum, (same as above, geography, satellite imaging, drilling technology, allow for understanding, measurement, management and interventions which increase ROI)

Retail, (same as above, demographics, purchase patterns, production and distribution costs etc)

But what of the Video Game industry and VW space?

Where are the standardized metrics? From whence do they issue? My conclusion is we must start with the basics first. Let’s understand that we know that we have an apple tree before picking the fruit....as it were

After all before we can get to higher strata of analysis, to find the nuance, we must first understand the basics.

The first order of definition is demographics of a valid sample size, I'll avoid the debate related to sampling size and validity for now, but go from the assumption that I'm fairly conservative in this.

Demographic strata are fairly easy to lay out; in roughly 15 minutes one can define what ones needs to measure in people. 25 data point suffice.

Now we can get into:

Duration
Intensity
Economic activity
Multiplicity
Active users

Duration:

Without reliance on data from a company (which is limiting and only measures duration of 1 population against 1 space) let us examine duration from a macro level. This will be difficult initially, yes, but constructed correctly provide greater insight later on, and be less limiting.

What I mean here is to get overall measures of market duration for this population segment we'll be finding what their duration was previously, and measure it going forward. The only subjectivity involved is that of population self identification with a given value set. Aka bias.

However, this metric becomes more powerful when we look at intensity as you've defined. It becomes telling when measured against things like churn rate, platform, and type of space.

We are limited to softer numbers here unless we get actual data dumps from companies, have a dashboard that allows for monitoring, like Business Objects, or use SAS, (this assumes they've bothered building measurement mechanisms) or build an integrated software platform (Decision Support System) for companies and that is currently underway actually....because hey why not have our macro analysis (cake) and micro analysis (and eat it too) as well :)

Intensity is a fairly straight forward and soft target to capture, what is elusive is capturing the nature of that intensity, this I've defined internally as "Interactivity", or rather "What we're they doing or what do they do most of the time" there can be A LOT of granularity captured in this area, and I'm certain this will be of interest to designers in particular, for various reasons. I've developed quite a few measures in this area, admittedly I am not an expert, there’s likely 3x more that I have not thought of, we'll get to them as we can and based on input....and maybe a few people will stop by with some helpful suggestions :)

I’ve actually set up a channel (forums) for direct input and suggestions for this purpose from closed beta onward, feel free to drop me a line.

Economic Activity A and B: Here I also believe inclusion of population consumption of games and VW's yields even more value. Churn rates related to interactivity, demographics, duration, intensity, etc.

More importantly we begin to understand more than the "What" of the population but also the "Why" and "How".

Much is owed here to the efforts of Castronova/Malaby/Koster in this area.

Multiplicity: Easy enough and straight forward. There are initially about 10 data points for this. What it comes down to is number of accounts maintained by individual and household, coupled with cost metrics, method of payment etc. and when you add metrics related to price sensitivity, adoption, conversion rates and etc. This area would begin to bear apple to apple understanding as it related to cost, per household, per anum, by VW/game type. And from a more business perspective, where to set price points, when and why....for in depth understanding on data mining this area we owe much to our local grocery store....

Active Users: A hard an elusive target again without those mechanisms’s in place as mentioned before:
Dashboard, SAS or DSS

One has to rely on self-selection related to value sets to get a macro understanding of this measure.

I'm sorry for the length of this post, but it's important to many people, and well, actually it's important to our start up :)

Finally:

"I understand the incentives to supply distort or hide these figures and I just don't care."

Neither do I, its more important than any one company's interests, its important to a large industry, its stakeholders and more important to those who enjoy VW's and Games. The people, who create, spend time, and money deserve to have better information, that’s relevant and actionable.

"And sure, we're all subject to using what companies actually decide to disclose, but that's no reason to not spell out a better standard."

I agree. What we're (GMM) doing is a pretty risky venture, we're going for it anyway. We're apt to make mistakes, because frankly we're learning and testing what will/wont work for our user base, and while I've got a pretty good idea about demand, getting validation is much harder when ones untested and proposing a different approach.

Either way the data will be generated, proprietary interests and obscurity not with standing, because the tools and mechanisms are in place to power the data acquisition, we're not innovating so much as aggregating possibilities and tools, and reliance on dated methods of analysis, (besides being costly) has failed to yield
results. This is our contribution, as gamers, to the solution.

People deserve better VW's and games.....

[randolfe_]

32.

IAmEric,

Those are fair points, and I certainly don't mean to cast anyone interested in studying the economic aspects of VWs as cultists. My primary point is that those immersed in all things VW/MMO can easily miss real-world constraints and frictions.

My contention is with legitimizing the reporting, study, analysis and eventual prediction of things economic using VW economies as a basis. It is folly for myriad reasons. Perhaps with enough advancement of VWs they will evolve into full-fledged economies.

But for now, and I'm betting for a very long time and maybe always, VWs represent *markets* and not *economies*. These terms are not interchangeable, one being a subset of the other. As a study, it is the black _market_, not the black _economy_, even though we may call it that for popular clarity. VWs are also merely markets; specifically in terms of RMT.

China and others worry about this because transactions currently fall outside of established regulatory mechanisms. The US also worried about internet barter for a while, but that was just another market. Once they established tax code and GAAP rules regarding virtual transactions, no one much cared anymore (and most of those profiting off of the novelty of it evaporated).

I will revise my earlier indictment of studying VW economies as such: study of microeconomics within VWs is perfectly legitimate. However, there is no macroeconomic there there.

[Dmitri Williams]

33.

@Andy. I still disagree, but probably not in the way you think. You mentioned the importance of test marketing, focus groups, acceptance testing, UI testing, customer lifecycle research, reseller acceptance, affiliate tests, win-back programs, segmentation research, etc. All of those things are valuable, but none of them are as valuable as they can be if you don't know the total population first.

You get some interesting leverage on phenomena, but then you don't know if it's generalizable or not. And that's a waste of a good opportunity. Any one of the tests you mention here can be made more useful if you also know if the sample used is representative or not. And you can't know that until you've had a census--outright or sampled itself.

As to using or not using some system because it will or won't make SL or other worlds look good or bad, that is for SL to worry about. If honest brokers report fair numbers and also give information about how to interpret them, then people will make of them what they will. I look at your point #1 and think, "Well, SL has fewer subscriptions, yet seems to have a lot of interest. I wonder how LL is making their money. I think I'll look further." And then I'd notice the numbers for Duration, Intensity, etc. and start thinking about how and why that world is or isn't fundamentally different. Will ignorant or malicious people misuse statistics? Of course. That's where experts are useful. But you still need to at least give the experts the data, or they're guessing in vaccuums.

Lastly, none of these statistics should be compiled or reported without some understanding of how the virtual world works. Without that, the numbers can fail at telling the whole story. This thread is simply about getting the numbers right.

34.

Hi Dmitri,

I'd like to throw a thought out there and see what happens. You said, "I look at your point #1 and think, "Well, SL has fewer subscriptions, yet seems to have a lot of interest. I wonder how LL is making their money. I think I'll look further." And then I'd notice the numbers for Duration, Intensity, etc. and start thinking about how and why that world is or isn't fundamentally different."

One of the reasons why I think SL is different (admitting that I know very little, i.e. nothing, about other virtual worlds) is that I think it has grown to be bigger than LL already. When you talk to executives at LL, very few of them have a clue what is going on in SL. I believe I learned from one of Castronova's papers, but can't recall the reference, that there are precedents for the residents of a virtual world to actually take control of that virtual world even if the parent company goes bankrupt. If SL hasn't reached that critical mass yet, I think it will very soon.

How might that kind of dynamic be captured in numbers? I don't think the metrics proposed so far address that (granting they were not intended to). However, I think that kind of internal economic, social, political dynamic within a virtual world is something equally worthy of study. In the case of SL, I actually think it is the dominant factor to consider. I think the parent companies will become less significant to the future of the virtual worlds they created as the populations increase. In some sense I think the internal dynamic is more interesting and more important than the external dynamic, e.g. how much the parent company is making, etc.

Best regards,
Eric

[Dmitri Williams]

35.

I think it'd be very unlikely that SL would continue if LL were to fold (which, BTW, is not about to happen). It's an expensive service to provide, and the population hasn't shown the willingness to pay for that service. Those very numbers about paid subscriptions/total subscriptions have to at least be considered a hint that the cash flow in from subscriptions alone won't suffice.

Now whether some internal engine (or an exogenous exchange-based one) would suffice is another question, and it's something that I'm sure could be measured.

As to whether the internal numbers would be more interesting or relevant for SL, I'm not arguing. They might be. I'm still interested in getting some basic population numbers as a starting point. Consider the premise of your suggestion: the users get to critical mass and take ownership of the world. OK. How do you know what critical mass is? Is it a % of a whole, some absolute number, what? You still need the data to make the argument concrete.

[Andy Havens]

36.

@Dimitri: You make very good points. And, in the abstract, one would also like to be "the invisible eye" and be able to do testing that doesn't let the subjects know you're doing testing. We'd also like to be able to do research that eliminates all variables that are unlike from system to system, so that we can study, for example, "is voice chat a compelling feature in MMOs/VWs regardless of in-game task, and what (if any) price elasticity might be related to providing that feature?" It's very tough, though to remove all "sideways" variables from any single-point study.

And that includes gross user stats.

Again (and I seem to repeat this in every comment), I'm not arguing *against* getting good pop numbers. They'd be great. I just want to make sure we don't sing (as Clay seems to be doing) a chorus of, "Head Count Above All Else."

I spent 10 years in marketing in cellular telephony during the time when it went from "rich toy" to "everyman necessity." During that period, we learned a lot of hard lessons about how to count (and how not to count) heads, and how different kinds of heads meant very different things.

One example, for example; prepaid phone service. For a long time, in the earlier days (until about 1997), it took about 6 months on average for us to earn back in ARPU the up-front cost of acquiring a new subscriber. That's one of the reasons we required a 2-year commitment. Regardless of the cost of advertising, the buy-down on the phone (we gave them away free; still do in many cases) plus other set-up and sales spiffs was that high. And that doesn't count advertising COA, which you don't figure into your earn-back, because every product has an ad COA (ours, at the time, was around $60, which net-net, would add around another 2 months to the earn-back; like I said, though... don't count that; think 6 months).

So... if it takes 6-months to make any profit on a new sub, and we want a 2-year commitment, we required a credit check. Bad enough credit? You don't get a free phone and you don't get to burn us. Then along comes prepaid service; you pay for the phone up-front, you pay for a block of minutes, when they run out, you pay for more. Much less of a credit risk, because we only pay for about (if I remember correctly) about 2-months up front in sales/co-op licensing and other fees.

So, at first, our big thought was, "Here's the answer for credit-risk customers that were falling out of the bottom of our current, standard, advertising/sales pitch. You come in for the regular deal-i-o, don't make the credit score, we sell you on prepaid." Made perfect sense to us.

It took us quite a few months to figure out that, though those folks were a small portion of the audience for prepaid, the biggest was, in fact, NOT the "credit challenged," but the "I hate getting a surprise cellular bill every month and want to control my costs even if those costs are a bit higher" crowd. And that crowd is a lot, lot bigger than the low-credit gang.

We were looking at a "gross number" -- folks who didn't make our credit score for post-paid -- and thinking, "this is our audience for pre-paid." We had the number right. We did all kinds of research based on that number. We asked those folks a ton of questions about how much they'd use prepaid, what they'd pay for it, how they'd like to use it, where they'd buy it, etc. etc.

But it was the wrong number. And it was dinky compared to the real number. And we ended up marketing to the wrong people; bad-credit folks with a little money to spend on a low-end product for the credit-challenged, instead of a shiny, happy product that you can use to control costs (even if that control-level is as high as some post-paid bills), give to your kids, etc. etc.

Good research on the wrong number, even if the number is accurate, is as much of a stank as a bad number.

And this is my problem with Clay's "Big Beef."

Ya, ya, ya. Linden's 2-million is bogus. But he also seems to want to compare whatever the "real number" is to WoW, LambdaMOO (?), "social software" (email? blogs? Mososo? dating services? wikis?) in general using the raw number.

And that, to me, is ridiculous. As is the assertion that he doesn't care what the experience is like; i.e, the "features." Because that is, essentially, an assertion that all social software is the same.

Right. Just like all books are the same. And all web sites. And all blogs. It's just words and pictures. So we can judge them based on one gross metric: popularity.

Social software is *not* all the same and serves incredibly different purposes depending on the audiences, the intentions, the features, the function and the goals. Good grief... Does anyone here really... REALLY think that WoW should be compared in any serious way to eHarmony.com (which has over 12 million registered users, btw)? Even though there are stories about romances that bloom inside Azeroth? That particular "social feature" of WoW is incidental to its function, which is to play an RPG.

Hell, I'm sure that there are romances that have jumped off Wikipedia. I fell in love with my wife in my high-school cafeteria. That doesn't make the lunch lady a yenta.

You want "standard metrics" for MMOs and VWs, my point is that you need to decide the *goals* for those metrics. Back to my prepaid story; we got the goal of our prepaid metric deeply, deeply wrong: we counted potential add-on customers from our bad-credit pool. Nope. Wrong number. And by counting that number very well, we failed (for awhile) to count the right number: good credit customers who would spend MORE money with us if given an alternate billing option.

So... Here are my questions for the learned crowd:

1. Do you really want to count a person who is contributing content to a VW the same way that you count a person who is playing an MMO? Because the loss of an MMO player is, essentially, straight churn (w/ the usual "may drag a few buddies with him" caveat... then again, for a griefer...). The loss of a "creative customer" in a VW will actually bleed off your content. Example: what would the death of Anshe "as player" do to SL? Is she really even considered a player at this point? If she decided, "Screw it. I'm outta here," what would the affect to SL be, compared to a newbie land-holder? But under the current proposals being discussed, her leaving SL would count as, "-1 to active users" of the system. Seems, to me, to be an "accurate count of a wrong thing."

2. If a VW is earning revenue through the sale of virtual items or land, how do you (or do you) track that against revenue from account fees? For years, we gave away free phones and made a metric ton of money on the airtime. "Give 'em the pipe, sell 'em the tobacco." Very old tune. So... if you want to measure economic activity for reasons of investment, financial health of the company, etc. etc., you need a way to really, really look at the "dollars go in, dollars come out" of a system like SL vs WoW. And "value goes in, dollars come out," as well. Because lots of what happens in SL that costs the users nothing, still generates "value for Linden" that, in turn, generates eventual dollars for Linden. It is the equivalent of "earned media." I'd call it "earned content," or some such nonsense.

3. Since we are talking about the "social-ness" of these spaces, some kind of metrics to include the ability to interact with others in the space would be, I think, a good idea. Is there a way, for example, to form groups, clans, guilds, etc. in game that persist from session-to-session? (binary yes/no). If so, is there a limit? If so, what is the average usage of such? There have been for example, stories of "Playing alone together" about WoW; i.e, how it is possible to play much of the game as a single-player adventure. I've played lots of my characters up to 15-20th level without doing much grouping. So... is there a way to say, "Most users of this service engage, on average, in X number of in-game relationships?"

If, for example, we found out (makin' stuff up, here) that yer average SLifer was a member of 12 groups and had 22 friends, but that WoW players were a member of one guild that had 9 members... Well, it's a bigger game, yeah... but much less "social." That's good to know from a potential customers standpoint, eh? A larger city, but much less friendly, eh?

4. Economic potential: Can you make money in-game. And I don't mean out-of-bounds RMT. If you need to use eBay, it don't count. If I want to make money right now, can I do it in WoW? No. SL? Yes. Project Entropia? Yes. So, in a sense, can we measure "player earnings in real dollars" for those players who choose to play "the economic game" on top of the "VR stuff?"

5. Multiplicity: I'm going to go out on a limb here and suggest that it doesn't matter. I, as a human user have only 24 hours in my day. If I want to have 2 accounts and pay for them, and one of our measures is paid accounts... I count as 2 paid accounts. If I log in as "Character A" for 2 hours a day and "Character B" for 1 hour a month, "A" and "B" each go into the "time played per character" measure accordingly.

I can subscribe to an almost infinite number of MMOs and VWs, eh? I can count as a WoW sub and an SL resident. I can be a Puzzle Pirate and a New Pet. As long as I am going to be counted again and again and again in multiple worlds -- and as long as we get our metrics straight for how we're going to do our counting -- I don't think we can worry about multiples in one world. If a platform allows for multiple accounts, they count. Now... that's different than counting multiple "toons" or "avatars" per account. Having multiple toons or characters is essentially a feature of the system. I can see a game where you get to rotate between hundreds; that doesn't signify anything. But accounts is different. You want to log on today as "Fred" and tomorrow as "Ginger?" Fine. You count as two people... but only as half a person each time in terms of the appropriate usage stats. That should work out in the wash. If it doesn't, our numbers are bad/wrong. Aren't they? Because somebody who logs on twice as long as one person (maybe as two toons?) is providing just as much "stuff" to the system as someone who logs on for the same amount of time as two. Right?

Which is the old conundrum... Do I want one $100,000 customer or ten $10,000 customers?

Do I want one 10-hour-a-day player, or five 2-hour-a-day players? And which is more valuable?

One vocal troll or ten silent audience members? Who counts more?

6. Are you going to count re-activations the same as new subs? IE, if I join a service, play for 6 months, quit for 6 months, and come back... is my return counted the same as a brand new guy?

That's important, because the ability to retain and retract lost subscribers speaks to long-term success. Also, it's more expensive (generally) to attract new subs. A service that does well in getting back lost subs on top of getting new ones... that's good eating.

7. Are you going to count involuntary churn in your churn numbers; i.e., people that kicked off by the service for no-pay, bad behavior, etc., or just people that quit under their own steam?

8. Are you going to have any kind of lifecycle measurement to quantify subscriber levels or loyalty or participation? By curve or bucket? For example, some comment was made that some percentage of SL players spend 80 hours a week on the grid. Yowza. That's heavy use. I'm sure WoW has folks at that "tier" of usage, too. Do we want to quantify just "intensity" or any of the other measures, too?

9. Sector metrics. Specifically, are joiners of a particular system "growing the pie" or are they coming over from another platform? Much of the early growth in cellular was totally new to the medium. Now? It's jumpers. In MMOs/VWs, we've all made this assumption that WoW players are new to the MMO sector, and that may be largely true. But are they new to gaming? What about SL? Are these folks totally new to the whole idea of cooperative, technologically enabled "play?" If so, that might explain some of Raph's recent entries about the insularity of SL's blogs. This would also involve measuring "entrance/exit" (what did you play before and after this) and "multiple platform" stats (what else do you play).

That's what I've got in my head at the moment.

[IAmEric]

37.

Hi Dmitri,

This discussion has been fascinating. It has taken many turns and spans across multiple blogs and multiple threads within each blog. Just to clarify one point, I agree 100% that getting at a meaningful number to represent the "population" of a virtual world is extremely important. I don't know how to link directly to a comment, but if you look at my initial foray into TN comments here:

http://terranova.blogs.com/terra_nova/2006/12/shirky_to_tn_re.html

(I guess just search for "IAmEric")

you'll see that we were trying to come up with a definition for a true "resident" of a virtual world (I had SL in mind). I thought that if enough smart people got together, we could come up with a satisfactory definition of a SL resident (unfortunately LL has already used the term "resident" to mean "registered account" which is NOT what I'm talking about).

The term "population" likely has various meanings for the various virtual worlds and it is likely to have no meaning at all for some. However, I also think having a meaningful number that is meant to represent the population of a virtual world would be useful.

Do you have any suggestions?

Perhaps, like Andy suggests, we should try to define more clearly what we are trying to get at by "population". What should the population number really represent?

My initial attempt was:

"A Second Life resident is someone who maintains a recurring and regular presence within Second Life. Their account must be at least 30 days old and they must have maintained an average of 3 hours in world per week since the creation of their account."

Then tateru offered this suggestion:

"I might have a looser definition of Resident than you. I'd say someone who spends time at least once every three months qualifies. Let's put a number on the hours. Say three hours, every three months. If I were running a bar, I'd call that a regular. I'd hope for a lot of those, rather than trying to rely on my die-hards."

Based on some feedback, I offered this refined definition:

"A Second Life resident is someone who maintains a recurring and regular presence within Second Life. Their account must be at least 3 months old, and they must have maintained an average of 3 hours in world during every 3 month period since their account was created excluding the initial 1 week period."

Tom offered this modification:

"A Second Life resident is someone who maintains a recurring and regular influence within Second Life. Their account must be at least 30 days old. In addition, they must have maintained an average of 3 hours in world per week since the creation of their account OR have performed an average of at least US$X per month in net financial transactions over the past 3 months."

I'm not particularly satisfied with any of these definitions, but its a start. I'm hoping someone with more clarity of thought than I have to come up with something more succinct that is agreeable to most reasonable people with an interest in the subject.

Once we have such a satisfactory definition, my proposal is to present our definition to LL and ask them to provide us with data, preferrably a time series, for the true population of SL as defined by us. They have no obligation to grant our request, but I don't see what harm it would do and from what I have seen of LL, they do not seem like unreasonable people and would likely grant the request.

What do you think? If each virtual world could come up with a coherent definition for "population" (it will likely be different for different virtual words), then that number should be roughly comparable across virtual worlds. It seems like it would be the easiest number to get at (aside from the meaningless number of registered accounts).

Best regards,
Eric

[Allen Sligar]

38.

"I fell in love with my wife in my high-school cafeteria. That doesn't make the lunch lady a yenta."

Andy, your always good for a great line at the right time...thanks

"That's what I've got in my head at the moment."

All good stuff and a great post, you know you bring up some things I've missed myself, and some things that "are frankly not within our scope at the moment, and some things that would require deeper analysis than is possible by an external organization without proprietary data.

"You want "standard metrics" for MMOs and VWs, my point is that you need to decide the *goals* for those metrics."

True, IMO that consideration (goals) should be independant of the data set to maintain neutrality and integrity for comparative analysis purposes. Thats apples to apples.

This is not to say that deployable solutions are not available, much of this depends on your preferance of vehicle (quantitative/qualitative). Various slogans apply to this metaphor of course....Drivers Wanted, nobody rides for free, etc.

have a good new year...

Allen

[Ken Fox]

39.

All of Dmitri's statistics could be collected without the help of the VW owner.

Duration: This requires account identity which is not recoverable without decoding the VW communication protocol or installing special software on the client. http://www.xfire.com/ is an example of an add-on friends list that would provide account identity.

Intensity: Average bytes per minute upload traffic. This number must be callibrated to specific VW communication protocols.

Economic activity A: Advertised feature.

Economic activity B: Advertised number. May be hard to figure out promotional discounts or special deals.

Multiplicity: Simultaneous connections per IP address. Most people running multiple accounts use broadband Internet and a hardware firewall. The firewall appears as a single IP address, but simultaneous connections can be distinguished based on the TCP connection id.

It's not necessary to have the VW owners' permission or cooperation to collect any of that data. Decoding the connection data may violate a TOS, but this is a third party not bound by TOS.

The Internet service providers can easily publish this information. It may take a government request, but I bet a consumer group could pressure the ISPs for the data. Lately the ISPs have been rolling over and providing data to anybody.

RIAA probably already knows the names and accounts of every LL user. ;)

[Ace Albion]

40.

Well, from now on, all being well, I won't be paying money from my CC to use SL again. I haven't gone basic, I just have enough L$ income now to cover my costs when exchanged to USD in my SL accounts.

I don't know how you measure this type of activity... You could still say- this account pays us $72pa plus $40pm etc and not worry about where the USD to cover that came from. Except that L$ income was, in some cases (maybe most) provided by people who bought L$- either as the stipend as part of subscription or from that Linden account that prints money from time to time to sell on Lindex. So in effect, avatars Angie, Bob and Charles are subsidising my SL- they're paying my sub and my tier. And we're possibly double counting money being paid to LL as the currency gets exchanged and back through subs/lindex.

Add some more confusion- I'm the named owner of an island sim, but I only have 1/8th share I truly call my own. So personally, it's about $25pm alongside a downpayment of $150-ish. Except that I, on one account, paid LL $1250 and $195pm. That money is actually the funds from six different people/accounts, funneled through me, and provided again by transfers of inworld currency to USD against my account- the other islanders get their money again from people who buy their stuff etc. Counting me as one paying subscriber (actually I have three premium paid subscriber accounts now) just seems way too wrong. The content/service providers in SL are filters through which a bunch of consumers- *many of whom pay nothing direct to Linden Lab- I have No Payment Info On File customers* are in effect, invisibly paying for SL. They are also paying Anshe etc in paypal offsite to lease virtual beachfronts. Anshe is "users +1" filtering that through to LL as one tier payment for one island, per 12-16 people or whatever.

I guess it's like imagining that WoW or something like that had some "guild subscription" option, where Mighty Guild of Shiny pays whoever runs WoW $1000 a month and then makes its own arrangement with guild members as to how much they pay the guild treasurer/accountant. It's the same kind of idea. It's also all off the books, except for "Treasurer Elf Guy is user +1 and pays us $Xpm".

I don't know how you can compare that kind of business against "this girl = $15pm, that guy = $15pm" all lined up in a row.

There's another economic effect of all this- In world effort funding subscription. I paid up my most recent alt to subscription because I had enough L$ I could transfer to it and I didn't have to worry about cashing that money out and dealing with tax etc- some people work SL this way- buying (leasing) islands etc all from inworld graft. I think EVE has something similar with its "game time codes" that people buy for actual money and sell for in game money. So you can earn in game to pay. That's the RMT stuff people find hard to count or prefer to not count, but it's intrinsic to SL for sure. Those are my anecdotes anyway, just to show how I don't think SL is ever going to be apples in this orchard.

[Clay Shirky]

41.

@Andy,

Right. Just like all books are the same. And all web sites. And all blogs. It's just words and pictures. So we can judge them based on one gross metric: popularity.

Andy, you are now putting words in my mouth. I have never said anything about demographics being the only metric -- there are lots of people working on economics or user experience or whatever, and I have never once proposed that those people are wasting their time.

I have proposed, and still believe, that that work would be better grounded if we had good demographics, and that the adoption rate, which is to say aggregate social judgment, matters, and it matters in a way that is comparable across platforms.

Think of social judgment on a two-axis chart: Try (denominated in integers), and Adopt (denominated in percentage points), where 'adopt' is a measurement of longer-term use.

There are services that many people try, and a high percentage of those adopt. We'll call those 'popular.'

There are services that few people try, but a high percentage of those adopt. We'll call those 'niches.'

There are services that few people try, and a small percentage of those adopt. We'll call those fizzles.

And then there are services that many people try, but a small percentage of those adopt. We'll call those hyped.

Your consistent claim has been that these kinds of comparisons are meaningless, but they are not, and the proof is that the press is reporting these figures as evidence that SL is in the first category.

And my complaint, consistently, has been that a) we don't know what category SL is in, and b) that the press, not Linden, has been the proximate source of that unknowing.

Your objections about the subtleties of the numbers would make sense of the press were making subtle assertions, but they are not. As I said in another thread, it cannot be the case that SL is both the future of the internet and not worth reporting right.

[Allen Sligar]

42.

@Clay

I'm going to have to agree with your breakdown on adoption/try (I rather prefer "aquire").

This specifically:

"And then there are services that many people try, but a small percentage of those adopt. We'll call those hyped."

Is prcisely why SL is confusing, and may both be the "future" and not "worth reporting."

We can agree that SL is a valuable VW space even given all its fualts and that its gotten a lot of media, and that this media has mostly been numerically "challeneged".

Also look at the means of aquisition, DD of the UI to the client, no real barrier to multiple account creation, etc. Thats a no brainer on population inflation.

These terms are commonly used with some meaning:

Registered/Subscribed/Active/Concurrent.

However its pretty well understood that when using DD as a mode of aquisition for new users the conversion rate from registered (I downloaded the SL client) to Subscribed (I have a premium account I pay for) is dimsmal, really seriously dismal.

Is doesnt matter if its SL or Orcs n Gnomes Online, the conversion rates for digital download games are low. Because DD becomes a volume distrobution game at that point, your not competing on price point (because aquisition is free) your competing on features (to garner adoption)

Therfore SL may in fact be the future and not worth reporting, at least the conversion rate issue isnt news to most people who follow games.

Although maybe its worth reporting to a different target audiance....

I am not an expert on such things merely an observer.

[csven]

43.

@Clay - "we don't know what category SL is in"

What are the choices?

[Clay Shirky]

44.

@csven: the choices I'm proposing are popular, niche, fizzle, hyped. The breakdown is in the above post.

[csven]

45.

@Clay - The problem I have with that breakdown is that I don't see those as categories in and of themselves. For me, adoption can mean different things depending on perspective. The assumption here seems to be that SL is *like* a videogame and *not like* a long-term use business application. I've often said I consider SL to be closer to a PLM, and it's not at all uncommon to hear of companies "adopting" a PLM platform, because for them the timeframe suits that term. I've never heard of anyone adopting a videogame. Not that it can't occur, but I don't think in those frames of reference in that way.

Years ago I chose an application. I've not used it seriously in over eight months. Have I adopted it? Your "repeat user" criteria makes that questionable, but the answer is a most definite "Yes", because I'll be returning to it... even if after a longer timeframe than most feel appropriate within a videogame frame of reference. I can do that pretty easily because it's open-ended, it doesn't rely on a host (which could go bust in the meantime) and it isn't a game with a pre-defined endpoint so that I can't continue using it in a way that's meaningful to me. Assuming that LL open sources it, Second Life is more like that application to me than it is to a videogame like WoW. Hence, trying to shoehorn SL "adoption" into the same frame of reference with a MMOG doesn't work for me.

I've been, by your definitions I'm fairly sure, a non-user of SL for the better part of a year. Yet I pay a monthly fee. Does that make me a "repeat user"? No. But I intend to return to using SL. Does that indicate some level of adoption? Yes. Do I intend to use SL in five or ten years whether it's a hosted service like today or an open source platform? It doesn't matter. The fact that this question is even able to be asked is, for me, a game changer.

Maybe the real issue isn't the word "Adopt", it's that you use the word "Service" which suggests to me a developer-centric/hosted-control mindset. However, if SL is open sourced, what then?

[csven]

46.

@Clay - I thought I'd add this example. The application I "adopted" also went through a very heavy "hype" period in the mid-90's. It was, when I now think of it, not too dissimilar from SL. And a very large number of people reacted negatively after trying it. And they dumped it swearing to never return to it.

That application is still around and lo and behold, it's now becoming adopted by the same people who called it out as being nothing but over-hyped crap. The application didn't really change; they and their perspectives changed.

[Andy Havens]

47.

@Clay: Your most recent comments directly above (about the placement of a service's popularity vs. adoption on the 2D grid), and here about how your motivation for all this is to keep companies from diving in and using SL as a platform to do things its clearly either not meant to or ready to do... come much closer to convincing me than anything else you've said.

If you'd started out your first article with a description of that grid and the categories: "popular, niche, fizzle, hyped," and your line about carrying ammo for programmers... and then making a statement to the effect:

"The recent over-reporting of SL's usage numbers, caused in part by Linden's shady term 'Resident,' and in part by the press' unwillingness to light up that shade, is leading many new users -- both individual and corporate -- to treat SL as a 'popular' service, when, in fact, it may still be in the 'niche' stage."

That statement, which (I think... I'm really, honestly not trying to put words in your mouth... just paraphrasing, 'cause that helps me process) seems to sum up what you've said in the last couple comments, is a lot easier for me to wash down than the, "Linden is intentionally snowing a gullible, chump press and a rube public that can't figure any of this out for themselves."

If you'd gone one step further, and added something like:

"Many of these new 'Residents' simply may not be sticking around. And this will be, in the long term, just as bad for Linden, Second Life and the VW community as it is for the readers of inflated statistics. Because any user or company who bases their expectations on gushing press tales of virtual crowds that haven't materialized is going to expect more from their Second Life than Linden can deliver. If you go into a park expecting Disney, but only get the County Fair... well, you're bound to leave disappointed."

That would have put it in terms that made the downside even more apparent in the context of your "popular, niche, fizzle, hyped" categories (I think).

I honestly hope I'm not badly characterizing your argument; truly, just restating for my own brain to get around. As a writer, that helps me to get closer to my own thoughts.

And I'm much closer to your POV than I was this morning on this. The "carrying ammo for programmers" and "grid" arguments are good ones. As Thomas can tell you, I embrace The Beginner's Mind; ie, I do admit room for my own instruction and change of opinion. But only after long, hard fights with smart people. ;-)

I'll tell you the two spots I'm still not on board with you, and you can decide if they're worth pounding any more on, or if this horse is dead.

First... your grid is 2D. I suggest a 3rd axis: "influence." There have been, throughout history, any number of truly popular services/platforms and other thingies (ooh, jargon) that have not ended up having much influence. People like vanilla. It's always there. It's popular. There are all kinds of popular pursuits, entertainments, etc. that deserve less attention because, well... they're the damn same as what came before. MySpace is hugely popular. It's clearly been hyped, too. It ain't niche and it ain't fizzled. But I would argue that its importance and its influence in how we work/play/engage with each other online (especially in the VW space) is, in the long-run, going to be rather low. We can get into that later, but I'm using it as an example. SL, on the other hand, even when it was (and still may be) niche, does many things that are and will be influential, I believe, to how the VW space will develop. Some of those influences are negative, some positive. But the "influence" axis helps provide some of the context that I keep braying about; it lets us address the fact that, for example, SL has been getting great press long before it had "popular" (if fake) numbers. It was niche/influential. There are many MMOs and social communication platforms out there that are niche/non-influential. Grouping SL in with them might be just as detrimental to getting the whole story right for the coders for whom you carry ammo as getting the numbers wrong.

Second, I'm still not sure that the "popular" number is what pushed the SL stories into the forefront of the press as much as your original angle asserts. As I've pointed out a couple times, there was a ton of press around SL back when even their bogus numbers were goofy-low. So the story, at that point, was still really a "this is niche, but really interesting" one. And the meat of much of the press hasn't really changed much. Yes, the numbers may be bigger and more inaccurate, and they may paint a "popular face" on what is still a niche product... but the description of the function and features is still very much the same as what we've been getting all along.

Now... those two points being made...

As a marketer, writer, poet and all-around loud-mouth, I understand that in order to get action in either the MSM or the blogosphere, we often have to abandon subtlety in favor of vim. If part of the strategy of your original articles was (is) to abandon subtleties such as "influence in context," and "the relative importance of the point I'm trying to make balanced against several others," in order to really bear down like a mutha wombat (no offense intended; I love me my wombats) on one single issue...

then we have nothing left to discuss. 'Cause I know that ya gots ta do what ya gots ta do.

With the caveat that non-subtle arguments are going to get picked to death by grognards like us on blogs like this until doomsyday.

Either way, I'm having fun. And, as I said, your last two comments have brought me at least 37% closer to your position on this. Good shew.

[Allen Sligar]

48.

@Dimitri et. al.

Wondering if this thread has concluded what standard metrics are desired....I'd like to check them against the data points in our data base and include any I might be lacking. I'll happily provide you with a data dump, or summary stats of you need after closed beta ends (although the sample size wont be representative but an example data set, because its a beta sized population!).

If your interested the only requirement I have are these:

At least 5 but no more than 25 data points, include valid values for these data points, and make sure they are both granular enough and mostly usable across multiple data points.

If you can meet these requirements I can adjuest them for uniformity against our system and load them up (it only takes about 10 minutes).

Shoot me an email if you want to discuss this off the thread. ceo{at}gamemarketmetrics.net

Allen

[csven]

49.

The one I'd like to see is one of the one's I listed above: The average amount paid to anyone for anything related to the virtual space per user per week across the world population.

[Allen Sligar]

50.

@csven

There are assumptions in your metric, but I'll break it down as follows in easily digestible text:

Do items in the VW/Online Game you play cost real money? Y/N

What is the averege amount of real money you spend on VW/Online game items per week. (X.00-X.00 amount) (Possibly 0-200.00? can we assume consumption over 200.00/week is an outlier?)

This simple metric you've outlined brings issues and is dependant on other captured metrics. However this elicits some results:

1. # of people who participate in a VW/online game with RMT

2. Of those people in 1, who do participate in such spaces, how many spend money?

3. What is the breakdown by expendature by %(I'd be breaking this into 25.00 ranges or rather 8 valid values such as: 0-25.00, 26-50.00 etc.).

4. We would extrapolate these against other metrics we've already captured to get an "apples to apples" comparison.

As you can see simple metrics often demand other considerations, this is all very mundane low brow and utilitiarian for TN however to maintain data neutrality and integrity of a data set it's important not to make value laden (such as your population always participates in a VW economy, and understand what RMT is) assumptions.

[csven]

51.

@Allen who said "As you can see simple metrics often demand other considerations

I'm well aware that there are other considerations.

However, your example #1, "# of people who participate in a VW/online game with RMT", doesn't make sense to me since it seems as if you're automatically dismissing VW/online games with *no* RMT. That automatically throws out games like Everquest which had no RMT yet did brisk business outside the game when Castranova did his study of it. Whether or not the developer permits or tracks user expenditure is irrelevant.

More important to me is that this is *not* an Access/Provider/Developer-focused dataset. This is a User-centric metric. And afaic, more than anything, it's the Users themselves that are the common element between something like WoW and SL. Hence, imo they're the best way to get at something approaching an "apples-to-apples" system.

[Clay Shirky]

52.

Worth noting that Rosedale has given David Kirkpatrick some numbers:

=====
#1. 1,525,670 unique people have logged into SL at least once. This is considering distinct email/payment info as distinct people, rather than IP addresses. [He is checking the unique IP address numbers but suspects they will be comparable.] [...]

#2. 252,284 people have logged in more than 30 days after their account creation date.

#3. For the answer to #3, Rosedale provided a spreadsheet which lists the number of new registrants by month and the number of those who were still returning longer than 30 days later. That data shows the astonishing growth I referred to - registrants in January 2006 were 20,000. In October they were 254,000.

While the percent of registrants still active after 30 days has, predictably, declined a lot since early 2004 when it exceeded 45%, it remains a substantial 15%. Of those 254,000 who registered in October, 39,575 still were active after 30 days. The absolute number of those still returning after 30 days grew 23% for October's registrants over those who registered in September.
=====

The full comments are here, starting with my original question. Scroll down for David's answer:
http://money.cnn.com/blogs/browser/2007/01/why-second-life-numbers-do-matter.html#comments

[Allen Sligar]

53.

@csven

"doesn't make sense to me since it seems as if you're automatically dismissing VW/online games with *no* RMT"

Its not a dismissal, its a distinction.

I make the distinction between VW/Online games that have RMT as a feature set, and those whose design does not encompass it in our data set, this allows for more granular analysis.

To clarify one is facilitated via overt design and tacit approval from the provider, one is facilitated by the grey market.

"Whether or not the developer permits or tracks user expenditure is irrelevant."

No actually its not, if you would have a better VW economy and a well designed system of exchange understanding the distinctions between RMT by design and RMT by 3rd parties is relavent.

"More important to me is that this is *not* an Access/Provider/Developer-focused dataset. This is a User-centric metric. And afaic, more than anything, it's the Users themselves that are the common element between something like WoW and SL. Hence, imo they're the best way to get at something approaching an "apples-to-apples" system.

I agree, to the tune of quite a bit of money, time and work on the part of my partners and myself.

Also understand that there is room for a "Access/Provider/Developer-focus" in data aquisition as well, they need data (as do Academics, marketers, students, investors etc.)just as much if not more than anyone else.

I really dont want to self promote here on TN but you can register via clicking on my name or email me direct at the address I posted previously.

In fact I'll need some input from the SL/There crowd in developing data points I likely missed since those spaces are not my forte.

[csven]

54.

@Allen

"Its not a dismissal, its a distinction.

I realize that. Hence my using the phrase "seems as if".

The issue I'm pointing out is in this statement you make regarding the relevancy of making that distinction:

"No actually its not, if you would have a better VW economy and a well designed system of exchange understanding the distinctions between RMT by design and RMT by 3rd parties is relavent.

Why assume *I* want a "better VW economy and well designed system of exchange"? That seems like it's taking on a developer perspective; a *control* perspective. I see no reason to make that assumption about what *I* want. The "User" perspective and the information I'm wanting is potentially of interest to parties with no stake - controlling or otherwise - in VWs and videogames.

"Also understand that there is room for a "Access/Provider/Developer-focus" in data aquisition as well, they need data (as do Academics, marketers, students, investors etc.)just as much if not more than anyone else."

Why assume I don't understand this?

What *I* desire is very straightforward: a single number per user per VW/videogame. If it can't be done, fine. If it can, I just want the number. If additional data is collected in the determination of the one number I want (and I would certainly expect there would be), and that's of use to other parties, fine. Only I don't want it, and I don't care if it's of use to a developer to improve anything. I want the one number only.

[Allen Sligar]

55.

@Csven

"Why assume *I* want a "better VW economy and well designed system of exchange"? That seems like it's taking on a developer perspective; a *control* perspective. I see no reason to make that assumption about what *I* want."

Your right it is an assumption made from my perspective. However, I'm neither a developer, nor an academic, those persuits are better left to others more qualified (besides I have a real aversion to self inflicted pain lol) just someone who likes games and VW's, who happens to like data as well.

I think most of the misunderstanding is in my unwieldy use of grammer, trust me I dont make assumptions about what people want when dealing in data.

"The "User" perspective and the information I'm wanting is potentially of interest to parties with no stake - controlling or otherwise - in VWs and videogames"

Ok I had to reread your post a second time to get your drift. zI think I'm clear on your meaning.

"What *I* desire is very straightforward: a single number per user per VW/videogame."

Thats fine, it doesnt concern me how people use the data, only that it can be reliably aquired.

[csven]

56.

@Allen - thank you for understanding. Now I'm curious, how would you approach something along the lines of my earlier post (Link) where the data is desired for distinct reasons coming from different perspectives?

I'd like to assume, if we're trying to get an "apples-to-apples", that those other areas can also be represented by a singular significant number (perhaps not a simple one; it could be some unitless amalgam). I'm doubtful the ones I've stuck in there are appropriate, but I'll leave that to the developers to sort.

btw, I posted those various "perspectives" as a first pass, there could certainly be others.

Thoughts?

[magicback (Frank)]

57.

@Csven and @Allen,

The whole concept of standard of metrics and the desire for easy to use small set of numbers (under five data points like GDP and population to get GDP per Capita) is a good goal in both RL and VW.

In RL, we got UN departments to get every member nation to use a common set of metrics. In VW, I think we need something similar.

As VWs are much more diverse than in RL, what could be of use is the small set of common numbers Cven desire and for the quant people a set of normalization index references to get everyone close to an apples-to-apples basis.

People is going to argue about the normalization methods used, but hey that's par for the course and good for improving the metrics.

Frank

[randolfe_]

58.

In RL GDP is a tortured measure of inter country comparison, where something as simple as home-ownership is interpreted differently in various Western economies national-income-accounting.

RL is much *more* diverse than VWs in this regard. In fact, since all VWs are subordinate to RL economies, PPP need not be established and comparisons can all be made in dollars and dollar foreign exchanges.

The BEA website has all NIA data available, along with methods, formulae and analyses which are used in producing US GDP metrics. If people insist upon measuring "GDP" from VWs then they should consider it fortunate that VWs are much simpler than RL economies.

Or, then again, we could just measure the RMT investment and profit and be done with it. Why complicate the metrics with dubious macroeconomic analogies?

[Allen Sligar]

59.

@csven

I did generally explain what we're doing (although not directly) in my post just below yours that you linked.

I'll try to address as best I can your points again, on this thread later in the weekend. However we start closed beta tomorrow and I need to work all night tonight :(

Also I do want to get a data set for Sl going thats relavent to the VW space. I think this thread will help immensly in that.

For the time being understand that Gamers Like Me is a contributory social networking site for gamers/VW participants. Encompassing not only user generated content, but in the spirit of Web 2.0 (sorry Andy lol) user generated data.

I look forward to adding you to our beta testing pool in a few weeks, we'll need more contributors from the VW space.

Allen

[Ace Albion]

60.

Yesterday, one person spent $15usd equivalent in my SL store, buying a bunch of houses "to see which one I liked the best". They'll probably junk most if not all of them. That's in one day, at one very small time content vendor, on a Thursday, not accounting for all the other activities this person may be involved in (and paying for) in SL.

So if you're looking at SL's numbers to determine how and if it's worth your investment as a company, or worth recommending to a company as an area of investement, you may want to bear in mind that a good amount of the *paying* residents are paying a lot- they think nothing of dropping $100usd on a patch of virtual scrubland, or tipping some dancer/dj $10-20usd on one night because they're a bit drunk or like to be seen as wealthy and generous. I don't think you get that in WoW, and counting eyeballs is probably only any good if your business only involves being seen.

[Clay Shirky]

61.

@Ace,

So if you're looking at SL's numbers to determine how and if it's worth your investment as a company, or worth recommending to a company as an area of investment...

Though be careful with this kind of advice. Prok posted a video of an SL exec talking about 7K people "making a living" in SL, but here making a living counts as having gross income of $1 or more.

That works well in a hobbyist framework, where the expenses are incurred as a labor of love, but when it moves onto a business footing, the cost of the time has to be factored in. If we calculate the costs of a single worker, who works for 10 hours a month at US minimum wage, the number of people reported by Linden with positive income, net of that relatively minor labor expense, would fall by 80%.

[randolfe_]

62.

Ace,

So if you're looking at SL's numbers to determine how and if it's worth your investment as a company,

As an investor I only care about any of that as a function of their market position and customer acquisition. I care about this very little unless it translates into earnings for the company; specifically in the form of free cash flows.

Even if people are generating "economic activity" within SL, that is as irrelevant to an external investor as what people are talking about on cell-phones is to a cell phone company. It only matters if it drives utility leading to "network effect" growth (and then only if it yields).

The amount of real money being made in SL is minuscule anyway. There is not near enough realized USD transaction volume to sustain anything more than "dabbling" except for a small handful of operators. I strongly suspect that the real-dollar realized after tax earnings of SL "Entrepreneurs" is usually well below their transactional costs and invested opportunity costs. And don't forget you have to file a Schedule C for all those lindens you cash out, paying both sides of FICA/FUTA and/or estimated taxes if you're modestly successful. All that for effectively below-minimum-wage ($7.50 in CA; $8.00 next year) spoils. Whew! Maybe a real-job isn't all that bad after all.

[csven]

63.

@Allen - I've been so focused on the Economic portion I neglected to go back and reread that (something I'd intended to do). Thanks for the reminder. I'll also have to give this some additional thought.

@Ace - I'm with randolfe. I hear lots of comments about people making a living doing business inside SL but I've wondered how many are a) tracking their expenditures - including time - and realizing that they could make a *better* living slinging burgers and b) properly filing their tax returns. In addition, SL residents are not equal. Someone "making a living" on SL's economy while living in the U.S. is making more money than someone living in, say, India. Apples-and-oranges.

[Andy Havens]

64.

@randolfe, who said: "Even if people are generating "economic activity" within SL, that is as irrelevant to an external investor as what people are talking about on cell-phones is to a cell phone company. It only matters if it drives utility leading to "network effect" growth (and then only if it yields)."

I disagree. Because the economic activity taking place inside SL is, in effect, a major part of "the content" of what makes SL interesting for the users. Measuring the internal economic activity in SL will be a way of determining many things about the dollars that will pop out. Even if $L wasn't transferable into $US, measuring those transactions would be a good way to tell many things about the healthy of the system.

And, just so we're clear... when I was in cellular, we were *incredibly* interested in what people were talking about on their cell phones. Why? Because it helped us determine where we were in various market/customer cycles. Now, we couldn't directly listen in, so instead we did psychographic research.

For example: in the early days, when cell phones were very much a "rich man's toy," the first group of road-warrior, non $100,000/year workers to begin making extensive, daily use of the service was... real estate agents. Mobile phones so vastly improved their ability to do their jobs, they became early adopters of a work-style change that predescesed many other market segments by a good 3 years.

So, for example... right now in SL you have lots of "doll house" fun going on. People spending money on clothes, hair, furniture, bling, skins, etc. Even if all/most of that "game economics" doesn't end up going back to investors, you watch. You learn. You measure. Because, at some point, some smart guy is (maybe) going to figure out the next thing that will bring a whole new layer of players into SL and get them to spend 10X as much money on the next level of "stuff" in a manner that will allow much more external/internal cash flow. Maybe it's some kind of educational system. Maybe it's virtual conferencing. Maybe it's TV/movie set prototyping. Maybe it's a one-to-many teledildonics system. Who knows?

But not being interested in customers' conversations? Oh my, no... You never learn more than when you eavesdrop on those.

[csven]

65.

@Andy - I think an integrated browser could be that game breaker.

[Andy Havens]

66.

@csven: Could very well be. The ability to incorporate Web content "in game" in SL is going to be hugely important whether or not it yield external profit for investors (though I think it will). The overlay of any medium into another ends up shedding unexpected results.

For example, anyone with any kind of Web-creation ability (blog, wiki, etc.) will be able to program a point-to-point teleportation network based on whatever criteria they like in-world, using SLurls. So if I have a blog with "Our Clan's Nightly Stomp Events," posted as a top-link, and then embed that in a prim... and a nice neighborhood club in SL agrees to be a place that will post a "ring" of "Stomp" page posts... all of a sudden you're leveraging a mash-up of blogs, SLurls, prims and pubs. Excellent. Same with MP3s, YouTube vids, etc. Whatever capabilities are transferable through the in-game browser.

At which point some of the objections to the, "A 3D space isn't as good as a 2D web for some things" come down. Because you embed the 2D in the 3D. Which doesn't kill all the objections. Just mute some.

[randolfe_]

67.

Andy

Spoken like a true marketer. Just for the record, I've been in telecom for nearly two decades. In the early days, up until the mid 90s, there were very strict regulatory firewalls which prevented (or should have prevented) you using any content information derived from the regulated voice networks within competitive markets, unless you're talking about one of the few pure-plays. Even then, you would have only been able to operate off of data derived only from within the cell network; anything transferred off of land lines would have fallen under what we now call the 272 restrictions.

And to clarify, I did not say that internal SL behavior data (which I still contend is not macroeconomic by any definition) is not interesting to the company. But it is not to external investors. If you can find anything more than a casual reference to "what our customers are talking about", let alone hard data, presented by any of the myriad telecos to the investment community, I shall stand corrected. My recollections of hundreds of quarterly calls is nothing more detailed than customer-acquisition-costs, retention and churn rates, and customer-lifetime-value.

[Andy Havens]

68.

@randolfe: We should get our terms straight. Because, in an abstract sense, many investors only care about, "Put X-dollars in the top of the funnel, Y-dollars come out the bottom." There are many investors who don't care about the specifics (or generalities) or even the morality of the companies in which they invest. If they can get 5% more cream out of pork futures than MMOs... away they'll go. But once an investor has money in a particular venture, I don't see that he's any less "internal" to the project than the company itself. Stock price goes down, investor loses money. If it's important to the company, it's important to an investor who's already in the hole.

As to listening in. I said: "...we couldn't directly listen in, so instead we did psychographic research."

We did focus groups and surveys. We listened to what our customers were telling us in our stores and on service calls and in the call queues. I was using "conversations" as a metaphor. When I said we were interested in what our customers were talking about, I in no way meant that we eavesdropped on any individual call. Not the "content" specifics of each or any conversation, but what they were trying to accomplish with the service. And we could find out quite a bit about that through the appropriate means available to us.

So... when our "customers were talking about" safety (in those surveys and service calls; not on their private calls), we listened and added roadside assistance services. When they "were talking about" wanting "phone bling," we listened and started adding funky phone accessories.

In the same way, I believe, seeing and measuring internal economic activity in SL should be an important "conversational sign" to investors that users are doing something different than users in, say, Puzzle Pirates or Guild Wars. The willingness of users to engage in micro-transactions related to the upkeep of virtual lifestyles is a different kind of behavior. Better? Worse? More or less profitable? I don't know. Not yet. But it's a major part of what makes SL somewhat unique. And I'd be willing to bet that the real dollars that pop out end up tracking in some interesting ways to the internal economies. It's like the overall health of a mall or a parking garage; it's going to depend on the health of the traffic nearby.

[Allen Sligar]

69.

@Andy
@magicback (Frank)

You both should email me....

[randolfe_]

70.

Andy

I apologize for missing your 'conversation' metaphor. Nonetheless, the firewalls as late as 1996 disallowed us from allowing even the most rudimentary customer data to pass between reg and nonreg sides of the house. I used to make a living designing marketing & sales systems compliant with those regs.

I agree with most of your logic. My original contention was about one of consistent, accurate industry self-reporting metrics. The information you are describing is not going to be disseminated to the public -- not even by public corporations -- in any consistent fashion. In fact, were I even an arm's length investor in a public corp. which published the type of information you're getting at, I'd be more concerned about the strategic responsibility of the directors than of the information contained within said data. AT&T won't tell you their detailed network trends either. They release only that which they are (a) required to under FAS and (b) think has strategic advantage vis-a-vis marketing message, signaling to competitors, or misleading competitors.

And as an investor in a private corporation, especially within a dynamic young industry group, I fully expect my portfolio company to behave exactly as LL is doing. Give out whatever data, in whatever form, of whatever accuracy they see advantageous to their own market growth and dominance. There'll be plenty of time for activist investors, 10Ks, and overpaid arrogant "analysts" later -- after I and the other investors have gone into harvest mode.

I think overall we probably agree on more than we disagree on. I just get my hackles up whenever I hear all these virtual-world-economy claims. Probably because I bought the hype and got burned by the reality.

[magicback (Frank)]

71.

I think I should make this comment in the SL thread, but what the heck...

This whole issue and the development over SL numbers remind me of the noise over the statement that video games generate more revenue than box office movie revenues in one year. Clarity over this statement and the numbers behind came much after the statement have made its round in the media.

And per randolfe_'s suggestion, ESA and the market forecasters did put out more clearer numbers to the public in a kind of industry self-reporting/self-lobbying manner. Investors and the public eat these data up like they ate up the initial tobacco data.

So, while industry standard self-reporting numbers are nice, independent third-party auditing service would be better. Magazine industry have bureau that audit circulation numbers, so why not MMOs.

Allen, I hope this service will be part of your business in the near future. I'll be contacting you.

Frank

[Andy Havens]

72.

@randolfe: Just to cover my arse in case various attorneys general are listening in ;-)...

I do not mean to say directly or to imply in any way that any of the wireless telcos I was affiliated with did anything that was in any way even *close* to violating any of the customer data restriction regs you're discussing. My point in saying that we were interested in what our customers were "saying" was entirely metaphoric; we were interested in *how* they were using our service; not just the raw numbers, but the uses to which they put the service. That information was often obtained through qualitative research that had nothing to do with actual service data; again, things like surveys, interviews, etc.

Investors themselves may not have had access to any of the company and customer specific data, either. But they could read industry reports and could see which companies were responding to changes in the demo. Which companies were providing better solutions for new users and new uses. That's "listening" to the market, eh?

The gist of my argument is that customer behavior has "qualities" that may differ significantly from an economic standpoint despite possible similarities in "quantity."

For example, a "lifestyle" and a "business" customer may spend the exact same amount of money and time using a particular service; let's say high-speed Internet dial-up. But the uses to which they put that service -- the internal qualities of use -- will differ considerably. And, therefore, so will the reasons for initial sign-up, the things that affect churn, the products you can cross-sell, etc.

If I'm playing SL as an alternative to an MMO, let's say, I may be very unlikely to consider involving myself in in-game economic purchases. I'm used to an experience that has me interacting with others in many ways, but not through micropayments. If one of SL's core business strategies is to compete with other MMOs (and we keep comparing their numbers to WoW, for example), then measuring this internal issue will be something that allows us to measure the importance of the feature in terms of attracting (or repelling) possible switchers.

Now... that will be a question for any product feature: does it help pull compared to its competitors? The question becomes, in this case, does the fact that one of the service features of SL track so closely to external, real economics provide a more transparent, friction-free, feature-hook?

I don't know, but I suspect that the answer is yes. If you have a movie about football, I suspect that football fans will be more likely to see it. If you have a service where one of the features involves spending money in-game in order to enjoy the experience more fully, I'd guess that the service will have one more way to attract revenue than a game that, all things being equal, does not. And measuring the levels of in-game economic activity (per user, by operational overhead, etc. etc.) would be one way to see if that feature does, in fact, track well to external profit.

I'd be less inclined to think that many other internal SL features are as important in aggregate. I don't care as much, for instance, that you can't have 200-ft tall avatars. It's interesting, but if they changed that rule, I don't see that tracking avatar height would be one of the things I'd want on my marketing dashboard as an investor. Internal economic activity though? I'm interested.

[Allen Sligar]

73.

@Frank

"So, while industry standard self-reporting numbers are nice, independent third-party auditing service would be better. Magazine industry have bureau that audit circulation numbers, so why not MMOs."

As a gamer, and data miner thats the same exact question I asked myself last May....

Problem solved via Web 2.0

PS: check your email :)

[Prokofy Neva]

74.

I think I'm beginning to see more of the dimensions of Shirky's daily 15-minutes-of-hate of Second Life. I had said it must be about it "not being a game," and about it leaving the confines of his control, as an old lifer on the Web 1.0 social software scene, and also leaving the bounds of social software to be more connected to real life and RL meaning.

He says, "Prok posted a video of an SL exec talking about 7K people "making a living" in SL, but here making a living counts as having gross income of $1 or more."

and then "That works well in a hobbyist framework, where the expenses are incurred as a labor of love, but when it moves onto a business footing, the cost of the time has to be factored in. If we calculate the costs of a single worker, who works for 10 hours a month at US minimum wage, the number of people reported by Linden with positive income, net of that relatively minor labor expense, would fall by 80%."

So here we're finally getting at the problem.

Clay hates it that SL isn't a game and doesn't fit in the ludology orbit of Terra Nova with which he's comfortable (and he's not the only poster on TN with that problem) and he hates it when it can't be "enough" like RL to pay actual RL costs of RL people. Gosh, damned if you do, damned if you don't.

The "exec" merely referred to her own statistics page which shows 7,000 avatars making a profit -- and BTW, the Lindens fluff THAT number by a) refusing to put in land sales as a business (which is just downright ideological and sectarian, and part of their bias and feting of content-makers and dissing of land developers) and b) putting these numbers as gross revenue not counting *tier payments* to Linden Lab. If you took away the profits after tier payments, they'd even be more small, though still impressive for some people. I'm someone who makes a profit AFTER tier payments and AFTER supporting non-profit sims so I understand just how hellaciously hard this is -- and there's only one way you do it -- not billing your hours, or billing them at McDonald's wages or less.

And that's fine.

Because it's virtuality, not reality, and it's in its early stages.

Clay can't seem to wrap his mind about this evolving technology that is not-a-game but isn't as robust -- yet -- as RL.

I remember when Ed Castranova said very matter-of-factly at SOP II that having a panel that year in 2005 about "virtual stock markets" and talking about Cyberland and the Metaverse Stock Market in SL would seem silly 10 years from now, because by then, the barrier between "those real stock markets in real life that are all electronic in their funds movement now" and "those virtual stock markets inside games and worlds" would be blurred if not erased. The panel wouldn't be about "those little game stock markets you play penny stocks in" but "all stock markets in the Metaverse" (at least, that's how I understood his thinking and I think he makes similar points in his book on synthetic worlds).

I think it's tremendously important that a pixel seamstress Simone Stern can make a living wage of $60,000 a year, and pay herself something like $18/hour US, even after tier and expenses. It's not great money; she'd probably make more in RL as a designer. But it's phenomenal none the less. It's tremendously important that people can make even $1.00 US out of a nothing pet rock in the sky, people. This is news. This is important. They're online anyway playing games. If they can now monetarize it, all to the good!

Inded, I wouldn't get involved in SL if I couldn't monetarize my time online studying the world, and I am fortunate that I can keep SL tabbed down in the other window and work on other things like translations and news articles so I don't have to think of every hour physically logged into SL as a burned hour in which I was paid $6.00 US and not $60 US.

The figure of 80 percent then, isn't correct, not for me, and not for many. Because any content creator with a vendor can leave his vendor out kachinging all day in SL and never log on. He can do that for 30 and 60 days and still make money, and merely come to the website to give instructions to move his funds to PayPal. So the zealous application of the Shirky shearing of numbers here is misapplied.

Yes, we've established that SL is very hard to measure. But lots and lots of imformation -- more than any other game or world company provides -- is listed on the website under "economic statistics". Furthermore, Philip Linden, by his own admission, is a real obsessive about database queries and fun searches of even the most arcane type. You can easily write to [email protected], an email he really answers, and ask him stuff and he actually will now and then run the dbase query or settle some arcane point of yours, he's that kind of guy, obsessed with the numbers probably more than you are because it's his thing that he hopes to make profitable.

I continue to maintain that precisely because people make and spend real money in SL, even if in miniaturized, micro-payment amounts as it is now

What I think some of you are failing to grasp is that this is how it's going to be, as I've been saying on my blog for ages.

They'll run the whole thing as a Metaverse.

You'll drive into the gas station, play some little game or answer some little crowd sourcing query or survey about consumer habits or whatever, and get some little Linden-dollar type simoleon that will eventually be collected and offset your gas price.

You'll write a review on amazon.com and they will monetarize your time online for you by giving you not even a credit on their books, which you might not want, but game currency in your favourite online game.

You'll be able to go the grocery store and use your Second Life debit card to pay for your food.

And so on. It will all be intertwined, it will all be everywhere.

[Andy Havens]

75.

@Prok: It's actually already kinda happening. A January 2005 article in "The Economist" (requires subscription) talked about how the US Dollar was dethroned last year as the #1 currency used in global money trading. It was knocked off that lofty perch by...

Frequent flyer miles.

Which sound an awful lot like "Linden-dollar type simoleons that will eventually be collected and offset your [airline ticket or other crap] price[s]."

Airlines and 3rd party companies are now trading miles at a rate that exceeds the volume of dollars or any other state-backed currency trading.

What's so different about, "Fly 20,000 miles and earn a free night in a hotel," vs. "Rent a virtual island for 3 months and earn unlimited free prim hair."

[Ace Albion]

76.

Well, Clay and Randolfe seemed to miss my point entirely, so I guess I wasn't clear enough.

My point wasn't about who is making what kind of money in SL right now, or that tired Linden schtick about earning real wages from SL, it's about who is *spending* it. When I make a comment about people tipping a DJ $10-$20, I'm not saying "Form up a DJ company the tips are good" I'm saying people burn money in SL just to look generous and wealthy, or to feel that way.

SL spenders like to throw money around. That has to be worth something in comparison to other MMO populations. The 36-100k spending in SL may be worth "a million" typical MMO subscribers just because of the crazy money they spend. And you want to count eyeballs? Here's a count for you- how many eyeballs *inside of* WoW does Nissan get?

[Prokofy Neva]

77.

Oh, that's brilliant Ace, yes. Count the eyeballs available for brands, and calculate their spending. And that information is available on the same statistics page:

Monthly Spending by Amount (2006 December)
Transaction Size Residents
1 - 500 L$ 59,980
501 - 2,000 L$ 23,858
2001 - 5,000 L$ 16,960
5,001 - 10,000 L$ 12,456
10,001 - 50,000 L$ 21,515
50,001 - 100,000 L$ 4,758
100,001 - 500,000 L$ 3,770
500,001 - 1,000,000 L$ 414
Over 1,000,000 L$ 397
Total Customers Spending Money In-World 144,108

Hey, there's a little number of real people we've all been ignoring shown per 30 days: 144,108.

It's very hard not to be in SL and not spend even a $1.

It's an interesting number because most people aren't going to spend much on an alt.

http://secondlife.com/whatis/economy_stats.php

[Ace Albion]

78.

I just caught up with Tateru's blog, which has some conservative (based on taking only 40% of transactions as "real") numbers.

http://dwellonit.blogspot.com/2007/01/usd-spent.html

$15 per day per avatar. Compared with $15 per month for WoW, that makes an SL avatar potentially 30x more financially active than a WoW one when participating in their chosen entertainment form. 36 x 30 is around that nice 1 million mark too.

[Prokofy Neva]

79.

Lawrence Linden once gave a talk where he estimated the average spending per average four hour log on was $1500 Linden, which at the time I recall was about 300/$1.00 or $4.50 US. It seemed way too much to me at the time -- keep in mind that while we're all familiar with people who can do stuff like buy the $4000 US sim off the mainland auction (!) or drop $10,000 Lindens on hair and shoes for their sim cuddle-bunny, most people fly around and spent pennies.

I would say go back to the numbers on the site and work the averages and see what we really have.

[Ace Albion]

80.

Yes, I agree that averages are just averages, and it's really about the spendthrifts and the paupers. That's the reason behind why I don't see much point in comparing SL numbers to WoW ones if you're trying to find ways of getting money out of these people (the point of investing?). Unless the only way you have in mind is "make WoW again but get them to pay me not Blizzard".

[randolfe_]

81.

A January 2005 article in "The Economist" (requires subscription) talked about how the US Dollar was dethroned last year as the #1 currency used in global money trading. It was knocked off that lofty perch by...

Frequent flyer miles.

Not to split hairs, I practically worship The Economist, but...there are in excess of $7trillion in global frequent flier miles?

Airlines and 3rd party companies are now trading miles at a rate that exceeds the volume of dollars or any other state-backed currency trading.

What rate are we measuring? (this is a thread about metrics)

And given the relative health of the world's airline operators, perhaps this proves my point about the potential irrelevance of enabled-economic-activity versus the benefit of the operating company(ies). I seem to recall back in b school that the frequent-flier programs were subject of many cases relating to how wrong things can go (price discrimination strategy turned game theory nightmare, creating numerous hostage customers which actively destroy customer-lifetime-value).

[KirkJobSluder]

82.

@Andy: Whatever capabilities are transferable through the in-game browser.

At which point some of the objections to the, "A 3D space isn't as good as a 2D web for some things" come down. Because you embed the 2D in the 3D. Which doesn't kill all the objections. Just mute some.

Which I think is missing the forest for the trees.

The only reason that we have a "2D web" is because when these systems were prototyped, we had cheap CRT screens, but we didn't have cheap voice synthesis, intelligent semantic agents, or a smart phone (relatively speaking) in every pocket.

Central to the goals of hypertext and hypermedia is media independence. By using a notation that provides automated systems with clues about the meanings within a given media artifact, that artifact can be presented to the user as typeset text, spoken language with inflection and emphasis, or a summary created by an intelligent agent.

Which is one reason why the Web 1.0/2.0 claims drive me up the wall. It has always been about collaborative, social, multi-media, multi-modal and ubiquitous discourse. It's not always been implemented that way, but one can just as easily produce a "monologue" with splines and polygons as with text. The problems of creating computer-mediated discourses have much less to do with technology and media than with politics.

I think Prokofy has a good point in describing how a "Metaverse" would involve bi-directional economic exchanges. I think it would also demand bi-directional and multi-modal information exchanges. I would be able to engage in dialog with you in a Virtual World using a large number of devices beyond just a computer monitor. I could be using (thinking about the future) a voice headset, a telephone keypad, electronic paper with 256 greyscale, adaptive software such as JAWS, an amateur radio bridge, or a 3D projection setting.

[KirkJobSluder]

83.

And now that I'm back from the meeting I rushed off to.

For the time being, computer interfaces for mass consumption are limited to about 1700cm^2 divided into about a million pixels. Given such limitations, designers will (probably) be forced to break the 3D metaphor by using 2D displays such as pop-ups, palettes, and HUDs to display some types of information. Which is not a bad thing because I think the possibilities for augmented reality outweigh those of simulated reality.

The other side of the 2D/3D discussion is that there are some complex relationships between human expression and communications media. Virtual World discourse has a rich, and interdependent existence with other forms of discourse. It is unlikely to gobble up those other forms. (And I don't really think that discussion is best summarized as 2D/3D.)

[csven]

84.

So where are we? I'd still like to see us arrive at a set of metrics that can - as best possible under the circumstances - provide a more "apples-to-apples" comparison across videogames and virtual worlds. Even if we can't currently determine the numbers for a particular metric, it'd be good to know that *that* metric is one we'd all like to see included.

[Andy Havens]

85.

@KirkJobSluder: The only reason that we have a "2D web" is because when these systems were prototyped, we had cheap CRT screens, but we didn't have cheap voice synthesis, intelligent semantic agents, or a smart phone (relatively speaking) in every pocket.

If that were true, then the only reason that all print books don't have copious, full-color illustrations and/or photographs is because printing monochrome text was/is cheaper.

We often confuse "different" with "better." I like reading the "Harry Potter" books. I like watching the "Harry Potter" movies, too. Some people I know prefer to consume Harry in one media but not the other. There are books I don't like the movies of, and vice versa. There are many books you couldn't and shouldn't do movies of. There is much text content that is best codified in 2D. There is much that is done now in 2D that will, I believe, move into 3D when that is available.

I didn't mean in any way to limit the discourse to 2D/3D issues. One of the objections Clay brought to the game in his articles was that "3D is a lousy way to shop." When we measure the effect of features on behavior, we'll want to know, for example, if the interface shifts between "web" and "thing-not-web" are inhibiting economic behaviors. If we can embed/mash-up a web into a thing-not-web... objection overruled. That was my only point at this point.

[KirkJobSluder]

86.

@csven: Well, one of the problems is that such metrics would depend on the social phenomenon you want to study. For me, I'd be interested in more social-network oriented statistics or looking at the number and types of messages between people. The latter would be useful for identifying the 90-9-1 curve. But I'm not just interested in comparing across VWs and Games, but also contrasting with other social software types.

@Andy: If that were true, then the only reason that all print books don't have copious, full-color illustrations and/or photographs is because printing monochrome text was/is cheaper.

Well, actually, the primary reason that all print books don't have copious, full-color illustrations is because monochrome is cheaper.

But, it is the true that SGML and early versions of HTML were intended to be completely ignorant of the final medium used by the audience, with speech synthesis included as a possible future medium. For a variety of technical and design reasons, we have the web as it is now. But conceptually the web does not assume a human audience using a browser and CRT/LCD screen.

In most cases, the people who write don't have much of a say in the final published design of the document. This is a good thing, because seeing Times+Arial or pages of Courier in a published work makes my heart sob. The typewriter-friendly manuscript submission format is also media agnostic to some degree.

Perhaps I'm being a bit too aggressive in defending the WWW as a concept, but I'm seeing too much straw-dog criticism of "the old web" lately. I'll wear my ideology on my sleeve here and say that I'm frustrated by the fragmentation of social software research and development by discipline and by software paradigm.

[Andy Havens]

87.

@KirkJobSluder: First of all, beyond a certain print run size, there is effectively no price difference between B/W and color anymore. If it would sell more books, they'd all have color illustrations. For any particular pub, it may be marginally cheaper to run short-run works on a 2-color press, but once you get into the other associated print costs -- set-up, design, binding, press time, shipping, storage, marketing, shelf-space -- the cost difference between 2/c and 4/c almost disappears. Of course you have to pay more (usually) to create or purchase the full-color images, and then laying them out takes time, and that costs money... but monochrome isn't, really, much cheaper per se anymore. Most paper dailies (including the ever discussed USA Today) made the switch to 4c a few years ago because the digital pre-press environment made 4c set-up and registration just about as easy/cheap as 2c.

That being said... Do you really think that all print would be improved by full-color imagery? Do you read novels and think, "If only this had more pictures?" Do you read economic or scientific papers, or psychological/sociological theories and wonder, "Why aren't there photos of the people being described?"

Text is good for many things. And beautiful text is beautiful with or without images, and can be ruined by bad typography, enhanced by great imagery or overpowered by bad design.

I'm not going after the Web at all. Not sure where you got that from. I love the Web. The flexibility is amazing. What's being done now with Java and AJAX and Ruby, etc. is so far beyond what we were thinking could be done on the Web 10 years ago that it almost scares me.

And Second Life being "outside" (or next to?) the Web may end up being what eventually (if left unchanged) kills it, more than goofy numbers, flying penii or in-world tax shenanigans. Because that's a barrier that doesn't exist for many other kinds of services that SL mimics on some level.

On the other hand, if SL is to remain "gamey," then maybe it's OK for there to be a wall. A "Fourth Wall." That good ol' magic circle. Because we don't want too much permeability between our first and second lives. A web site is a web site is a web site. But there's only one Azeroth (OK, lottsa shards, but you get my drift). The uniqueness of a virtual space may, in fact, be what makes it a "world" and not just a thing in our world.

That's partly why I argue that MySpace is not a VW. It's a service. It's highly social. But it's too connected with our world; too linked to Target.com and YouTube and Google and porn and sports and bands and and and and. How is that a "world." It's a part of "the" world.

It's a shard of reality. Not an alt.

Now I've made my head hurt. So I'm going to go play XBox.

[csven]

88.

@KirkJobSluder who said "Well, one of the problems is that such metrics would depend on the social phenomenon you want to study."

Which is exactly the point I was making earlier. Feel free to contribute what you think is a worthwhile metric to allow you to make the comparisons you wish to make and which you feel is worthwhile. I want to know and compare how much money a User spends per virtual space; hence my modification to the original breakout.

[KirkJobSluder]

89.

@Andy: That being said... Do you really think that all print would be improved by full-color imagery?

I didn't say that, and I'm not certain where this tangent fits in. And I'm not certain that we really have a disagreement here. Or rather, the disagreement comes because we are talking about two entirely different aspects of communication. I don't see the web as a medium, I see it as a publication process.

In my view, it is limiting to talk about a "2D web" because the web was explicitly designed to support whatever presentation method could be created and desired by an end user. The 2D-ness and 3D-ness is entirely in the design of the user agent.

Since we are going to talk about media. I will say that there is one ideal medium for all text, including all web objects. This is a medium that defines cool, and if adopted would make everything so much better. And that medium can be nothing, if not dramatic performance by surrealist robotic automatons crafted from antique doll parts and kitchen implements.

But as a person who is collaboratively creating a web object in the form of this response, I can't force readers Terra Nova to abandon the fashionable green border and light gray datelines in favor of toy soldiers and tiki mugs expressing ominous portents of the coming year. So I'll just suggest that I should be represented in this discussion by a sleepy-eye doll mounted on a glass butter churn filled with vegan sausage.

@csven: Which is exactly the point I was making earlier. Feel free to contribute what you think is a worthwhile metric to allow you to make the comparisons you wish to make and which you feel is worthwhile. I want to know and compare how much money a User spends per virtual space; hence my modification to the original breakout.

Honestly, I've approached my questions using several metrics and have not found a satisfactory one yet. My suggestion is that looking at discourse connections provides an estimate of the scale of community which can be compared with other online communities that don't use the same kind of pay models, or have a monetary economy. But online community researchers have been stuck on these questions for over a decade.

[Andy Havens]

90.

@Kirk: Cool is only ideal if cool fits the bill. A perfectly cool medium that allowed for perfect recipient manipulation of the stream wouldn't necessarily be the best for many teaching modalities.

I hosted a bulletin board once where the default install included an "unacceptable words" option. My fault for not checking out all the administrator features, but... the way it worked was to substitute any of a string of "bad" words with other, less "offensive" ones.

One of the bad words was "dick." The substitute word on the list was "manhood."

Problem is... the bulletin board in question was related to the 2004 election. So every post (until I figured out the problem) was filtering every mention of the Republican vice presidential candidate and changing his name from "Dick Cheney" to "Manhood Cheney."

Funny, yes. "Cool," in the McLuhen sense, yes, in that it allowed for an interesting interaction between administrator and users. But, really... subtractive from the process.

Cool is generally better. And more *options* are always better. But just because you have an option (another "D") doesn't mean you need to leave it turned on all the time.

And, yes... I don't think we're actually disagreeing. Just having a banana goldfish in a solid-silver bathtub between primitive nutcrackers.

Your Friend,

Jimminy Cricket

[IAmEric]

91.

Speaking of ethnographic research, this looks pretty cool:

Turning the Second Life client into an interviewer terminal

[Prokofy Neva]

92.

I had to ask the Lindens this morning why it would be ok to make an OS viewer to scape marketing data and interview avatars who may not realize they are subjects in a marketing survey (the issue of consent wasn't explained), but that readers of TN have to jump through hoops to get a sociology research project within SL approved. Makes no sense.

I also want to take this opportunity to:

1) page Clay Shirky now that WoW claims 8 million to ask whether there are any penetrating statements about WoW to be made given the lack of full disclosure about the architecture of this number and

2) note that Zee Linden posted new statistics for landowners this week that surprise even me, 48,000, when it was just 42,000 the other day.

Some people think these are goldfarming-type accounts to scarf up the Linden-subsidized first-land. But since they've been out of the first land for weeks, that wouldn't account for it.

[Mario Menti]

93.

@Prokofy: if the issue of consent wasn't explained, then that's my fault, and is mainly due to this being a proof-of-concept, rather than an application currently being used.

*If* it turned into something more than that, then a user of this type of application would typically be a market research agency bound by legislation and guidelines laid down by the professional bodies they are part of (e.g. covering respondent consent and data protection issues). I don't doubt that these guidelines haven't fully caught up with virtual world issues yet, which is in fact something I believe market research bodies should start considering, as this type of survey activity is inevitably going to happen.

But to sum up - what you saw on the video on my site is a techie proof of concept, which explains why it misses much of the non-technical context that would be needed if it went one or more steps further.

[Nobody Fugazi]

94.

""Belle Isbell" is the interviewer coming up to me, and once I say ok to take part in the survey, she becomes a data collection program - until the end of the survey, when she thanks me and walks away." -- Turning the Second Life client into an interviewer terminal

Saying 'ok to take part in a survey' seems like informed consent. A 'survey' is pretty well known as a data collection technique.

I don't *like* it, but it is what it is - and what it is does not seem to be a ToS violation or even unethical. People have a choice.

[Ace Albion]

95.

http://secondlife.com/knowledgebase/article.php?id=062

Is the link provided as recently as three weeks ago relating to Linden Lab's research ethics guidelines.

I guess it's open season now.

[Allen]

96.

Did we get any clunclusion or summarization about the metrics based on the discussion on this thread?

[Ace Albion]

97.

Everyone got too busy whacking the all new new colored shinier rats in WoW, or failed to pay their internet bills when they got stung too hard by the Great SL Ponzi Scheme.

[csven]

98.

Ace said: "Did we get any clunclusion or summarization about the metrics based on the discussion on this thread?
...
Everyone got too busy whacking the all new new colored shinier rats in WoW, or failed to pay their internet bills when they got stung too hard by the Great SL Ponzi Scheme."

Having just recently returned from a cross-country trip (I departed a day after my last post), I was wondering the same thing as I get myself caught up. However, from what I'm seeing, I guess that in spite of all the yelling and screaming about numbers, actually finding some partially-acceptable means of comparison on which most people could agree wasn't really all that important.

[Alvis Brigis]

99.

A new service, Lifecrawler has just launched. It allows SL users to stream their experiences to the web. Check out the piece on 3pointd. How does this figure into the metrics of use? Over time, this technology will enable tons of external SL users and dramatically extend SL's reach.

[Andy Havens]

100.

@Alvin: Cool. Combine this with live-voice-chat (can't tell if Lifecrawler will stream audio, but somebody will eventually) and you've got live, amateur machinima.

YouSecondLiveTube.