« How Did We Miss This? | Main | Babbage & Cabbage »

Jul 13, 2005

Comments

1.

I'd like to know why current (and presumably) forthcoming titles do not actively use some sort of anti-cheat system ala punkbuster or valves VAC (valves anti-cheat) systems.

2.

Because putting in an anti-cheat system costs money. Most companies don't have the resources of Valve. Even valve put in anti-cheat after the product was a blockbuster, not when half-life was forthcoming.

3.

Imagine a continuum of AI mimicry of players: from bad (NPC) to perfect (Turing Test). Questions:

1.) Are there inflection points: pointing to diminishing returns in terms of player experience?

2.) Is there a "behavioral uncanny valley"?

4.

What the are the general processes required to make a game go "gold"?

How many people have to put their stamp of approval on the program? Seeing as many games have come out recently with large numbers of bugs, or patches have been released that are rolled back because they are so horrible they literally break the game, is there someone who sees these things? Is there a certain number of "acceptable" bugs for release?

How long is the typical life of a game? From production and release to death and end of support?

5.

How do we define 'hardcore' and 'mainstream' and 'casual'? Is there a universal definition, or does it have to be broken down for certain cases? MMOs don't demand much playskill, does that make them casual games? Then again, they require immense time-investment, does that make them hardcore? Is the guy who buys Deer Hunter 5 and Nascar 2006 at Walmart mainstream, or is he a non-gamer? Or is the guy that buys the traditionally accepted big-hits (GTA, HL2, Halo etc) the mainstream? What about the mainstream of people that only play online analouges of real games (scrabble, poker, etc)? How can we define what a mainstream gamer is versus a hardcore gamer, and what can we glean about gaming culture from that definition?

6.

Here is something I'm curious about:

How does persistence in virtual world affect the various Bartle player types? For example, is persistence a feature that benefits explorer-type players? Is it required for the achievers?

-Nathan J

7.

And can the Bartle player-types be supported by research, or are there perhaps other paradigms that co-exist or supercede them?

8.

And it's the questions that drive us... right, Neo? What ever happened to that spoon, anyway? Hrm...

I’ve worked on the service side of the online games industry since 1997, and pretty much since the start have had a hand in virtual world governance. It’s an extremely fascinating - and always challenging - fusion of business, gaming, law, culture, and society.

I placed business first, as it was intentional: with some notable exceptions, most of us in the industry are simply not in the position to use our virtual worlds as test labs or personal sandboxes (which is why the model is often regurgitated, just with some twists). Sometimes the wealth of academic research doesn’t seem applicable because the “business” part of the industry seems overlooked. While many of the concepts and ideas that are proposed are absolutely genius, quite honestly most of the time we struggle just to keep a regular production cycle and get content out the door and into the hands of players.

Even if it made it to the discussion table, often grand plans and desires are usually tempered with fiscal and time restraints. Shareholders/financial backers/the guy who signs the check are understandably not too keen on having their money risked with experimentation.

So for you that really are interested, would it be helpful to have a crash session for academics who want to understand what really goes on from concept to launch (design, planning, production, beta testing, marketing, distribution, etc)? Or on the post-production “live” environment, where the actual business side of operations really kick in (support/legal costs, content updates, bug fixes, etc)?

Perhaps something like this could be created at AGC or at a similar conference?

Finally, is there much research out there is based on mass-statistical data like Nick Yee's Daedalus Project? Is that sort of data collection normal, or is it usually more abstract? As I'd said above, (and sorry I sound like a suit here) we just do not work well on abstracts... perhaps there can be more information and research that devs can really look at and create a real plan on?

Great topic Dmitri!
-Will

9.

For those who play MMOGs: "If you weren't playing this game right now, what would you probably be doing?"

For those who design MMOGs: "If you could have any one specific feature in your next game without regard to time or expense, what would it be?"

For academics: "What one type of data related to virtual worlds would you most like to be able to completely and accurately collect?"

--Flatfingers

10.

Mark - you may be interested in this empirical approach to motivations of play as an alternative to Bartle's Player Types:
http://www.nickyee.com/daedalus/archives/001298.php

As for research questions, my undergrad advisor was a personality psychologist, and we've always mulled over the idea of MMOs as goldmines of personality assessment data. Surveys for personality assessment are just so awful these days (even the Big 5). But here we have longitudinal profiles of players who are deeply invested in their choices and behaviors. Understanding personality via actual behavioral measures would be so much more interesting than survey measures.

11.

What metrics should a virtual world collect/provide in order to make future research possible? Obviously, WoW provides a tool for some data extraction, allowing researchers to examine hours-per-level, etc. What other types of data should be provided?

12.

Nick Yee>you may be interested in this empirical approach to motivations of play as an alternative to Bartle's Player Types:

I still haven't figured out why you think these motivations are an alternative to my player types. They overlap to a great extent, and the parts where they don't overlap fit in with other parts of my theory. You could argue that your results disprove parts of my theory (and I can argue that they don't), but that doesn't make for an alternative theory; rather, it suggests that the existing theory is either outright wrong or in need of repair.

A true alternative would come at the subject from a different angle completely, for example the 3-state model used by role-players. Also, it would be a theory, not high-quality data alone. Theories need whys, not just whats.

I suppose that for people who don't need a theory, your motivations are indeed an alternative to my player types. I'd wonder, though, why people want to be able to categorise players if they can't then use the categorisations for anything.

Richard

13.

Mark Wallace asks:

"And can the Bartle player-types be supported by research, or are there perhaps other paradigms that co-exist or supercede them?"

I'd just like to say there are other behavioural paradigms relating to play. We published the first report on our DGD1 (demographic game design 1) audience model last year, and we have a book coming out next month on the subject.

The big difference between Richard's system and DGD1 is that Richard's system is specifically intended to model online play; our system is intended to identify patterns of game play across all video game entertainment (and, for that matter, board and card games).

James asks for more clarity in the relationship between Casual and Hardcore gamer. This comes down to how you choose to draw your lines. For the DGD1 model, we used a self-analysis to divide respondants into two clusters - Casual and Hardcore. The question was: Do you consider yourself (a) Hardcore (b) Casual (c) Don't know. Anyone responding (a) was counted in the Hardcore cluster, any other answer was considered Casual.

Nicole Lazarro sorted her subjects into the same clusters based upon (if I remember correctly) number of games played per month. We gathered this data alongside of the self-analysis, and agree with Nicole's approach too. Anyone who reglarly buys/plays more than 12 games a year tends to fit the Hardcore pattern. (Indeed, 6 per year is a sufficient boundary condition). I prefer self-assignment as a sorting criterion, though - it strikes me as more honest, as you never declare someone is a Hardcore gamer without their consent.

If you're interested in audience models, then I invite you to my blog. It's not even a month old yet, so I'm still finding my feet, but we could use input and feedback from people with their heads screwed on, which Terra Nova has in spades. The blog is at onlyagame.typepad.com.

Incidentally, our model does not directly correlate with Richard's; neither did the Bartle types model correlate with Myers-Briggs type (which was our starting point). When you cluster people by Myers-Briggs type, a predeliction for Explorer dominates. This is not evidence against the Bartle model - it just means that the behaviours involved don't statistically correlate with Myers-Briggs in any direct fashion.

One final note, I'm in the industry side of the equation, not in the academic side (although I have an academic background). I believe we need audience models for sound commercial reasons - but on a personal level, I believe in trying to support a wide range of play needs... something the industry isn't doing very well right now.

My thanks to the Terra Nova people - it's a great community here.

Chris Bateman
Managing Director
International Hobo Ltd

14.
Richard said: I still haven't figured out why you think these motivations are an alternative to my player types ... Also, it would be a theory, not high-quality data alone. Theories need whys, not just whats.

I've never called the player motivations model a theory. That's always been a comparison you have made. We are coming at the same question with very different approaches though. What I have is a model based on empirical data from over 3000 MMO players. What you have is a theory based on personal experience and conversations with 15 MUD wizards. Hence, alternative.

Richard then said: I suppose that for people who don't need a theory, your motivations are indeed an alternative to my player types. I'd wonder, though, why people want to be able to categorise players if they can't then use the categorisations for anything.

For me, the motivations model has always been intended as a bridge to explore other variables. Having an empirically-grounded assessment tool allows us to actually answer questions about interactions with other variables (such as age, gender, and usage patterns) rather than to simply speculate about them. But indeed, for people who only want speculative stories, there is no need to employ an empirically-grounded tool.

And there are fundamental conceptual and taxonomic differences between what you theorize and what the data shows. But we've had that discussion many times before. We might just have to agree to disagree on this one.

15.

Nick Yee> Understanding personality via actual behavioral measures would be so much more interesting than survey measures.

Hmm. While observation is a useful approach, I'm not sure that makes surveys less useful.

The data you get from both observation and survey depend on the particular setting in which the responses are generated. Just as answers to survey questions will depend on what questions are asked and how they're asked, observed behaviors also occur in a context -- if the context is restricted, the data collected (descriptions of behavior) may only be valid for that context. Watching how someone builds a shed doesn't mean that person enjoys building sheds.

This point about context also pertains to studying gameplaying behavior. In the first place, it already narrows the context to playing games. (Gameplaying behavior doesn't necessarily correspond to general behavior.)

And in the second place, the specific game that respondents play conditions their behaviors. The rules of a game limit what people can do in that game, and subtly or overtly push players toward other behaviors.

How are these effects meaningfully different from the way that the specific questions of a survey limit the range of responses?

Game designer bias == survey designer bias. Observing what players do in a game world may actually tell you more about the lead designer's personality than about the personality of the players!

Finally, a commercial game isn't designed to be a personality assessment tool. But a survey can be tailored to that end.

So while I agree there's value in seeing what people actually do when they play Diablo2 or SWG or ATITD, I'd suggest that this observational approach is a supplement to surveys, not a replacement for them. As long as it remains vastly cheaper to design a survey than it is to design, implement and operate a game, surveys will remain a valuable assessment tool.

Given your own use of surveys, you probably already recognize this; it just seemed worthwhile to note a few of the details.

--Bart

16.

Chris Bateman> We published the first report on our DGD1 (demographic game design 1) audience model last year, and we have a book coming out next month on the subject.

Chris, thanks for joining the discussion.

I was referred several months ago by Lisa Galarneau (thanks, Lisa!) to a .pdf file of a brochure at your International Hobo site describing your model of gamer personality. It was an interesting take on what makes gamers tick; I hope an occasion will arise for you to share more details about it in some appropriate thread here.

There is one thing in your initial message I'd like to comment on:

Chris Bateman> our model does not directly correlate with Richard's; neither did the Bartle types model correlate with Myers-Briggs type (which was our starting point). When you cluster people by Myers-Briggs type, a predeliction for Explorer dominates. This is not evidence against the Bartle model - it just means that the behaviours involved don't statistically correlate with Myers-Briggs in any direct fashion.

Some personal study I've done suggests that while the Bartle types may not correspond with the classic Myers-Briggs types, they do (in my opinion) correlate very well with the four temperaments model of David Keirsey, which in turn correspond to particular groupings of the sixteen Myers-Briggs types:

Killer (Bartle) :: Artisan (Keirsey) :: ISTP, ESTP, ISFP, ESFP
Achiever (Bartle) :: Guardian (Keirsey) :: ISTJ, ESTJ, ISFJ, ESFJ
Explorer (Bartle) :: Rational (Keirsey) :: INTP, ENTP, INTJ, ENTJ
Socializer (Bartle) :: Idealist (Keirsey) :: INFP, ENFP, INFJ, ENFJ

I put together a more detailed exposition of this theory in the Will the Real Explorer Please Stand Up? thread here on Terra Nova if you're curious.

Chris Bateman> on a personal level, I believe in trying to support a wide range of play needs... something the industry isn't doing very well right now.

So true. Here's hoping this can be changed!

--Bart

17.

"Observation" may mean data collected without the subject's knowledge, not necessarily through actual visual observation (we social scientists call it "unobtrusive observation"). The holy grail for guys like Nick and I is to get server-side data on behaviors to then match up to survey questions of our own. Then you get to ask the person how much they like building sheds and also have a hard count of how many sheds they actually built to add to their entry. Put those two together, add some ethnographic work for context, and you can start to answer some tough questions.

So how about those questions? Let's not sweat the methods here. Just blue-sky it, I asked.

18.

Why can't there be big games targeted to women, and why are the games that do exist that appeal to women (animal-oriented games like Virtual Horse Ranch, for instance), completely ignored in this forum and everywhere else, thus ghetto-izing women and perpetuating the falsehood that "women don't play games."

What games DO women play? What makes a female want to play? How much potential market is there for games that appeal more to women than to men? How can the industry get past the Barbie syndrome? What is the significance of the appeal of player housing ot both women and men (Barbie means something, yes, but she may mean something different than everyone has been presuming. She may mean that women like "play dollhouse" in a game; have their own virtual space that they can manipulate.) Is the idea of a game for women like the idea of the romance book genre, something to be disdained but milked for profit?


Etc. There are a billion questions on this topic, if anyone would just notice the pink elephant in the room. :p

19.

While you have some good and valuable questions jj, I'm not sure I agree with your characterization that TerraNova participates in "ghetto-izing" women. Sure, those of use who can't grow beards are much out numbered by the beardy here, but I've never felt marginalized or ignored for being a woman -- and quite the opposite on a few occasions, in good, professional ways.

There is a general lack of gender stereotyping talk here on TN (phrases like "games that... appeal to women [like] animal-oriented games" and "women like "play[ing] dollhouse" in a game" tend to get under my skin, but that's just me), but we do talk about women in games quite often: Women Speak, The WGC, Gender, and MMOGs, Chicks and Joysticks, What Women Want, and Survey: Women Over 40 Dominate Online Games, just to name a few from the past year. And a member of the dev team for Virtual Horse Ranch has been known to post here, so we're far from unaware of VHR and games like it.

There is a great deal of research that could be done into what motivates a wide variety of people to play or not play games, but I think singling out doll house or animal-oriented games as examples of what all women gamers like does more to ghetto-ize women gamers than anything else said here on TerraNova.

20.

Lee Delarm >> What the are the general processes required to make a game go "gold"?

How many people have to put their stamp of approval on the program? Seeing as many games have come out recently with large numbers of bugs, or patches have been released that are rolled back because they are so horrible they literally break the game, is there someone who sees these things? Is there a certain number of "acceptable" bugs for release?


During its final beta testing stages, builds of a game are considered “release candidates”, which are exactly what the name implies. The QA team and hardware compatibility labs test these builds as suggested final versions that could be shipped to players. If it passes, the final release build is placed on a master CD and sent off for reproduction… as soon as that Master CD is sent off, the game is considered to have gone “gold”.

The pass/fail test differs with every company, and usually each product has its own unique circumstances. In the end usually involves a mix of any number of people acting as department representatives, and can include the Producer(s), Lead Designer, Quality Assurance Lead, Director of IT/Operations, Marketing, and other executives at various levels,

Each of those positions weighs in differently, and some carry more weight than others (but politics is nothing new ). For example, during a Go/No Go meeting for a title’s launch:

o The Producer may want to get a game out of the door because of budget or time constraints.
o The Lead Designer may want to hold off to get one more push of content on that Master CD instead of making it patch to version 1.1.
o The QA team may not sign off because of a particular “showstopper”, the term for a critical bug (or it may sign off and note that the game needs an immediate patch).
o The Finance team may want the game out the door to meet stockholder expectations which were set two years earlier, saying the game would be out during a specific quarter (timing is crucial for those big titles).
o The Sales team may push for release by, say, mid-month or retailers would not have free shelf space for another three months.
o The Operations team may say they need another two weeks to get a security flaw fixed.
o The Marketing team may push for immediate release because they have spent the budget, and hype and fervor for the game may slide if there is a delay.
o The Customer Service team may request a delay for more work on tools which will help support the game adequately.

This is just an example scenario, but this might help illustrate all of the dynamics that come into play for a product (or major episode or content patch).

But sometimes, as you referenced, something is just missed completely. No team ever wants to have to restart a server … or worse, to revert a patch… because of something we missed. It’s not because of a lack of effort; it just sometimes happens. However, though we do make mistakes, they’re some of the best learning experiences. Growing pains indeed.

Hope this helps Lee and is at least a little primer into what sort of info youre looking for.

-Will

21.

Bart:

I really want to see a paper on your results! We have been working with Temperament Theory (not Kiersey's version though - I can't take his esoteric names, even though I love Greek mythology). Can you find a way to send it to me? (I'll check the blog link you gave).

I've just knocked up a quick analysis of the data we have to check your hypothesis; this is what I got:

(Apologies for the formatting - I can't seem to get it tabulated).

Ach. Social. Expl. Killer
Rational 722 889 2024 546
Idealist 422 1147 1269 264
Artisan 138 206 282 107
Guardian 259 300 512 175

Rational 17% 21% 48% 13%
Idealist 14% 37% 41% 9%
Artisan 19% 28% 38% 15%
Guardian 21% 24% 41% 14%

As you can see, our data skews badly towards Explorer. That aside, Idealist is 'in the lead' on Socialiser (which fits with the Temperament model, since Idealist is associated with the Diplomatic skill set).

It looks like we may have a big sampling bias in our data. Thankfully, the online data is secondary to our core research. :)

Looking forward to seeing your data!

Chris.

22.

Chris Bateman>I prefer self-assignment as a sorting criterion, though - it strikes me as more honest, as you never declare someone is a Hardcore gamer without their consent.

Yet you seem perfectly happy to declare someone as casual without their consent (ie. the people who answered (c) to "Do you consider yourself (a) Hardcore (b) Casual (c) Don't know").

Richard

23.

Nick Yee>I've never called the player motivations model a theory. That's always been a comparison you have made.

But my player types model is a theory. If you're offering an "alternative" to it, you have to offer a theory. Otherwise, it's like saying a categorisation of animals based on their habitat is an alternative to evolution.

>What I have is a model based on empirical data from over 3000 MMO players.

No, you don't have a model. You have high-quality data from which a model con be constructed, but that's not a model.

>For me, the motivations model has always been intended as a bridge to explore other variables.

This is fair enough, although producing correlations with no explanations has to run out of value eventually. Sooner or later, you're going to have to speculate as to why demographic X has strong preferences for facet Y, and then build a theory to support that.

>But indeed, for people who only want speculative stories, there is no need to employ an empirically-grounded tool.

Enough people have read my original paper, and enough virtual worlds have been developed with its advice in mind, to give weight to the proposition that it isn't all just hot air, and there might actually be some substance behind the speculation.

>We might just have to agree to disagree on this one.

Yes, no need to rake over the same stuff again!

I only posted on the subject this time because you said that your empirical approach was an alternative to my player types; if you'd said "complement" rather than "alternative", I wouldn't have jumped in.

Richard

24.

Bart Stewart>Finally, a commercial game isn't designed to be a personality assessment tool.

Just as an aside, I had an MSc student create a game that was indeed intended to be a personality assessment tool (although it wasn't commercial quality). It was pretty good as determining Myers-Brigg personality types from how an individual played the game and the decisions they made during it, except it was a poor discriminator for one of the axes (the feeling/thinking one, I think).

Richard

25.

I've been involved in public policy research for about 15 years and in that community there is, as here, a difficult academic/practitioner problem. These issues come up also in business schools.

Communities like that have certain institutions to bridge the gaps.

- Journo-zines like Harvard Business Review, Foreign Affairs, and the National Journal are places where academic research is written up in short bits for practitioners. I kind of think of Terra Nova fits this role.

- Governments and businesses regularly make data available, and expend considerable resources to harvest and publish data, while remaining completely agnostic about its use. In our area, Bruce Woodcock's MMOGChart fits that model: the companies let it be known to Bruce what their numbers are, and make no restrictions on what any of us do with them. From this perspective, Dmitri's question, which is laudable for reaching out to developers and implicitly urging them to consider how basic research can help, is in a sense not demanding enough.

Right? One model of research is "We have a question. Tell us what data you need to answer it; we'll give you the data, you'll answer the question, and we'll all be happy." That's the consulting model. The advantages are that it can be done in an environment such as Will describes, where people need to see concrete, profit-relevant results yesterday.

Then there's the VC model: "We're not sure what the questions are, but you've convinced us that there might be something in this dataset that interests us. Why don't you take a look for a few months and further convince us that you've found something we can use." The advantage is that the company, being blase, can prune what data it gives out and not be too concerned about whether it gets anything useful.

But both of these models have a problem. Just as academics may not respect the needs of a running business, the people in business, in these models, don't respect how academic stuff works. Namely: it's not really an input-output system. The value we create typically comes from some unanticipated finding.

If you ask an academic to answer a specific question with a specific set of data, don't be surprised if you don't get your question answered - and that's because, on reflection, the scholar in her sent her off to examine something else that struck her. That other thing is way more interesting, and potentially valuable to you, than the question you initially posed. But you wanted your one question answered, and you gave out a thin stream of data for that only, so she couldn't follow the trail while staying true to your agreement. And off she went.

Almost all of the value generation in academia is grounded in reflection, consideration, and unbounded thinking. A young industry has no time or money for that, of course. But mature businesses and government determined, early in the 20th century, that it would be smart to release as much as possible and sponsor research into everything. And so, today, other than proprietary IP, you can build datasets about almost anything in society using publicly-available sources. Sometimes you have to prove you're a professor doing non-profit research. But it is all there. It is in fact rare that an economist would ever be involved in collecting data - our entire discipline functions under the assumption that someone else collects data. We just use it. And the result has been mountains of empirical research about all kinds of questions that people who are actually doing things never get the time to even think up.

Thus, strangely, it is easier for me to analyze production and prices in America than in Norrath or Azeroth. America is a country of 300 million people. It's kind of a wierd fantasy country, but I think I understand the culture, kind of. Yet even though it is a big, strange, place, it's easier to study the economy there just because the devs gather and release all their data.

Look, people developing and running MOGs today can't even know what questions they need answered. OK, obviously there will be some things they'd like to know. And for that, you can hire consultants (*evil grin*). But I mean the big questions, the research that could put an entirely new mode of development on the map, discover new revenue streams, open whole demographics - you don't know what those questions are.

And we can't tell you, unless you release your data. All of it.

So: what if a MOG decided to release every single user metric they collect: all trades, all communications (participants and length, not content), all movements, all actions, all account activity, all costs, all revenues (some of that is required by annual reporting anyway). Put it all in a database. You can lock it up and make sure only people working in accredited academic institutions can get it, and then only after signing a privacy waiver (our review boards require such things anyway).

Wouldn't that be smart? Then the company's work is not to generate a question and find an academic to solve it. The company's work would be to make sure they watch the research coming out and capitalize on it before others do. And that's not hard to do. There's a thing called 'sponsored research' where you can support a professor's work and, through that support, purchase the right to be the first to develop any IP - without stating what the IP is in advance. THAT's the right model. It works for big pharma. Why not game development?

Can you imagine what this community would produce inside ten years if, say, the PARC/Yee data were expanded to include EVERYTHING about WoW? Wow. And if Blizzard was smart, they'd be supporting people like Nick, so that they could take advantage of what was found.

Even better: give us a server. Sure, we could talk about leasing it or whatever. But then we're paying you, and we, academics, are single practitioners with no budget for such things. We have to get grants or sponsored research contracts to pay for such a thing. Yeah, we can go to General Motors or the State Department and say 'hey, give us a bunch of money that we can give to Mythic for a DAoC server.' And then we fight the fight of "games are not toys" with those folks. And then sometime in 2015, when they've finally gotten it, they release the money. Meanwhile, the organizations that really should be supporting this, the ones with immediate gains sitting right before their noses, the game companies, let 10 years go by without any basic research on the phenomena that make or break their companies.

OK, I know Near Death isn't going to be able to do this. But Blizzard, Mythic, NCSoft, Sony, and Microsoft sure can. I know people - friends - at all these companies. I've talked to them all about this, and received polite shaking of the head in response. Their response makes sense in the current environment: "You want game companies to give out money to academics for unspecified research! Nuts!" But because this is actually a normal mode of business/academic relationship in many mature industries, I think that eventually one of these companies is going to do it, and I predict that that company will win the consolidation wars now going on. (AND will get the industry out of the sequel/licensing rut too.)

So: Don't tell us what you want to know. Give us the data and a few servers, and give us some support, too, and we'll tell you what you've been missing while you've been up to your ears in code, art, forums, and the Gantt chart.

26.

Excellent post, Ted. I don't hold out much hope for major change just now given the insularity of game dev management, but it'll happen eventually. FWIW, we're working with faculty at the UT-Austin McCombs School of Business now... though I'll admit that finding that line between discussing what's uncomfortable yet valuable and what's giving away the store is a difficult one. This is why most MMOG developers, working in one of the most dynamic, ruthlessly Darwinian business environments around, are reluctant to release any data that has even the slightest possibility of hurting them -- or even just of helping someone else.

27.

Richard said, Enough people have read my original paper, and enough virtual worlds have been developed with its advice in mind, to give weight to the proposition that it isn't all just hot air, and there might actually be some substance behind the speculation.

Then, as defender of your model, it's up to you to explain the lack of correspondence with quantitative data such as Nick's. Your model was no doubt helpful in the early days when few people had thought substantively about play styles and preferences and when there was no quantitative data available. But now there are more researchers, new methods, and new concerns as the market broadens and changes. Data and conclusions like those coming from people like Nick Yee and Nicole Lazarro, among others, must be factored into any relevant model of MMO gameplay construction.

Don't get me wrong -- there is great value in non-quantitative models leading the way. LeBlanc and Hunicke's MDA model is one I find especially useful, and we have our own gameplay segmentation model as well. But such models must take into account what's already known. When an ad hoc model no longer correlates with more extensive observational and quantitative data, either the model must change or its adherents must show how it encompasses or somehow answers what is now known.

28.

EdwardSo: Don't tell us what you want to know. Give us the data and a few servers, and give us some support, too, and we'll tell you what you've been missing while you've been up to your ears in code, art, forums, and the Gantt chart.

I completely agree with you, Ed.

I'd even like to see a standardized 'open source' toolkit that researchers could use across game worlds to identify themselves and legitimately acquire game data in a standardized way from any game world that supports the toolkit.

Developers could choose what information they allow the toolkit access to, but at least the means of gathering virtual world behavioral data could be begin to be standardized, allowing cross-world comparisons and comparisons among different researcher groups to be more reasonably drawn.

--Aaron

29.

I'm down with the manifesto, Ted. I'd like to have every developer read that, so let's put it somewhere prominent. You can add me to the "undersigned."

There really are workable models and I fervently want to get the industry working with us, giving us data, reaping rewards and letting us do high-quality research thinking outside the box. I agree that it'll help them long-term, but the Western business mind takes a long time to get to that point. Maybe I need to brush up on my Japanese. Zen-zen dame-deshita.

On that note, Mike, have you considered the sponsored research model Ted outlined? Would it address your concerns?

I was just thinking that a leading wedge might be to get devs to think in terms of self-interest just to get the ball rolling. One or two tangible examples of a researcher creating data that lead to some game or industry improvement might be proof of concept for folks, or something to take to a boss as evidence.

I'd share my current work in progress as a potential example, but I don't want to give away the store either ;)

30.

What do I want to know? Hoho.
What is most carefully hidden?

In the games industry.
Where the money goes in. Where the money goes out. What Mark Felt (supposedly) said to Woodward and B.

In games academia.
Who/what decides what gets thought, taught, conferenced, blogged, written, reviewed, published, cited, admired, demonized, and, most importantly, googled? Social scientist, heal thyself.

All the stuff that is not carefully hidden, that server use stuff, for instance, which Ted talks about, eventually gets known. Sooner or later. But I’m not sure some stuff -- like the above -- ever really gets known. And until we know the hidden stuff, how are we gonna know anything else?


[I am getting some seriously long loading times for TN pages, by the way.]

31.

Mike Sellers>Then, as defender of your model, it's up to you to explain the lack of correspondence with quantitative data such as Nick's.

I don't see a lack of correspondence; in my view, Nick's data supports my theory.

Richard

32.

A real, scientifically valid, long-term, peer-reviewed, large population, multi-site, ethnographic study of the effect of virtual world "play" on political, moral, ethical, etc. worldviews and social behaviors.

From that, a look at how specific design affordances encourage antisocial and/or hostile behavior (ingame and out), and (particularly interesting to me) how specific design affordances encourage socially constructive behavior (ingame and out).

Extended to provide some insight into the question: "what is the most effective way to use virtual world technology to have the greatest positive social impact on the widest population?", and "how may sublte design changes have deep effects on players?"

This type of study will *never* be supported or conducted by the industry, because they are afraid of the answers--which is really shortsighted, in my opinion. If it is ever done at all, it is done for proprietary advantage and not shared. Much as the industry wants to be able to quantify for advertisers how influential their games are on player's consciousness--so they can sell ingame product placement--they do *not* want any solid data that might indicate cumulative negative impact on society from playing socially-disfunctional (i.e., virtually all) commercial games--or to be confronted with the failure to include basic constructive values in game designs.

Independent academic research might provide valuable insights.

Or course, this is not a hard science, but there is still a lot to learn by conducting solid research and sharing findings with the world.

33.

Oh, but it COULD be a hard science!

I like your questions, galiel. I have many of the same.

34.

I agree with Ed.

I want a mega database that different people can plug in and create something like www.boxofficemojo.com for MMOs.

Querying the data may review statistical indicators on such simple question as "what do people want out of a MMO" and maybe even lead to a better understanding of more metaphysical questions of "does AIs have souls" and the other questions Galiel poses.

Frank

35.

Chris Bateman> I really want to see a paper on your results! ... Can you find a way to send it to me?

I wish I had one for you! The writeup over in "Will The Real Explorer Please Stand Up?" is the closest thing I have currently to a sensible essay (since until recently I didn't even know who might conceivably be interested in my perspective on this subject).

Regarding the trends in your online data, I suspect you may be right that some form of bias is affecting the numbers. Myers-Briggs data indicate that there are significantly more Sensing types (Artisans and Guardians) than iNtuitive types (Rationals and Idealists) in the general population. And I find a similar observation made by Richard in his Designing Virtual Worlds -- the Killers and Achievers have tended to outnumber the Explorers and Socializers. (Though that's support only if you buy into my theory of congruence between the Bartle Types and Keirsey's Temperaments.)

For these reasons, I also suspect the number of Socializers in your data is overstated compared to the number of these folks who play the most popular MMOGs (though your broader definition of online games may indeed include more Socializers).

Another point (which Richard also mentions in his book) is that it's tough to get a good handle on Killers since most games actively try to discourage that playstyle. This may be depressing the numbers of Killers showing up in your data.

(Incidentally, may I suggest "Manipulators" as a replacement for the term "Killers"? I believe it better describes the motivation of those who demonstrate the playstyle by not connoting that these people are only Player-Killers. Seen in this light, Politicians and Griefers enjoy manipulating people, while exploiters enjoy manipulating the game's rules.)

And finally, to try to keep Dmitri from swatting me from straying too far from his initial question (*g*):

Although the idea of opening up all of a commercial game's observed data to an academic (even one who signs waivers) seems attractive from the academic side, I think there's an obvious alternative from the biz side: let an existing employee do the data analysis.

It seems much more likely that a publisher will prefer to do analysis in-house than to farm it out to anyone else. In the first place, it's safer to have a demographer taking your paycheck if anything embarrassing is uncovered -- waivers are nice, but who knows what an independent academic might actually say in public?

In the second place, some (most?) publishers will likely think, "Why should I give this data to some high-end academic when I've already got smart people working for me in CS who can analyze my data? I can probably get most of the value at a fraction of the cost, and I don't have to worry as much about the results going to my competitors."

Am I mistaken in believing that most publishers will make this kind of calculation?

If I'm not wrong, how can these folks be persuaded to consider independent academic review of their valuable data?

--Bart

36.

> Yet you seem perfectly happy to declare
> someone as casual without their consent

True enough. :) My experience is that Hardcore is a more sensitive word than Casual.

37.

Bart:

Thanks for your detailed comments! Application of temperament theory to patterns in gameplay is a firm interest of ours.

> It seems much more likely that a publisher
> will prefer to do analysis in-house than
> to farm it out to anyone else.

Would you be shocked to learn that most publishers and developers invest nothing in firm audience models? I have seen evidence of one Japanese publisher having such data (but I've not seen the actual data)... In the West, I've not seen any evidence of anything other than marketing models.

Chris.

38.

Bart >>

Richard's book separates his 4 Player Types into 8 Player Types. That makes it much harder to work the Keirsey Temperament congruence, but probably still possible. I have no idea how, let me show why.

Keirsey separates each of his Temperaments into two per, as well.

Artisans => Entertainers (Feelers); Operators (Thinkers)
Guardians => Administrators (Thinkers); Conservators (Feelers)
Idealists => Advocates (Perceiving); Mentors (Judging)
Rationals => Engineers (Perceiving); Coordinators (Judging)

Unforunately, Keirsey is breaking them down in a completely different way. Bartle has 3 axes; Keirsey has 4. That screws up the entire analysis, giving us 8 Types and 8 Sub-Temperaments, but no easy correlation.

Then it hits me. The other axis Keirsey uses is Introversion/Extroversion. By sheer coincidence, and of no actual pertinence *ducks*, Bartle's third is Implicit/Explicit.

Bartle's Axes: A/N (Acts/iNteracts); P/W (Players/World); I/E (Implicit/Explicit)
Keirsey's Axes: I/E (Introvertive/Extrovertive); N/S (iNtuitive/Sensing); T/F (Thinking/Feeling); J/P (Judging/Perceiving)

Planner (Achiever; AWE) => Supervisor (eStJ); Provider (eSfJ)
Opportunist (Achiever; AWI) => Inspector (iStJ); Protector (eSfJ)
Scientist (Explorer; NWE) => Inventor (eNTp); Fieldmarshal (eNTj)
Hacker (Explorer; NWI) => Architect (iNTp); Mastermind (iNTj)
Networker (Socializer; NPE) => Champion (eNFp); Teacher (eNFj)
Friend (Socializer; NPI) => Healer (iNFp); Counselor (iNFj)
Politician (Killer; APE) => Performer (eSfP); Promoter (eStP)
Griefer (Killer; API) => Composer (iSfP); Crafter (iStP)

This exercise in correlating made me far too tired to actually go through Keirsey's descriptions and double-check it with Bartle's descriptions. Someone else gets to do that. My suspicion is that it WON'T work out, and that no amount of shuffling will get the types to match up perfectly.

39.

Reading this, and probably because some of my current work involves algorithms related to spatial subdivisions and self-organizing maps, I'm reminded of the difference between and octree and heterogeneous cluster analysis methods (really, this is relevant).


Bartle and Kiersey have (in different ways) subdivided the psychological space evenly, like an octree, without regard to the proportion of individuals falling in each octant (or quadrant in Bartle's original terms). This provides a clean taxonomy, but one which makes no a priori reference to actual behavioral distributions.

OTOH Yee's work is more bottom-up, creating descriptions of behavioral classes based on extrapolations from individual data points (albeit with some hypotheses to guide, and thus limit, the data collection). The disadvantage of this method is that it doesn't necessarily create a clean taxonomy; the advantage is that it gives appropriate weight to actual rather than presumptive groupings of behaviors.

It may be that choosing between these is purely a matter of style or preference. Using Yee's data is messier, but then people are messy. Using Bartle's or similar subdivisions provides clear-cut differentiators, but may over-emphasize the taxonomy itself over the prevalence of people or behavioral patterns within each category.

So in the spirit of this topic my question is, whichever one you choose, what makes these or other typologies/taxonomies relevant or useful? As a developer or an academic, what does knowing there are different types of players or different behavioral clusters do for you? How does this knowledge change your behavior?

40.

Speaking from the development side of the equation, having an audience model informs the game design process here at International Hobo. When we first break ground on a concept, one of the first questions is: which audience(s) is this game for? (Usually, 3-4 clusters). That allows the concept for the game to be developed with reference to the play needs of that audience. We're not designing MOGs though, so we don't have many models to choose from (basically, we have Nicole's and our own at the moment!)

41.

I've written a detailed response to Michael's suggested correspondences between Keirsey's eight subtemperaments and Richard's eight subtypes of player.

To try to avoid going too far off-topic here, I've posted that response back in the old Will The Real Explorer Please Stand Up? thread.

--Bart

42.

Bart wrote:
In the second place, some (most?) publishers will likely think, "Why should I give this data to some high-end academic when I've already got smart people working for me in CS who can analyze my data? I can probably get most of the value at a fraction of the cost, and I don't have to worry as much about the results going to my competitors."
*************

If this is a consulting model question, the answer is that some academics have the time and resources to do this better than others and some businesses are better off farming out than others. Personally, I'd like to see anyone beat the talent/price ratio offered by Ph.D. students. Also, my impression--correct me if I'm wrong--is that no one is doing any R&D other than Microsoft, and to some small extent EA via their USC connections. I don't see the industry doing much long-term, outside-the-box research and testing. Also, CS people don't have all the answers. If they did, everyone would love every piece of software ever. There are significant portions of the academy which work on the more human and social side: comm studies, sociologists, inf. systems, etc., etc.

If this is not a consulting model, the answer is "What cost?" Many of us would fund the work via grants at no cost to the dev at all, or simply be doing it as our own regular research jobs.

43.

Almost all the significant R&D that happens in MMOG developers does so in an operational context: what will work and improve the software or service? It's far from a perfect model, but OTOH the rate of technical advancement is pretty fast. Even our research, such as it is, is being done in an operational context for the US Defense Dept. We're pretty far out from some perspectives, but the operational/product grounding is always there.

Dmitri said, some academics have the time and resources to do this better than others and some businesses are better off farming out than others. Personally, I'd like to see anyone beat the talent/price ratio offered by Ph.D. students.

It all depends on what the goal of the research is, what the timeline is, and perhaps most vexing of all, how familiar PhD students are with game industry processes, restrictions, and timelines.

Right or wrong, academia is viewed by most on the industrial side of things as being slow, a year or more behind the times, and interested in esoteric questions that have no bearing on pressing issues in product development. A question that will be answered in a year or two -- much less three or five -- is irrelevant. Most game devs don't have the luxury of looking that far ahead. That may be short-sighted, but it's the current reality.

As I asked before, to me the core thing I want to know is, if you do some research based on one MMOG or another, how will this be significant? What behavior will it change? If it's just in the "that's cool" or "nice to know" category, it's a non-starter. The question isn't "what cost?" The question is "what benefit?"

Before I got into games, I designed user interfaces and taught corporate User-Centered Design courses as a consultant. There the bar was high as well: even when we could show how UCD techniques could cut development costs, cut training and service costs, and increase product longevity, companies were still reluctant to try out something new.

Here the situation is much the same. Sure PhD students are smart and talented. To be blunt, so what? For this to be relevant, that talent has to be able to dovetail with the already fast-paced development environment and bring some new and operationally important result to the table. It sounds harsh, but without that academic work will continue to be viewed from the industrial side as a curiosity at best.

(Disclaimer: I'm speaking in generalities of course; there are some research-oriented game devs out there, but they are few and far between.)

44.

My experience (such as it is) is in line with what Mike described.

Although I haven't worked for a game developer or publisher, I suspect that what I've seen in my employers and what I've had to do as a project manager is probably similar to a game producer's function. Namely, every decision -- and I mean everything -- goes through a cost/benefit filter to minimize risk.

The thing about cost is that it's not always just a financial thing. Having to organize lawyers to draft an NDA or other asset-protection instrument is a cost. Having to create another layer of management to work with the academics is a cost (although some people consider "empire-building" a benefit as well). Having to create a new process for extracting and securely transferring data to the academic team is a cost. Having to supply and possibly even maintain computing hardware and software to the analysis team is a cost (unless they're willing to supply and maintain their own). Having to spend time and effort justifying a long-term project instead of defending other projects that might pay off sooner is a cost. Having to persuade your boss that your company's internal assets should be trusted to external people is a cost.

To an academic, the benefits of exposing internal data for analysis may seem obvious. But producers can't look only at potential benefits. The question every producer asks is, "How can I justify doing this when I've got other things that need doing?" If they're to be persuaded that some action is worth the risk, a serious attempt needs to be made to identify and quantify the likely costs of that action (as well as of inaction) along with the potential benefits.

Showing a producer a detailed document that objectively assesses both the potential benefits and the likely costs is how you persuade him or her that the risks of action have been minimized. Such a document, I think, would be the most powerful tool for making this desirable business/academic partnership happen.

--Bart

45.

What I need to know is "where's the technical equivalent of TN?" :)

I'm a CS prof who reads TN but doesn't have lots to contribute on the social side. I've recently been talking with a colleague who wants to reopen the 'interest management' question that was more-or-less dropped by the academic VR community a decade ago.

(Interest Management is e.g. figuring out for each player what other players are near enough to bother sending movement updates/rendering data over the network)

With so many MMOGs in operation, we'd have to ask the following sorts of questions before we start work to stay honest:

Developers - how well do your interest management implementations work? is the performance adequate? what back-end hardware would you consider? how large a simultaneous server populations do you *want* to support? how important is reliability/accuracy in your game? is dead-reckoning acceptable? what is your target bandwidth per user?

Designers - how well do your interest management tools suit your game? do they limit you in any way?

46.

Tom > What I need to know is "where's the technical equivalent of TN?" :)

It's either the MUD-DEV or v-world tech lists, or both.

47.

Terra Nova

when was the drilling station developed?
when did it start producing oil?
how much oil does it produce a day?
how many people does it employ?
how long it thee oil expected to last?
wjere is the oid processed?
what dangers exist for the drilling stations?

The comments to this entry are closed.