Unobtrusive Personality Assessment

If you’ve ever gone through any kind of personality assessment for a job interview, a group dynamics workshop, or a psychology experiment, you know how transparent those assessment statements read:

- I find it difficult to get down to work.
- I feel I have no control over my life.
- I make friends easily.

Or the worst of all:

- I usually lie if I know I can get away with it.

The reactivity of standardized personality assessment scales is problematic, and clearly if it were easy for personality psychologists to observe people without them knowing and systematically record their behaviors, one would be far more certain about the assessment.

As MMO environments get more complex, more social and more nuanced, the breadth and depth of behaviors and choices we could unobtrusively track from individual users is quite staggering. One could easily track:

- the average number of characters this person interacts with daily
- the average length of conversations / utterances with another character
- frequency this person is in a leadership position (guild / group leader)

In fact, from the first two alone, one could find out how well connected an individual is in the social network. In real life, stealth marketers pay big money to find trend-setters. In a virtual world, you can figure out who the “connectors” are with great precision.

But you would also be able to build incredibly detailed personality profiles of individuals. And not only that, we would also be able to figure out how an individual ranks on those personality scales against a dataset of at least 200k other individuals. That degree of breadth and precision are impossible with traditional personality assessment techniques.

Now, one might argue that the pretense of separate avatar identities and unrealistic settings precludes us from finding out anything “real” about someone in the MMO environment, but that’s not necessarily true. We know that players in existing worlds are emotionally invested in their avatars and the worlds. And it’s precisely because people are personally invested in their avatars and the virtual world that every choice they make becomes personally revealing.

For personality psychologists, being able to track and record the behaviors of thousands of people unobtrusively through an automated process was never possible. Now they can do exactly that.

Comments

[ren]

1.

One of the versions of me does online service design – including the finance and business aspects.

It is certainly the case that, as a gross generalisation, for many subscription based on-line services the subscription really only gets you to break even. What makes the money the meta-data, when this can be individualised and include behaviour it becomes very interesting \ valuable indeed.

As people may have seen there is a lot of activity going on now in tracking P2P activity – this is becoming quite complex linking data from radio play to P2P activities and even planning market research around targeting radio play in specific national areas so comparative data can be gained.

I’ve personally looked into the economics of interactive TV on a global scale and here the value of the ‘click stream’ data was huge, especially given some of the statically tools one can throw at this stuff.

Many services are predicated on the notion that part of the ‘cost’ of the service is that your usage data is provided to advertisers. Given that we know that many EULA state that every key stroke i.e. every conversation is owned by the developer, they already (believe) they have the rights to these data. So the question becomes whether useful data will only be gotten from places such as say There were the brands cross the “imaginary boundary“ or whether behaviour data from places like EQ will have value outside that which game designers get.

Mmmm 1984 the game. . .

Ren

[Jeff Cole]

2.

"Knowing" the network and reliably exploiting it are two very different things. Certainly, identifying "hubs" is very valuable.

But, don't lose sight of the fact that there are many, many, networks. Different information is communicated differently on these networks. Because of their intra-game "hubness," I might well solicit (and heed) from an inta-game "hub" advice on where to hunt or how to upgrade my gaming system. I am much less likely to solicit (or heed) their advice on other issues without some demonstration of extra-game "hubness."

Jeff

[DivineShadow]

3.

Jeff, all,

But within the context of Online Spaces this can be leveraged very easily and effectively. Case in point: Electronic Arts. They run Ultima Online, Earth and Beyond, The Sims Online and soon Ultima X: Odyssey. Specifically, TSO has such a relationship network mappping construct there and it can be queried freely by any player. Everyone can see how many friends/enemies I have and how many friends/enemies they have in turn, as many levels down as they care to dig, with virtual faces and stats along the way. It would be trivial for EA to take this web, pick out the top 10% highest ranking TSO memebers and offer them a free box and 12-month suscription to their upcoming Ultima Online X: Odyssey game.
Not sure how this would impact the TSO userbase, perhaps they could do this to popular ex-TSO members. The power is there, just waiting for a good idea.

[Brask Mumei]

4.

"It would be trivial for EA to take this web, pick out the top 10% highest ranking TSO memebers and offer them a free box and 12-month suscription to their upcoming Ultima Online X: Odyssey game."

Or, a better idea IMO, would be to offer them free subscriptions to TSO. Your high valence customers are the ones you want to keep. People stay in these worlds for as long as they have friends in the world, so high valence players are keeping a disproportionate number of other players locked in.

On the other hand, if you WANT to shut down a world, (say, to transition to a new one) giving the high valence players free boxes & early access may be a good idea.

- Brask Mumei

[DivineShadow]

5.

Brask, perhaps the best thing to give them is a voice. Ask them what they would like, not in terms of trinkets, but in terms of what would they like to see in the game.

Gotta be careful with the free=worthless association!

[Brask Mumei]

6.

Ouch! Excellent point, DivineShadow. One also has to worry about the accountants deciding to impose fixed duties in return for the "free" accounts.

- Brask Mumei

[Raph Koster]

7.

There are at least two large companies out there that are very seriously using social network data accumulated from games, and working directly with folks such as Duncan Watts to examine the data.

Alas, Sony is not one of them. ;)

I don't know of any efforts to monetize the data, and I don't know of the specifics of privacy law regarding how valuable this data may be to third parties.

[Edward Castronova]

8.

Depends on whether they embed RL products in the game-world. If they do, then clicks or views translate into interest and can be used to target marketing. Wrangler might want There's DB just because the game has Levi's in it.

[AFFA]

9.

Going back to the idea that a game can act as a personality assessment, I'm not convinced it would work any better than traditional methods. I act differently on-line than in person. I act differently at work than at play. I don't answer personality test questions honestly. Thus, I believe personality tests are of limited use.

That being said, I think that a game designed specifically to test for moral choices and personality traits would work as well as a traditional test. Which is not to say that I'd trust either one (or even use them if I was hiring).

[Cory Ondrejka]

10.

While I tend to agree with AFFA that just as it is trivial to mislead traditional personality tests and that a game that announced its intention to be a test would suffer from the same defects, behavioral data on a large set of people that can be correlateed with demographic and psychographic data is always interesting -- although there are clearly legal and ethical concerns about using the data without the permission of the participants.

Perhaps more exciting are experiments that parallel those of real world social biologists. The participants are told that they are involved in a study but the instructions are generally a mix of misdirection, bait-and-switch, and outright lies in order to obfuscate the actual goals of the study and to reduce the ability of the subjects to dirty the data. Given game designers' hard earned skills in the areas of misdirection and obfuscation, I would think that this would be fertile ground if only the right introductions could be made.

[Edward Castronova]

11.

And let's not forget that the traits of most interest are often ones we are unconscious of. Many people believe that they act differently online. But psychologists (and therapists) are looking for clues to the deeper self, the one whose character is consistent with both the face worn in RL as well as the face worn in the VW. To them, I imagine, our personalities on- and offline are not quite as different as we think they are.

[Tek]

12.

Yes, that is a good point Ed. It may be more correct to say, "We act different in real life." There, our actions have consequences we can't remedy by turning the power off. So when online, we generally get to do the kind of things we wanted to do in RL but couldn't. People will type things they would normally hold back. As such, I can imagine that analyzing a person's habits in a VW may show what their prime motivations and social tactics are, what that person is at their "core". And if you watch the movie you get to see what happens when it stops spinning (don't, if you haven't yet :).

[Brask Mumei]

13.

"I act differently on-line than in person. I act differently at work than at play. I don't answer personality test questions honestly. Thus, I believe personality tests are of limited use."

There is no face behind the mask - we _are_ the mask we present to others. Most of us believe there is some true-self underneath the patterns of behaviour we exhibit to others, but for all practical purposes that true-self doesn't exist.

I don't think one can fill out personality tests honestly. That doesn't mean they aren't of use. Consider this question:

"You are in a class room of 30 students, split into 6 groups of 5 students each. You notice that while your group is doing the individual assignments individually, the other groups are working together to complete the individual assignments. What do you do?"

Unlike the "Do you feel unmotivated to work?" questions, this is a bit harder to parse. One could whistle-blow, ignore, or convince your own group to also share individual work. None of these are the right answers. No matter how dishonestly someone answers the question, they are still telling you what THEY think you want to hear. This tells you a lot about their understanding of the purpose of the test. Eg, if this were a regulatory agency hiring inspectors, one may want the whistle-blowing slavish obedience to rules. In a university setting, the "ignore" option is likely the best situation. And in an MBA program, failing to collaborate is likely showing a failure to understand to point of the exercise :>

When asking seemingly stupid personality questions, the real question isn't "Do you act..." but "Can you tell which answer is the right one?" Surprisingly, some people apparently can't.

Now, back to the topic of 1984 the MMORPG:
1) The data you get back is REAL data. It does reflect people's ACTUAL choices when value is at stake. Thus, it does accurately reflect peoples "true self". After all, your true self is nothing more than the composite of the masks you present. Arguments about "its not my real self" hold no water, IMHO.
2) The transference of behaviour patterns from online to offline is not guaranteed. Consider betrayal. One may think that someone willing to betray someone online would thus be the sort of person to betray someone in the real world. You could chart this by detecting broken trades, broken guild allegiences, etc. However, you need to go behind the data to see if the person playing realized it was a betrayal. If I betray someone in Paranoia, that isn't a transferrable betrayal as I considered the betrayal a part of the game. Similarly, newbie-killing murderers in UO, if they consider being killed as a newbie a valid part of the game, are not exhibitting the anti-social behaviour most will charge them of. For a less emotionally charged example, consider Quake - when I kill people who haven't yet had a chance to find a good weapon, most would not consider it an anti-social action.

MMORPGs are very complicated games being played for many different reasons. Each player has a radically different idea of what social contract they have entered when they play the game. Some think it is "Kill or be killed", others "Cooperate", etc. Compare this to a card game where usually the game rules AND social rules are laid down at the outset.

I think if you want to build a personality test, you will have much better luck constructing a multi-player game with a very limitted scope and well defined social contract. For example, mod Quake to give it some Uber weapon. Then, see:
1) If people recognize the uber weapon isn't fun so voluntarily stop using it.
2) Stop using it on the request of others on grounds of it being "lame".
3) Keep using it over and over again.

Targetted tests like that, IMO, tell more of transferable personality traits than behaviour on a MMORPG.

- Brask Mumei

[Cory Ondrejka]

14.

The uber weapon test is done all the time when games are released. Per Raph and Will's many delightful talks on the subject, players undertake a massive investigation into the parameter space of the game and rapidly optimize their behavior, exposing broken and unbalanced elements and forcing the developers to release patches, nerf the feature or item, &c.

Obviously, there is still interesting research opportunities there (how closely does player behavior minic tit-for-tac, or any other evolutionary stable strategy, for example).

[AFFA]

15.

I'll conceede that game would be useful for experiments that cannot be conducted in real life. "Breaching" experiements, for instance, are no longer performed at universities (and may even be illegal). However, I believe that getting reliable data would require a game that was set up to achieve this, not simply mining data from existing games.

While researchers think they know more about me than I think, I think they know less than they think. Particularly advertisers, which have a great deal of private psychological research behind them... And yet they cannot convince me to buy things I do not need or live beyond my means.

[Nick Yee]

16.

Echoing Ed's and Brask's comments - the assumption that the 9-5 mask is the "real self" is flawed. We change masks according to who we are interacting with. We don't talk and interact with our spouses the way we do with our parents, our children, our mentors or strangers in the street. But are any of these "masks" any more "real" than the other? The projection of the self in any meaningful context produces clues about the person that are meaningful and interpretable. The fact that virtual life doesn't replicate real life isn't really an issue at all. Wherever you have people who are emotionally invested in a decision or an action in an environment, you can learn something about what drives them.

There's another way to think about this. Standardized personality assessment techniques all rely on self-report measures. Now that we can rely on longitudinal unobtrusive behavioral data, aren't we clearly going to learn more about who an individual really "is"?

[Paul "Phinehas" Schwanz]

17.

Nick> "We don't talk and interact with our spouses the way we do with our parents, our children, our mentors or strangers in the street. But are any of these "masks" any more "real" than the other?"

Drifting further off topic...

Perhaps the "masks" aren't any more real, since you've already defined them as masks. On the other hand, I have relationships with higher levels of committment and trust in which I am much more transparent. Furthermore, I believe that growing toward intimacy in my relationship with my wife involves being ever more transparent and trusting. In this context, taking off the masks and affectations in order to reveal the true self is indeed the goal. If this is just another "mask," then I'm at a loss understand what a coherent concept of intimacy would entail.

--Phin

[Michael Chui]

18.

*puts on philosophy helmet*

I came to a realization of myself about half a week ago: I am the sum of my pretending. Since we're painfully off-topic anyway, I'll explain quickly...

When you speak of "masks", you speak of a set of habits that rise up and squash your "other masks" when interacting with a set of specified persons. Thus, in theory, your "true self" is present only when you believe you are alone, because then you present a mask to no one. Self-delusion aside, that is. =)

But human beings are unquestionably social creatures, and it would probably be erroneous to claim that social creatures are sincere only when not being social. As it happens, I think that every fakery and pretending, every mask or facade is a part of you. It is part of your identity. You would not put on that mask if it was not you. It was your choice to wear that mask, or, in better terms, you chose to adopt that set of habits in the presence of that set of persons.

So, a mask is part of you. Yes, it's only a PART of you, which means it's inaccurate, but no human being can be fully known anyway. The masks he chooses for his time in a MMORPG will tell you a few things, but, in my opinion, it will be based on his intentions for playing.

If this gamer plays MMORPGs for fun, assessing his gameplay will tell you what he thinks is fun.

If this gamer plays MMORPGs to rip off corporations, his gameplay will tell you his ability to handle business aspects in an environment promoted to entertain by immersion.

That's the best I can think of for now. I'm sure someone who's had an actual job can think of better ones. =)

[ren]

19.

Just to be evil_marketing_Ren (again) and adopting an ethics \ smethics approach:

If one believe the EULA then a games company own every single key stroke that a player makes. Hence the own any conversation about anything on any game server, as well as all the behavioural data and meta-data such as social nodes.

How to moneytarise this ? > Data mine the feck out of it !

Given the amount of time spent in virtual worlds the richness and diversity of data for the user group seems mind boggling. Just as the companies that are doing analysis on P2P activity after radio play, with a VW one can could analysis on just about anything as people talk about just about anything but what’s more there are soooo much data to correlate things against. Some marketing person somewhere must be thinking about selling data analysis services against these data by the bucket load.

Ren
www.renreynolds.com