Mimicry, Melting Faces, and The Proteus Effect

Those of you who know that The Daedalus Project isn't my main research project at Stanford may have wondered what it is I actually do at Stanford. Here's the short version. I melt faces.

At Stanford's Virtual Human Interaction Lab, I work with Jeremy Bailenson in using immersive VR (iVR) to explore social interaction and self-representation in digital environments.  The over-arching theme is something we refer to as Transformed Social Interaction (Bailenson et al., 2004) . Given that digital environments allow us to break the bounds of physical rules of interaction, what can we change and how do these changes affect social interaction in digital environments? For example, in RL, we can only maintain eye contact with one person at a time, but my avatar in VR can maintain eye contact with everyone at the same time (because every person I interact with sees their own version of the world).

Here are three different lines of research that have gotten media coverage lately.

Mimicry

We unconsciously mimic people in real-life, and in fact, the more someone mimics us the more we like them. In our study (Bailenson & Yee, 2005), we tried to explore whether agents could leverage this advantage against people in digital environments. Participants were put into iVR and listened to an agent deliver a 3 minute argument. In the treatment condition, the agent mimicked the participant's head movements at a 4 second delay. In the control condition, we played back the head movement from a previous participant. We found that participants who were mimicked by the agent were more likely to agree with the argument. These participants also rated the agent more favorably. In other words, mimicry offers one simple way for agents to gain social advantage with human users.

This study has also been reported in the NewScientist and Wired.

Melting Faces

We like people who are similar to us, whether this is because they share our values, because they speak the same language, or because they look like us. Image morphing software has been around for a while, so what would happen if a political candidate took on some of a person's facial features? One week before the 2004 election, we took a representative US sample of voting age people (via Knowledge Networks) and blended their face with either Bush or Kerry (40% subject + 60% candidate). Participants always saw a split-panel image - with one candidate morphed with them, and the other image morphed with another participant (so that we could make sure that whatever effects we found weren't simply due to morphing artifacts). Not one of the participants detected the self-morph, but there was a significant effect of the self-similarity manipulation. People were more likely to vote for the candidate that they had been morphed with (see Bailenson, Iyengar, and Yee, 2005 for details and morph examples).

This presidential study built on some earlier work (Bailenson, Garland, Iyengar, Yee, 2005). Related news coverage can be found at The Washington Post and near the end of this other article at The Washington Post.

The Proteus Effect

Virtual environments let us alter our appearances dramatically, but as we choose our avatars, do our avatars change how we behave and interact with others? In one study, participants placed in iVR were shown an attractive or unattractive avatar in a virtual mirror (see video of example of mirror). The attractiveness of these faces had been pretested. The participants then interacted with a confederate in the same virtual room. The confederate was blind to condition and always saw the participant with an average face. We found that participants in the attractive condition were willing to move closer to the confederate and revealed more personal information than participants in the unattractive condition. More importantly, when we asked participants to guess the goal of the study, not one detected or guessed the attractiveness manipulation. In other words, participants in attractive avatars became more friendly and more intimate with virtual strangers.

In another study, we manipulated the height of the participant's avatar to be 10cm taller, 10cm shorter, or the same height as the confederate. Again, the confederate was blind to condition and always saw the participant to be the same height (trig was used in background to correct gaze angles). The participant and confederate then engaged in a negotiation task. We found that participants in the tall condition were significantly more willing to make unfair offers than participants in the short condition. Also, participants in the short condition were significantly more willing to accept an unfair offer than those in the tall condition. And again, participants asked to guess the goal of the study did not detect the height manipulation.

In other words, even though choosing our avatars is oftentimes framed as a one-way process, the avatars we choose actually come to change how we behave in cyberspace (paper under review). These findings have been reported in the NewScientist, NPR, and Gamasutra.

A full list of published papers from the lab (with PDF links) is available here.

Comments

[Bart Stewart]

1.

Fascinating stuff, Nick, for both the behavioral theory insights and the possible practical applications for virtual world design. Thanks for posting this.

Actually, that applications aspect is a little scary. It's no secret that humans can be open to behavioral manipulation under the right conditions, but how far should such knowledge be applied in virtual world design? Suppose a MMOG were to be consciously designed to apply many of the kinds of effects you've documented (as opposed to the current semi-conscious application of some of these effects) -- at what point do there begin to be ethical questions? And what happens if the participants in such a MMOG do figure out they're being manipulated?

But those aren't the questions I really wanted to ask.

Here's what I wonder: Each of the three investigations you cited here seemed (and please correct me if I'm wrong about this) to confirm a pre-held belief on the part of the investigators. We (Westerners, at any rate) like people who act like us; we like people who look like us; we favor attractive people and tall people. These things -- as you point out in your links -- have already been demonstrated in other circumstances, so it's completely understandable to expect them to hold as well in sufficiently immersive online virtual (iVR) circumstances.

That takes nothing away from your work. As I said, it's fascinating to see how strongly these behaviors still operate even in virtual worlds.

But it does lead me to wonder: Are there any experiments you run that don't yield the expected results that would be considered worthy of publication? What investigations in iVR haven't shown the same effects as similar studies in RL?

It seems to me that those results might be even more interesting than confirmations of expected results. Not just because of the Popperian "falsification of hypotheses" approach to figuring out how things work (although that's important), and not just because it minimizes the perception of investigator bias (though that's important, too), but because it's the surprises that produce the most valuable results in scientific research.

There's a quote attributed to Isaac Asimov that's relevant here. To expand it slightly, the exciting breakthroughs in science generally don't happen with someone running down the street shouting, "Eureka!" but with someone alone in a lab late at night muttering, "Hmm, that's funny...."

I'd like to hear more about investigator beliefs that haven't been confirmed....

--Bart

[Nick Yee]

2.

Hey Bart - Great questions. I think the experimental paradigm is supposed to deductive, but I think in practice it always ends up being both inductive and deductive.

So the main findings of The Proteus Effect were actually supposed to be the pilot study to a more elaborate study (framed by a different theoreotical background). We weren't thinking about The Proteus Effect (the way it is framed now) when we ran those studies. And originally, we had theories that predicted contradictory things. So for example, a person's height impacts confidence most during adolesence (when self-concept is forming). So people who were tall when they are teenagers (but relatively short now) tend to be more confident than people who were tall when they were teenagers (but relatively short now). So we had reasons to hypothesize that a 30 sec in a virtual mirror (or height change) would have no effect.

That there was such a strong effect in the attractiveness study was the "hmmm - that's funny" moment for us. Afterall, it would have been as easy to argue the opposite findings - people's self-concept are relatively fixed after adolescence (and psychological therapy in general shows great resistance to changes to self-hood). That's what led us to the height study which had even stronger differences. And then, we said, ok - what's going on here?

The other surprising thing across all these studies was the 0 to low detection rates. No one detected the face melting, or the attractiveness manipulation. When we started the studies, we were very worried that rates of detection would be so high as to problematize our findings.

Of all these non-detection rates, the face melting is most surprising. Think about it - you bring in 35% of someone's face into a photograph of Bush/Kerry, and no one in the sample detected it.

[ChrisBateman]

3.

This really is fascinating work - looking forward to picking up the research papers if I ever find a spare moment.

Some information I'd like to see in connection with this is how the subjects were sourced, as this could bring in biases in your sample sets, and this in turn could have an effect on detection rates. Not that the detection rates are the core of the work, but all the same - I would personally need more information on the test subjects before concluding the lack of detection was significant.

Related to this, just how many test subjects were involved in each study?

In fact, I think this is a general problem in people-oriented research in general; studying how stage hypnotists select subjects for performances suggests strongly that treating all people as interchangeable elements in a study may not be a sound approach for all topics of investigation. It's unfortunate that it adds too much of an overhead to these kinds of studies to include personality metrics in the data, but without it the results always risk being slightly incomplete.

On the other hand, that's the nature of science right there: all results are by the very nature of the process incomplete. :)

Take care!

[Nick Yee]

4.

For the mimicry and proteus studies, we used Stanford undergrads (about 16-20 participants in each condition). For the face morphing study, we used a nationally representative voting sample (using Knowledge Networks) - about 250 people in sample.

But yeah - the tradeoffs between control and generalizability is an inherent tension in quantitative methods. (After all, what population of people do Stanford undergrads generalize to?)

[Andy Havens]

5.

The one thing about this that I find most interesting (and kinda scary), is the idea that my avatar and/or VR experience of myself and my surroundings in a VW may be completely different from another person's experience of my avatar and/or surroundings... despite our sharing the same space. And -- more to the point -- that those differences might be specifically controlled in order to generate desired responses in users.

There is always, of course, a perceptual difference between what two or more people see/experience in any given scene, even in RL. You are you, I am me. You are there, I am here. Your "version" of the event will be different than mine. Perceptual drift; very old story.

But what's new, here, is that a third "someone" -- the programmer/producer/director -- is now having some level of direct control over both of our perceptions. So while it may be that our perceptual drift is only as great as it might be in RL -- if there is no directorial interference -- there is room, as this post describes -- for a wide variety of such interference.

Suppose, for example, we accept the proposition that attractive avatars are less likely to be attacked in PvP. Suppose the game designers want to discourage PvP attacks. Now, I may design my avatar to look very, very fierce. And my avatar will look -- on my screen -- exactly as I have designed it. But the system may be programmed to portray my avatar *more attractively* to other players. Fewer piercings, less scowly, shorter, bigger anime eyes; I have no idea. But let's just say it's based on research like that done by Nick: this set of avatar features is less likely to be attacked. So, no matter what your avatar looks like to you, to others, it will appear "smoother." Thus cutting down on PvP.

Why do that? Perhaps the designers have discovered that fewer PvP instances lead to longer subscriptions. I don't know. This is just an example.

What worries me a bit more than this kind of virtual-reality-bending in games (per se), is what will happen when we start using more realisitc avatars for more "mundane" activities? Banking, working, education, social events, etc. If a system or engineer or controller can, in some way, "distort our lens" without our knowing it... that's messed up.

And what about hacks/bugs/viruses that make your avatars and/or interactive agents more/less attractive? Again... weird stuff.

[Bart Stewart]

6.

Nick, those are just the kinds of things I wondered about. Thanks for the additional information.

And Andy, I think the questions you're asking are darned good ones. In any field, when a new technology grants someone more power over how other people perceive and experience their world, questions of ethics need to be addressed. It's true in modern medicine as we gain more control over life, and it looks to be true for virtual worlds as well.

As developers increase their ability to control in real time the experience of participants in virtual worlds, at some point we may need something like a code of ethics that describes how this power will be used. Unregulated power doesn't always corrupt, but that's not the way to bet.

I should add that researchers who point out these handles on human perception aren't necessarily doing anything wrong thereby. They're illuminating parts of what it means to be human, and that's generally a Good Thing. Even so, it's valuable to see acknowledgement from researchers that there's an ethical component to their work when that work can be used to manipulate people. It may not be their responsibility to dictate how their work will be applied, but they are in a good position to offer recommendations for ethical usage.

"'Once the rockets go up, who cares where they come down? That's not my department,' says Werner von Braun." -- Tom Lehrer, "Werner von Braun"

--Bart

[Ralph]

7.

What intrigues me is the possibility of allowing other researchers to experience (and collaboratively improve and develop) much closer to the same testing conditions as those used in the studies.

It hadn't occurred to me that digital research lends itself much more readily accessible to peer review. I imagine in the near term there will still be input/output device irregularities, but how long will it be before you can publish your studies as an entire observable interaction on a desktop computer or via a web service?

Not to get off on a tangent but it brings to mind an interesting topic -- the perception of time, and how immersive an experience it would take to make varying distortions of time unnoticeable.

[N. Ng]

8.

Nick,

This is fascinating stuff. I'm curious to see whether the behavioural effects carry over when engaging in relatively-rich interaction with NPCs in a single-user environment (think Facade).

When is the Proteus paper likely to be available? I take it it'll be available at the link you provided.

Thx.

[Nick Yee]

9.

Hi N. - The Proteus paper is still under review. It's been presented in abbreviated form at conferences. If you'd like a copy of the current manuscript, email me at: contact AT nickyee DOT com

[M.]

10.

I did notice in my stint in EQ that trolls and ogres were very rare. However, on the racewar server where they were grouped together as a team with the dark elves against the "good" races, it was usually very competent or even very deadly players who took up these races. They also tended to be more professional and closely knit when it came to PvP. The "chaff" of the evil team mostly gravitated towards the dark elves, the "pretty" evil race.

Stats didn't do much for characters in EQ when I was playing, it was pretty much equipment. So what you're looking at is at most a 5% or so edge as a troll or ogre against "weaker" races, with barbarians being a "prettier" race that shortened that gap even further and who also possessed the ability to slam without a shield that really set the large races apart. The most conspicuous trait of the ogres and trolls (other than how ugly they were) was how big they were; and, speaking mechanics, this was always a disadvantage.

But I personally found it quite charming to smash three higher level "lighty" characters to dust as a giant ogre with mis-aligned teeth in a stupid grin with two different colored eyes and craggy toenails sticking out of my chain mail, and in the end I thought much more of the ogres and trolls than of the dark elves, and the race became an exclusive (and, in my eyes at least) exalted brotherhood that was made so by the choice of its non-members.