A new study from Craig Anderson and colleagues at Iowa State University claims to be definitive proof that violent video games cause aggression. The claim is based on a meta analysis of studies in the media effects paradigm, which has been discussed here before. There will debate about the validity of the study's conclusions, just as there has been about the individual studies on which this meta-analysis is based: Were samples properly chosen? Can you learn anything from a study of correlations? Do short-term effects matter? And so on.
The debate misses the point. For the sake of argument, let's grant every scientific claim that Anderson and colleagues make. According to Professor Anderson, exposing a person to a violent video game makes the person feel more aggressive. As to the size of this effect, Professor Anderson expresses it thus: "These are not huge effects--not on the order of joining a gang vs. not joining a gang. But these effects are also not trivial in size."
Very well. Let's say we have 10,000 perfect studies showing positive effects of videogame violence on individual aggression, and these effects are big enough to show up on the radar consistently without being the biggest blip anyone's ever seen. If this were the state of the world (I don't think it is, but again, for the sake of argument), what would we make of it?
Professor Anderson: "From a public-policy standpoint, it's time to get off the question of, 'Are there real and serious effects?' That's been answered and answered repeatedly. It's now time to move on to a more constructive question like, 'How do we make it easier for parents--within the limits of culture, society and law--to provide a healthier childhood for their kids?'"
Hear hear! I say. And yet - what an odd time to be saying this. Why would a professor ever decide to study a question that, by his own admission, is not very "constructive?"
Indeed, why has there ever been MediaFX research? I ask this because it's fairly obvious to me that nothing in the current MediaFX paradigm is relevant to policy issues. The questions asked in this paradigm are not relevant to decisions we make in the real world. This problem is revealed most clearly, as I have said many times before, in the paradigm's obsession with statistical significance. Researchers in this paradigm are looking for an "effect" without having any idea what an "effect" might mean. Put another way, there is a difference between statistical and substantive significance. Policy assertions - such as the assertion that we must do something so that parents can help their kids make healthy choices - are substantive assertions. There are many substantive proposals on the table. Regulate game sales. Intervene in parent-child relationships. Have a videogame safety program in the schools. Restrict and codify content. Restrict advertising. Massage social norms through PSAs. There are many things that can be done. We should only do them if there is substantive evidence in their favor. But the MediaFX paradigm does not talk about any of these things. It only talks about "statistically significant effects of Medium A on Individual Feeling B."
Results like this, by themselves are devoid of meaning. What evidence do they provide? Consider a thought experiment. Suppose you have N studies showing that exposure to videogame violence causes an X% increase in aggressive thoughts. It is mathematically true - as part of the mechanics of statistics - that for any value of X, there exists a value of N such that X is "statistically significant." In other words, no matter what X is - no matter how substantively small or big the aggression effect may be - you can cook it into a "statistically significant" effect simply by adding N. If your meta-analysis of 50 studies does not result in a 2% average effect appearing to be "statistically significant," a meta-analysis of 500 or 500 million studies will. Just keep adding studies until you get the magic asterisk of statistical significance. The asterisk is sufficient to satisfy professors and editors. But real people in the real world balk, and properly so, at the effect size of X. "Two percent?!?!?!" they say. "That's tiny!" And they are right. It is tiny. It may be statistically significant, but it is not interesting from a policy perspective. (And with due respect to those trained to believe so, Cohen's d does not solve this problem. No statistic spit out by a program or algorithm can replace the human mind in assessing whether a number matters.)
Thus, the evidence provided by the MediaFX paradigm is not ab initio substantive evidence. We have to take the evidence that's been produced and inquire further: Is this substantively meaningful?
Let's consider: Is it substantively meaningful that someone exposed to a violent videogame feels X% more aggressive in the aftermath, where X is a small but reliably non-zero number? Well, let's consider the result in light of policies that might be enacted in the real world. Suppose we had a PSA campaign against violent videogames. Suppose this campaign cost $20m. Suppose it reduced exposure to violent videogames by 15%. I am being generous here in assuming that a low-cost government policy could actually have a large effect on behavior, an assumption not warranted by any evidence I am aware of. But let's just say. Fine, we've spent $20m and the number of people exposed to violent videogames has fallen by 15%. Is the world a better place? Sure, er, maybe. According to the evidence, these 15% of game players are now X% less aggressive. I guess that's a good thing in and of itself. Or is it? What does it mean? How important is that change? Does this change make our world $20m better off? Is it worth $20m to have reduced post-gaming aggressive feelings of 15% of gamers by X%? If you want to say yes, you have to explain to me how that X% reduction in aggression among those people has made the world a better place.
Now, that's the tough sell, isn't it? Is there anything in the research showing that a policy of reducing violent videogame playing has an effect on the social level of violence? No. Is there evidence in the paradigm that people who choose to play fewer violent videogames gain substantive benefits in other areas of their lives as a causal result (and not just a correlation)? No. Is there evidence that an effects size of X would be sufficient to make a policy of cost Y pass a benefit-cost test? No. Not only is there no evidence; there can be no evidence, not the way the paradigm is constructed. The paradigm is constructed in such a way that these issues never even appear. There is no evidence on substantive questions in the MediaFX paradigm not because of sample size problems or failures to perform random assignment or lack of longitudinal analysis, the things these debates always run around. There is no evidence on substantive questions because substantive questions are not asked. Period.
The MediaFX paradigm generally does not provide evidence on substantive questions. It simply asks "Did watching this TV show make you feel different today?" Why does that matter? Who knows.
How ironic, then, that Professor Anderson now says that it's time to "get off" the effects question and move on to research on the policy question, which he seems to view as "What's the best way to force parents to save their children from this evil thing?" I am eagerly awaiting his work here. But as far as I'm concerned, he should have started working on that eons ago; that's the only relevant question to begin with; one wonders why researchers ever "got on" the MediaFX question in the first place. What has it produced? The studies so far are not tales told by idiots, but they are certainly full of sound and fury, and even more certainly, they signify nothing.
As a new reader of your blog, and not knowing the cultural tone, you do a good job at rightly deconstructing the argument - some video games cause aggression. Like you, I say so what!
As a 54-year old player of COD6 (although COD4 is better) I do walk away from experiencing the game being aggressive. Okay? Usually I then migrate to my next daily item of the day. (No, not kicking small dog).
Your argument is solid - so what.
Posted by: Jaeger Vollmar TSL | Mar 31, 2010 at 21:52
Thank you for this insightful post Edward. As a game studies scholar who was trained in media studies, including many years learning the mediaFX canon, I understand and welcome scholarly inquiry about media influence. While I find the methodologies of the mediaFX tradition extremely limited (as you point out, even the "best" results do not offer usable evidence toward tangible policy decisions), I celebrated one statement Anderson made in his latest release. It IS time to move beyond a yes or no answer to media effect/influence both in academic circles and in the public discourse. But doing so is proving very difficult.
Every time I am interviewed about video game violence, as I was recently regarding Anderson's work, I wrestle with the reporter who (usually) has already written the story before he/she contacts a single source. The narrative option presented to me is either "video games are bad" or "video games are awesome". Thoughtful media studies scholars have struggled with this for decades. How can scholars have a conversation with the public AND each other that assumes video games play important cultural, social, educational, personal and interpersonal roles in our lives, but do so in very complex, nuanced ways? Gamers are, rightfully, defensive about this line of inquiry. Politicians are, too often, opportunistic about Anderson et. al. conclusions. And academics are, sadly, territorial about methodology.
Do video games influence players? Of COURSE they do! But my response to Anderson isn't, "so what?" It is, "now what?"
Posted by: Nina Huntemann | Apr 01, 2010 at 11:35
I'm an intensely peaceful person. Hugely non-violent. And I enjoy very violent videogames (along with ones that aren't). So that may color my view of this. But my thought upon reading this was:
"Wow. Me with a +2% increase in violence is still an incredibly peaceful person."
I've had one fist-fight in my life; in the 4th grade, and I was jumped. I've only ever had about 10 arguments with people that would qualify as such; mostly they're just friendly and amicable discussions, even when we disagree. I just don't take part in anything even remotely violent in real life.
2% increase? That's me flipping one more guy the bird on the highway over the next 8 years.
Posted by: Andy Havens | Apr 01, 2010 at 17:34
This post is so spot-on to the problem with this type of research, which alway assumes that any aggressive feelings are bad feelings and that anything within society that causes aggressive feelings must be removed.
We live in a time that is so much safer and less violent than the times past. Maybe video games are actually part of the reason for this relative peacefulness.
Some aggressive feelings are normal and healthy, and games can be a safe outlet for those feelings. Should we stop kids from watching sports because they might feel aggressive when their team loses? Should we stop them from reading Charlotte's Web because they exhibit a "statistically significant increase in sadness" ?
We're human - let's be allowed to feel a little bit.
Posted by: Kevin Abbott | Apr 08, 2010 at 22:16
Anderson's conclusion "people learn. content matters" emphasizes a factor implied throughout the paper, and possibly the study itself, that the emphasis is on representation rather than the structuring of behavior through game mechanics and rules. Would replacing the graphical and audio layers of a COD 6 multiplayer capture the flag game, for example, with cutesy fluffball creatures lobbing smiley stars of different properties while retaining the exact same rules yield similar results?
Another pertinent question seems to be: if one ran a similar study to measure the effects of basketball, soccer, fencing or jujitsu would we get similar increases in aggression? And if we would, do we also mobilize policy makers to curtail such activities?
Do we then go on to measure the increase in aggressive behavior yielded by extended exposure to traffic? a day in the life of a bank clerk, accountant, used car sales rep? Better still, a night out at the bar/club?
Posted by: Gordon Calleja | Apr 13, 2010 at 06:38