AI Game Masters?

I am at the (American Association for Artificial Intelligence [AI])AAAI-04 Workshop on Challenges in Game AI. A question lurks: is the AI challenge for MMOGs to develop "Game Master" capabilities that can manage/adjust play intelligently to the player experience?

On Robin Hunicke's blog we find reference to a longer conversation along these lines. Is the development of sophisticated character-based AI for MMOGs largely an enterprise of diminishing return for the entertainment MMOG?

Comments

[ren]

1.

Nate > To rephrase this, is development of sophisticated character-based AI for MMOGs largely an enterprise of diminishing return for the entertainment MMOG?

Yes.

But what part of MMO development is not an enterprise of diminishing returns?

But even it if its diminishing returns are we indeed on the right path here.

How much do realistic NPCs actually help a game? Are the not many instances where a vending machine is fine? After all in chess, I don’t want a bishop to actually be a member of the clergy, I just want it to be an inanimate thing with a pointy top – even animating it does not add to chess for me.

If we do want realism then one line of argument is the Synthespians debate we had a while ago.

[Ian Wilson]

2.

Nathan,

I think if I understand your point correctly, in terms of a "Game Master", that rather than sophisticated character based AI being overrated it actually becomes a necessary requirement to enable the introduction of a "Game Master" system.

A Game Master system (or even an empowered player) would be changing the game play based on real time interactions with the players. In any environment of sufficient size these interactions could not, realistically, be defined ahead of time. The game masters primary means of interaction with the players would be through the use of NPCs. What this means is that the NPCs require sufficient behavioral abilities and flexibility to be able to act convincingly (according to their roles) in a very wide variety of highly dynamic situations, in real time without prior knowlege (i.e. a script).

Unfortunately currently employed techniques (i.e. scripting/state machines) do not scale and eventually collapse under the weight of their complexity, circular dependencies, inconstancies, contradictions, fragility and general unmanageability (see AI expert systems). This means that moving from simple, rigid behaviors to flexible, sophisticated behaviors requires moving to the use of techniques that currently the industry has regularly contemplated but rarely implemented because of their “pandoras box” like qualities and simple pressures like schedules and budgets.

When NPCs can be “directed” by high level commands only then I believe a rich Game Master system will become possible otherwise the task will be too overwhelming for system designers during development or designated players in real time.

Ian

[Fred Hapgood]

3.

> is development of sophisticated character-
> based AI for MMOGs largely an enterprise of
> diminishing return for the entertainment MMOG?

Not as far as I am concerned. I am a big fan of
the so-called 'first-person sneaker' games like Thief and in those games, if not in others, interaction with AIs is at the heart of the experience. The cleverer they are, the more they notice, the wider the repetoire of their responses and behaviors. the better. I would be happy to pay double for smarter AI. In fact happy doesn't capture it.

[Doccus]

4.

One of the things I'm looking forward to seeing is one of the features Blizzard announced for World of Warcraft. During army v. army warfare, slots that are "empty" will be taken up by NPCs, evening out the numbers.

This has been one of the most frustrating aspects in PvP gaming. A group of individuals, whether talented or not, can always win if they outnumber the competition 10:1. "Zurg for teh win!" is the phrase most often used to describe this :)

Now, if games could give some minimum level of sanity to their NPCs, making them approximately as powerful as player characters and smarter then newbs, THEN it becomes a test of tactics and skill.

The frustrating aspect of zerging in PvP+ games has led many to leave the genre. At least for the PvP crowd, smarter AI NPCs and better usage of them would do wonders to enhance the gaming experience.

D

[Tony Hoyt]

5.

From my perspective, the point of diminishing returns may not be far off from now, but at the moment, MMOG AI seems to be highly lacking from at least the big budged productions currently active and taking subscriptions.

In many cases, the AI seems to be simplified to deal with Latiency, simplified game play and with a very generalized UI for character control only.

Now, is the problem related to CPU time? Can you make a lot of hyper intelligent Mobs that react well in all situations, or is it an issue of convience. It's hard to make believeable AI, and typicaly it's easy to exploit the more complex you try to make it. So go simple and let player interaction steal the show instead of trying to make the world look 'alive' and instead seem rather 'broken'.

Then again, it could simply be that MMOG's, no matter how complex we think they are now, are still very very very simple still, as they slowly grow up to match the Single player game counterparts in areas they have long dominated at, better user interface to the world and not just the individual characters.

It's easy to make your character cast a spell or swing a sword, But why do they make it so hard to have a NPC that has a conversation branching tree and make it seem interesting?

Then again, many players instantly ignore such concepts of 'story' in mmog's, perhaps making it futile to install them except as alternative devices to develop characters faster.

Tony

[magicback]

6.

Nathan Combs>A question lurks: is the AI challenge for MMOGs to develop "Game Master" capabilities that can manage/adjust play intelligently to the player experience?

I think there is an inflection point where a trend of diminishing return results in a change in trend.

A good example of an AI system design that may yield interesting results in this area is Wish's Live Content system. It's only a GM "storytelling" tool, but it tries to match the best of computer AI with human intelligence.
The end result may be what Ian and Doccus describes.

I hope so.

[Michael Chui]

7.

To rephrase this, is development of sophisticated character-based AI for MMOGs largely an enterprise of diminishing return for the entertainment MMOG?

I could be wrong, but isn't anything in an entertainment industry always an enterprise of diminishing return? You give them one thing over and over again and they'll get bored. Fun seems, to me, to be hinged on some presence of novelty.

[weasel]

8.

Abstracting truly 'intelligent' AI with respect to planning, tactics, and reaction to world changes, away from being tied to individual mobs, is a solid architectural move in line with what was discussed in the Muse of Fire thread a bit ago. Calling such an AI a Game Master is really just a matter of nomenclature. Tactical AI, Societies, Director AI -- all roughly the same concept with slightly different points of focus.

Is the current method of character-based AI an enterprise of diminishing returns? Surely. Is it at a point of diminishing returns? Not yet, but I don't think it's that far away.

The key to GM AI, to stick to that terminology, is that it isn't either/or. We don't have to reach the point of diminshing return for character-based AI to reap the benefits of abstraction. We can start adding group-based behavior and planning AI now, even before we manage to get character-based AI to do proper pathfinding and collision-detection.

Eventually, both avenues of development will converge at similar play experiences: situations like having a stronghold of enemies react intelligently to disturbances and invaders. But would it be easier to code the individual character-based AI guards to sound alarms, hear alarms, coordinate searches, and take defensive positions relative to disturbances; or would it be easier to code an AI routine that watches over the stronghold, that plans and orders approximations of these behaviors and centralizes the event history?

The GM AI watching the stronghold can work on an abstracted data set. It doesn't need to worry about special attacks, tactical movement, collision detection, etc. It just needs to process observations sent to it by the various defenders (under attack at X,Y; unconscious guard at U,V), and issue coordinated orders to the defenders. Similarly each character-based AI needn't worry about trying to coordinate movement with other AI or relaying information-duplicating event histories. It just needs to worry about behaving appropriately within its immediate context.

While the state of the genre certainly still has room to improve on character-based AI, the time is indeed at hand where we can realize more improvements for less effort through AI abstraction.

[Edward Castronova]

9.

Given the personal development sub-theme identified in Bartle's book (we go virtual worlding to learn about ourselves) it seems to me that one of the most critical AI agentes needed is a mentor bot. Something with a bit more depth than the NPC Guild Leader who says "The guild needs you a lot! Go fetch [item] from [person] in [place] and return to me!" and then says "Thank you much! I am so glad you brought us [item]! You are so important to us! To commemorate your great deed, here is your [ceremonial piece of gear with worse stats than junk you can get off an orc]." A great opportunity for affective computing here - give us a bot who cares about our self-exploration and growth.

Beyond that, it's probably healthy to look at problems under the assumption that all efforts eventually encounter diminishing returns.

[Nathan Combs]

10.

Ian>

A Game Master system (or even an empowered player) would be changing the game play based on real time interactions with the players. In any environment of sufficient size these interactions could not, realistically, be defined ahead of time. The game masters primary means of interaction with the players would be through the use of NPCs. What this means is that the NPCs require sufficient behavioral abilities and flexibility to be able to act convincingly (according to their roles) in a very wide variety of highly dynamic situations, in real time without prior knowlege (i.e. a script).

Weasel>

AI a Game Master is really just a matter of nomenclature. Tactical AI, Societies, Director AI -- all roughly the same concept with slightly different points of focus.

A related paper is Robin Hunicke (Vernell Chapman)'s "AI for Dynamic Difficulty Adjustment in Games that she presented at the workshop. The design here could be (as I gleaned) an invisible hand reaching into NPCs and dynamically tinkering 'em to suit the player experience.

On Edward's idea about a "mentor bot" - I believe that Robin also mentioned earlier work in an FPS she modded using "ghosts" (if that was the term she used) to do something like this. A unanticipated problem, though, was that it detracted from the dramatic arc (mentors were too helpful).

[Flatfingers]

11.

As weasel said, some of these issues are different perspectives on a VW's being capable of reacting to what its inhabitants do.

Having said that, "Game GMs" (I'll call them GGMs) and NPCs need similar kinds of abilities -- just at different levels of abstraction. Specifically, they both need to be able to react in plausible ways to world phenomena.

This implies four requirements for both types of intelligent actors:

1. Both need to be able to detect existing environmental states.
2. Both need to be able to calculate desired environmental states.
3. Both need to do goal-directed planning to identify actions that will efficiently convert detected states to desired states.
4. Both need to be able to assert those actions within the game world.

The difference is that while NPCs should perform these functions at a tactical level, GGMs need to operate at a strategic level.

NPCs need just enough brainpower to be able to observe local phenomena and decide what actions they should take immediately to react to those phenomena. (Locality in space and time are what define tactical action.)

Usually this is about noticing and responding to attacks, but other factors can and should be considered as long as they're local and immediate. For example, guards in the latest Thief game are capable of "hearing" noises and "seeing" shadows close to them, then reacting in simple but appropriate ways to those events. This tactical intelligence makes their behavior seem highly plausible, which makes the game world feel "real" -- there's immediate and appropriate feedback to what game inhabitants do.

For a less happy example, NPCs in SWG seem deaf and blind to firefights that players can easily hear and see. Instead of armed NPCs running to defend their allies, and unarmed civilians fleeing inside the nearest building until the shooting stops, everyone just sort of lounges around despite the blaster bolts flying through the air. The behavior of these NPCs seems implausible because they fail to detect local environmental events and react appropriately to them. They don't demonstrate the tactical behavior we expect from that type of object.

And that's really all NPCs need to be able to do. Reasoning ability beyond the merely tactical gets us into the realm of diminishing returns; NPCs don't need to be long-term planners.

A GGM, however, does need that kind of strategic thinking ability. That's how human GMs create meaning in games; they organize set-piece events over large areas of space and across long periods of time to build up to a satisfying large-scale climax. Similarly, an effective GGM needs to be able to detect gameplay states over large space and across relatively long durations, determine preferable states, and generate appropriate actions that will cause the desired changes to occur.

Failing to have such GGMs (as most MMOGs do) leaves the world feeling disconnected, as though nothing one does matters in any larger and more meaningful way. Some of this can be corrected by developers actively intervening with story arcs that attempt to connect the disparate quests and game objects, but this takes time and effect. If a GGM existed that could do this sort of thing well, that would be a significant burden taken off the backs of developers.

The point of diminishing returns for Game GMs would probably be day they get so good at strategic planning that they no longer allow developers to change "their" game....

[AFFA]

12.

A little off-topic, but I'd like to play devil's advocate here. WoW's idea of adding NPCs to PvP battles may not succeed.

If the NPCs are not effective, it will not really even out the battle. If they are effective, it will create a "barrier to entry" for players that are even slightly less effective--i.e if they are not a high enough level, twinked enough in equipment, and skilled enough as a player to do better than the NPCs, then they will be most unwelcome.

This might be partially solved with non-traditional balancing. For instance, if the NPCs are more numerous and serve more as decoys than as combatants (but this requires enemies to appear, in all respects, as players--tricky, even for apparently simple things like mimicking the zig-zag movements of a real player). Or if one side gets a few weak NPCs, not based on the numeric differences in players, but for some formula that tries to calculate the relative "power" of each side. But I doubt any point system will be percieved as fair, no matter what your data mining indicates.

There is also something to be said for the planning and dedication necessary for "z3rg 4 teh w1n." While I'd love an alternative to the zerg strategy, I suspect victories in WoW may feel a bit... hollow. Zerg strategy also allows average or below-average players to win occationally by doing nothing more complex than /follow. Giving weaker players a win now and then is not something I would lightly dismiss.

There are a number of other issues that could be percieved as "unfair." For instance, to what degree NPCs appear to be affected by lag...

* * * * *

As for the actually topic of discussion, I believe computers can do almost everything a GM can do (failing only in reacting to--or even allowing--unexpected creativity). It may be diminishing returns now, but alot of work has been done on this already (mostly outside the game industry). There are no stellar successes so far, but eventually someone's going to put these approaches together and improve them:

http://www.aaai.org/AITopics/html/drama.html (esp. Brutus)
http://www.igda.org/writing/InteractiveStorytelling.htm
http://www.erasmatazz.com/

[Richard Bartle]

13.

Nathan Combs>The design here could be (as I gleaned) an invisible hand reaching into NPCs and dynamically tinkering 'em to suit the player experience.

This is something I'm not happy with at all in virtual worlds. Players will get no sense of progression if, no matter how skilled or unskilled they are, the bad guys always feel the same.

A super-powerful character can't go killing low-level monsters for the sheer joy of watching them go down with a single hit, because the AI will "learn" that they are having too easy a time of it and boost the monster's stats to make it more of a fight. The character didn't want a fight, though - they wanted to let off some steam.

Similarly, a low-level character going up against a powerful monster is basically training the AI to lower the monster's stats, or provide some healing potions or whatever, until the monster isn't too difficult.

What players need is for monsters to be reasonably static in their degree of difficulty. This doesn't mean the have to be dumb, or that they all have to have the same AI. What it does mean is that they should behave consistently. If they don't behave consistently, players don't know whether they (the players) are improving or not. Monsters can learn, but only until they get killed; they respawn as new monsters, who need to learn all over again.

This kind of tinkering is equivalent to making the virtual world adapt to the player, rather than the other way round. It's an AI technique, but it's not used for controlling discrete NPCs;
it's like having an intelligent universe. "Oh dear, poor Johnny wandered into the swamp and can't move very fast, I'll just reduce the viscosity of swamp in his location so he can speed up a bit". It's the same kind of thing. Johnny shouldn't have wandered into the swamp in the first place, but by helping him out he doesn't know that.

As for the effects when several players try to interact with the same object, dynamic difficulty is really going to make things look weird to some of them.

While doing some consultancy for a major virtual world about 3 years ago, I was asked whether I thought this was a good idea. I said it was a very bad idea, and I stick by that. If skill level doesn't count for anything, why bother learning?

What next? Selling people difficulty reductions?

Richard

[gus andrews]

14.

Educational games are one arena in which game-master-level AI would definitely be helpful. It would be good to see games which could adjust to a student's zone of proximal development, allowing for lower-level game objectives which the student could master without help and higher-level objectives which the agent or another player might help with. Ideally, playing to this zone is something teachers do. Speaking in wishful tones, really good AI -- not saddled with ridiculous class sizes -- might be more sensitive in some ways to an individual student than an overworked teacher could.

[Richard Bartle]

15.

gus andrews>It would be good to see games which could adjust to a student's zone of proximal development

I saw something like this a few (er, OK, 20 or so) years ago, for teaching children with low mental development. It found the right starting level by some kind of binary chop technique, then used the child's reponses to each question to gauge the difficulty of the next question asked. It was only a question-and-answer suite, but yes, I can see how such a system could easily pay dividends in an educational virtual world.

Richard

[AFFA]

16.

I'd call changing mob stats "cheating" if it was obivous or "dynamic difficulty" if it was not (it would probably always be obvious in an on-line game). I see the GM's role as narrative, not balance. Maybe a personal failing...

A few game AIs can adjust their strategy, learn new strategies, or even adjust difficulty without it feeling like cheating. Galactic Civilizations, Max Payne, etc.

Unforunately, the closest to machine learning I ever got was making mobs remember power-up locations.

[Nathan Combs]

17.

I'd call changing mob stats "cheating" if it was obivous or "dynamic difficulty" if it was not (it would probably always be obvious in an on-line game). I see the GM's role as narrative, not balance. Maybe a personal failing...

This is an interesting Q (perhaps only because I've discussed it half-dozen times in the last month, it seems)... under what circumstances is AI "cheating", in fact "cheating?"

Put it another way, are some advantages and deviations afforded to our AI okay for the sake of drama and gameplay? Deviations from what norm?

[greglas]

18.

gus andrews>It would be good to see games which could adjust to a student's zone of proximal development

This is actually a pretty conventional feature in good educational software these days. Relatedly, there's a concept in education called mastery learning, where there is an optimal challenge level to keep students motivated. The problem with prior techniques, the theory goes, is that students who performed poorly never mastered material and this created an educational syndrome of sorts. To motivate students and build a core of understanding on which to build later steps, it's important to keep *all* success rates high.

Is this important for gamers? I don't think so. First, as Richard noted, non PvP MMORPGs rarely have much to do with skill. (Kill rat, kill rat, kill rat, get coin, kill rat -- level up!) Second, you want the bar to be high, because the higher it is, the better you feel when you jump over it. In a lot of skill-based computer gaming, the success rate of play is pretty miserable.

Ninja Gaiden is completely impossible to win and extremely popular -- need I say more?

But on Nate's original point -- yes, to the extent AI can personalize the VW experience, I think that is something designers need to shoot for. I think this might be as much about ascertaining player typology on the fly and gently nudging players in directions that they will find interesting.

[Tom Hunter]

19.

"What next? Selling people difficulty reductions?"

-Richard

I doubt you could make money selling difficulty reductions because I don't think people would value them. My impression is that people play these games in part for the challenge. If that is correct then reducing the challenge reduces the value and very few people will pay to reduce value.

I don't think that people who hack games to make them easier are actually trying to reduce the challenge either. In Richard's book he explores different play styles, how they behave and what kinds of challenges they look for. I think his theory of play styles is fundamentally correct and that people who hack games are enjoying the challenge of the hack and of showing how they can manipulate the game. I doubt the prime motivation is to kill more bunnies per hour.

Of course hackers and other efforts to manipulate the game strike at the heart of efforts to make an AI GM. The players will quickly find ways to exploit the system and the consequences of these exploits will cause problems in the game fairly rapidly.

[Tobold]

20.

Creating a full neural network AI for NPCs would be not worth the trouble, but many NPC now are so full of "artificial stupidity", that a minor improvement could reap good returns. The same NPC often tells you the same story with identical words, even if you completed his quest already. Nearly every review about City of Heroes remarks how nice it is that the NPCs you save from the thugs actually thank you for that, and that isn't that hard to do.

Instead of the NPC saying "There are wolves behind my farm, go out and kill them" every time you click on him, why not make him minimally more intelligent? "You promised to kill those wolves, but they are still there", if you accepted the quest but didn't do it yet. And "thank you for killing those wolves, but now there are some more", once you finished it shouldn't be so hard to do.

[gus andrews]

21.

greglas: >students who performed poorly never mastered material and this created an educational syndrome of sorts. To motivate students and build a core of understanding on which to build later steps, it's important to keep *all* success rates high. Is this important for gamers? I don't think so.<

Among gamers, no, but how do we know there isn't a similar syndrome among non-gamers? I considered myself a non-gamer for years because I was never really "good" at them; it was always my sister Sylvie who beat them, not me. Addressing mastery and motivation issues might actually be something the industry would want to do if it's losing new players for this reason. I mean, hardcore gamers are really skilled these days... how are n00bs supposed to play things like Ninja Gaiden and not conclude that gaming just isn't for them?

[Ian Patalinghug]

22.

can u giv me some Ragnarok Game Master Skill Hack??... if u can thx..