This is where I try to maintain a list of games that have used--or which promised to use--particularly interesting AI approaches. The trend towards games that provide better AI has increased markedly since 1995, something which I feel can only help the entire industry as well as give an individual game better "legs" in the marketplace.

Caveat #1: Virtually all games (especially strategic ones) have some kind of "AI", and often use interesting and novel techniques. The focus here, however, is on the more unusual or ground-breaking approaches. Sometimes the focus is also on what was promised but not delivered on, since learning from failures is every bit as important as learning from successes.

Caveat #2: Remember to take anything you read here with the appropriate levels of salt. There is often more hype than fact in some of the claims, and oft-times a game will be redesigned at the last minute due to problems implementing a revolutionary new AI approach (witness Avalon Hill's Third Reich, below). This list is focused on those game which do, or promised to do, particularly interesting things.

If you know of games I have missed (whether an older game or an upcoming release), or if you can shed light on an aspect of the AI in one of the games listed, please feel free to e-mail me. I'll add it to this list as quickly as possible.

Game:       Age of Empires I/II
Type:       Realtime Strategy
Publisher:  Microsoft
Developer:  Ensemble Studios
Release:    Available Now    
Web:        http://www.ensemblestudios.com

Details:

Another entry in the crush of realtime strategy games, Age of Empires looks to provide the strategic, civilization-building appeal of Civiliation with the realtime action of Command and Conquer. The AI is interesting in that it attempts to learn from the player, and can be tweaked a bit with user scripts to boot.

Developer Dave Pottinger of Ensemble Studios, has wanted the AOE AI do preserve information from scenario to scenario. Here's an overview he sent me just prior to the game's release:

   For Age of Empires we did do limited learning to augment a pretty
straightforward approach to the AI.  The computer knows how to do
certain types of strategies that are common, including things like
frontal assaults.  When you play any of our campaign
scenarios the first time, the game is even.  However, as a human, you
carry over information about the scenario when you replay it the next
time.  So, we let the CPs do the same thing.  They remember where you
attacked them or they attacked you, etc.  We also let the CPs remember
your general playing tendencies so that they can improve playing against
you in the randomly generated games.  This has helped the quality of the
AI out a lot.  Well enough, in fact, that we'll be able to ship the game
with an AI that doesn't cheat.  However, we may do a "Doom-style Nightmare
mode" where the AI overtly cheats by way of getting a resource boost at
the start just to pound on people who like that kind of thing.

   The scenario replay learning feature was actually created out of a
desire to have people who didn't win the first time get a different
play experience when they replayed.  The goal here was to remove the
need to just optimize your strategy well enough so that you can
eventually beat the scenario with the same thing you tried to do the
last five times.  If the AI does something markedly different (yet
still intelligent, etc.) each time you play, then you get a more
enjoyable experience, I think.  It does help out replaying scenarios
that you've already beaten, too (though that wasn't the genesis of the
idea).
  
   The AI primarily saves which types of units they like to build along with
a few other things.  AOE is very much a rock-paper-scissors game
(infantry slaughter archers, but cavalary rocks infantry, etc.), so
concentrating on the contextual unit prefs of players is what provided
the most useful info and conveniently takes up very little memory:).
We did try a lot of other things, though:).

   FWIW, the playbook was the last military manuever model I did for AOE.
We've kept the other three for variety and level of difficulty sake, though.

   I guess I'd have to add that I've yet to see an AI that can't be beat by
some strategy that the developers either didn't foresee or didn't have
the time to code against.  Learning is a great way to help alleviate
that problem and thus create a better playing experience.

Game:       Baldur's Gate/Tales of the Sword Coast
Type:       RPG
Publisher:  Interplay
Developer:  Interplay, Bioware
Release:    Available Now   
Web:        http://www.interplay.com/bgate/

Details:

Promising to provide the 'definitive'Advanced Dungeons and Dragons role-playing experience, Baldur's Gate (BG) has some rather interesting AI-related capabilities built in. Players are able to directly edit scripts that control the actions of their NPCs, somewhat like the level of control provided by Dark Reign. The following is taken from Interplay's site:

   All NPCs have their own AI scripting, outlining their basic reactions to 
basic situations. At anytime, the player may "override" what the NPC is 
currently doing. AI may be turned off or on at will. The scripts can be 
modified to some extent; they help create a NPCs personality and adds to 
the immersion level of the game. There will be several levels of scripting 
available. We want to allow players to be able to modify scripts not at all 
(the scripts that come with NPCs should be quite adequate), a little (e.g. 
load in cautious mage, aggressive fighter, etc.), or a lot (e.g. if enemy 
gibberling sighted then cast magic missile at it). It will be up to the 
individual player to decide how much to tinker with AI on his or her party 
members. 

IF
	// If my nearest enemy is not within 3

	!Range(NearestEnemyOf(Myself),3)

	// and is within 8

	Range(NearestEnemyOf(Myself),8)
THEN
	// 1/3 of the time

	RESPONSE #40
	   // Equip my best melee weapon
	   EquipMostDamagingMelee()
   	   // and attack my nearest enemy, checking every 60 ticks to 
           // make sure he is still the nearest

	   AttackReevalutate(NearestEnemyOf(Myself),60)

	   // 2/3 of the time

	RESPONSE #80

	   // Equip a ranged weapon

	   EquipRanged()

	   // and attack my nearest enemy, checking every 30 ticks to
           // make sure he is still the nearest

	   AttackReevalutate(NearestEnemyOf(Myself),30)
END

Game:       Battlecruiser:  3000AD
Type:       Space Exploration/Adventure/Economic/Strategic
Publisher:  Take 2
Developer:  Derek Smart
Release:    Available Now
Web:        http://www.bc3000ad.com

Details:

Battlecruiser: 3000 AD (BC3K) and its various followons claim to be the first commercial game to feature neural networks (it's not; others beat it to market). A your-ship-alone-vs-the universe style of game in the tradition of Star Control, virtually every NPC (non-player character) in the game is driven by a neural network. This includes each of the 125 crew members of your ship, which is quite impressive technically. The computer opponents also use neural networks to guide negotiations, trading, and combat.

Commentary:

No game has caused more controversy than BC3K, much less its AI (or lack thereof, depending on whom you ask). Successful or not, it's on this page for what it has tried to do. Only the user can decide if it's successful....

Since I first put this posting up I've discove#f0d303 that a couple of other games featuring neural networks did in fact hit the commercial market back in 1994 and 1995, and one game used neural networks way back in 1987 (!) but which didn't make it to market (so it doesn't count). The two games which got to market are Jellyfish and TD-Gammon, each of which began as public-domain software that later were distributed commercially. Since backgammon is a "classic" game I've got details over on the classic games page.

In an email from Derek he revealed that he has actually developed his own language, named AILOG, in which the AI code is all written. I find this fascinating, and hope to learn more about why he did this, but in the meantime here's some more info about how NNs work in the game:

Anyway, here's brief overview of how a NN is used in BC3K. I have to be
brief due to time constraints but feel free to send me questions.

The BC3K Artificial Intelligence & Logistics, AILOG, engine, uses a neural
net for very basic goal oriented decision making and route finding when
navigating the vast expanse of the game's galaxy. In some cases a
supervised learning algorithm is used and in other areas, an unsupervised
one is used. It also employs some fuzzy logic where a neural net would not
suffice. Some aspects of a SOM model and various Backpropagation variations
are also employed.

So, AILOG, contrary to popular belief, and hyped just as much as the game
itself , incorporates several basic AI and NN techniques in several of
the engines. Some training (for supervised learning algorithms) is
basically done at a low level where ship and personnel characteristics are
involved as well as route finding in the galaxy, route finding within the
ship, engineering repair assignments, internal ship combat with marines and
intruders/escaped prisoners. It also involves threat identification and
decision making, ie 'I am a Terran Insurgent flying an interceptor fighter'
going up against a 'Terran GALCOM fighter' what do I do? or 'I am a
Gammulan Criminal up against a Terran EarthCOM ship. My ship is 50% damaged
but my weapon systems are fully functional and my goal is still unresolved'
what do I do? Similar decision making is done at other levels such as when
an alien nation builds and deploys SAM batteries or aircraft. Pound them
enough and the next time you come back, they've gained enough smarts to
either deploy better models or increase their decision threshold. Wars are
fought and war, invasions are scheduled and executed, all due to
pre-trained scripts and algorithms programmed to 'fire' under the right
conditions. Stuff like that. Alas, the game, unfinished in it's current
state, makes it a little impossible to full realize and understand the true
potential of this game. That's being remedied as I strive to complete it.

As you know, there are various forms of NN technology such as employed in
pattern recognition, adaptive resonance, Kohonen SOM etc. I simply took
what I had and without re-inventing the wheel, made it work for my game.
Most of it culled from numerous books and papers on the subject just like
any other book that I've read on graphics and other techniques I have in
the game. This will be my first true experience in using this sort of
technology outside of the vertical market I came from. With the advancement
of technology and the introduction of such technology in games, the use of
neural networks is long over due. I am positive that several titles out
there use neural nets for decision making but no-one's making a big deal
out of it. The game EF2000 comes to mind as I understand they used neural
nets in their Wargen module. I seriously doubt that BC3K is the first title
to employ this technology at any level. The game has been hyped so much
that 'neural net' to a casual gamer just  became another buzzword and
something to look forward to. At least that's my opinion.

Regards

Derek Smart

Game:       Black & White
Type:       God Game
Publisher:  Electronic Arts
Developer:  Lionhead Studios
Release:    Available Now
Web:        http://www.lionhead.co.uk

Details:

Black & White (B&W) promised to bring everything that Peter Molyneux had experimented with in his previous games (Populous, Theme Park, Dungeon Keeper, to name a few) to fruition in one package. Peter's always been on the cutting edge when it came to interesting A-Life technologies in his games, and B&W promised to continue that tradition.

According to both the press and various interviews given about the game, B&W was to feature "...AI technologies that are revolutionary and never been used before...". The game promised that your creatures would be able to learn all kinds of things, both from the player and each other, as the game progressed, creating a dynamic world that will change according to the player's actions. The game takesto this to the literal extreme...as the player plays the game, his actions and motives shape the world... if you're good, the world will become a place of sun and light, whereas evil players will create a world that slowly becomes a twisted, dark realm filled with monsters. (Just a guess, there will be more dark and twisted worlds than "Disney" worlds.)

In an interview on one of the Black & White web pages shortly after the game came out, Peter spoke a bit about how the AI of the game's creatures operates:

   ....only a small number of behaviours (have been put in) so 
   far, but enough for the creatures to behave convincingly. 

   I summoned a creature using the debug testing menu, 
   and it happened to be near water, and the creature 
   caught sight of his reflection and lumbered over to 
   have a look. Another time he became hungry, looked 
   around, found a villager, stomped him flat, picked him 
   up and ate him. Each of these little sequences of 
   behaviour is coded by Richard. 

   The creature might have several desires: to find out 
   about something, to satisfy his hunger, to go 
   somewhere. Each desire is associated with an 
   'intensity'. Whichever is the most intense, and can be 
   satisfied nearby, the creature satisfies. He might be 
   hungry, and see a villager and a fence around, and he 
   has a table saying how each of those objects 
   (villager, fence) will satisfy his hunger. 
    
   When the creature was just freshly created he didn't 
   know that fences aren't edible. And ate some. 

   This sort of design is fairly typical for AI in 
   computer games. But where Black and White differs from 
   convention is that the creatures can be taught how to 
   behave. You might want to teach your creature to eat 
   enemy villagers but not your own. This is how it's 
   done: For each object, such as a villager, there are a 
   number of attributes (sex, age, allegiance...) The 
   first time the ape eats a villager, and you pet him in 
   reward, he thinks that it's good to eat all villagers. 
   But next time you give him one of your own villagers, 
   and he eats it, and you punish him. So his mind 
   becomes more refined: he recognises that some 
   villagers are yours and some are not, and it's okay to 
   eat the latter but not the former. And with more 
   training you can further refine his mental model of 
   the world. 

   To some extent, the creature watched your actions and 
   emulated them. If he saw you casting a spell he'd 
   learn how to do it and start casting it himself. More 
   such observation-and-copying will be added to the game. 

   When the creature has just performed an action (such 
   as eating a villager), and you want to tell him what 
   you thought of that action, you hold down your index 
   finger on the creature. The camera zooms towards him 
   so you're looking at him full on. You keep your finger 
   down all this time. If you move the mouse gently 
   around his head, or arms, or feet (or, yes, groin) he 
   likes it and laughs playfully, and he knows that you 
   approve. Or you can move your hand to one side and 
   sweep it to the other, hitting him, and it hurts. Or 
   move your hand all the way to one end of the screen 
   and *BLAM* smack him one and it really hurts. You can 
   smack his feet out from under him and he'll fall over. 

Game:       Beasts
Type:       Sim-life/Ant-farm/God Game
Publisher:  Unknown
Developer:  Creature Labs 
Release:    Second Quarter, 2001

Details:

Building on the successful A-Life technology originally pioneered in the Creatures line of games, Beasts aims to take the whole experience one step further--by moving your charges out of the artificial world and into the "real world".

The "beasts" in question are Yeti, who live high in the mountains in a valley threatened by mining. The player must guide their Yeti tribe in such a way as to eventually drive away the mining conglomerate before it can destroy their homeland.

Beasts features a number of enhancements on the basic Creatures technology, including dominance hierarchies, realistic mating habits and complex social structures. To win, players will need to deal with a number of time-pressured missions. Between missions, they will have a more general responsibility for their charges, ensuring that their Yeti prosper and multiply and are ready for the next challenge. Multi-player options range from death-match conflicts between Yeti tribes, to open-ended play, in which players can observe their beasts interacting with no set goals (much as with the original Creatures). Annual seasons and living secondary ecosystems will provide additional challenges.

Commentary:

As with the Creatures series, Beasts will be less of a game than a digital ant-farm. The basic chemistry-based neural net technology remains in place, though apparently with much greater complexity. The new mission-based option is obviously intended to answer the complaints of some players that there wasn't any purpose to what they were doing in the earlier games.

This should be interesting, if only because it's the next evolution of the Creatures A-Life technology. More when I know it....

Game:       Civilization:  Call to Power 
Type:       Strategic Wargame 
Publisher:  Activision
Developer:  Activision
Release:    Available Now
Web:        http://www.calltopower.com 

Details:

Civilization: Call to Power (CIV:CtP) is the third in the classic series of games. Lead programmer Steve Marrioti had said during the development cycle that the new CIV:CtP would feature a non-cheating AI that would use a "...small number of powerful over-arching potentialities. In situations where there is no clear-cut choice for the AI--if this happens, respond by doing that--we're relying on fuzzy logic".

Of more interest perhaps was the plan of the development team to provide full and total access to the AI to players. The AI was to be built as a dynamically loadable .DLL file and the interface spec made public, thus allowing budding AI developers to write their own AIs if the ones that ship with the game weren't good enough. Specifically Activision promised:

Game:       Cloak, Dagger, and DNA 
Type:       Tactical Realtime Wargame 
Publisher:  Oidian Systems
Developer:  Oidian Systems
Release:    Freeware
Web:        Full version (free): here (Thanks Don!)

Details:

Cloak, Dagger, and DNA (CDDNA) was the first game I'm aware of that used genetic algorithms. It as intended to be the first in what Oidian called a "planned family" of games using genetic algorithms, but sadly there wasn't enough support and they went out of business.

The game is still available via download, however, and it's worth the trouble. Somewhat similar to Risk, a map is broken into regions, some of which contain factories. Possession of factories both brings income to the player and provides bases at which to build more units (either armies or spies). Armies are necessary to take and defend areas, while spies cannot fight or be killed and hence prove to be useful inteligence gathering tools. Combat is calculated based on the number of units in a given area, with the defender getting a defensive bonus. There can be any combination of four players, and the game does appear to support network play, which is 2 on 2.

The heart of the game is its use of genetic algorithms to guide the computer opponent play. It comes pre-stocked with 4 'DNA' strands, which are rules governing the behavior of the computer opponents. As each DNA strand plays it tracks how well it did in every battle. Between battles, the user can allow the DNA strands to compete against each other (and/or the player's DNA strand) in a series of tournaments which allow each DNA strand to evolve. There are a number of governing rules for DNA strand mutation, success, etc., and the user can edit a given strands' DNA ruleset if so desired. You can maintain a huge library of DNA patterns in the registered (free) version.

Commentary:

Having played extensively with this game I can report that a.) the concept is interesting, b.) the concept seems to work, as the AI does get smarter over time, and c.) the genetic algorithm "lab" that comes with the game needs a lot more work to make it accessible to Joe Gamer. The tools do provide remarkable insight into the training and evolutionary process of each DNA strand, however, as well as tools to directly tweak the rulesets being evolved. However, anybody with some background in genetic algorithms is advised to check it out; anybody seeking to learn how they work should seek basic knowledge elsewhere, then come play with this game.

All in all a very cool game using a very powerful approach.

Game Developer Magazine (the magazine if you're a game developer) ran an excellent article by Don O'Brien (the designer of CDDNA) about the game's AI, how it works, and why it does what it does. I was particularly struck by the use of some of the techniques discussed in the Influence Mapping thread that went on over in comp.ai.games a while back. I highly recommend it to anybody interested in learning more about genetic algorithms in general and their implementation in CDDNA in particular.

Don O'Brien contacted me on 2/15/02 with a quick missive. He was amazed at the interest in the game and, out of the generosity of his heart, he has released the full version of the game to the general public! We've got it here here and the site and it's VERY worth the download. The game is simply an excellent lab to play with. If you do decide to download it, drop Don a line and say thanks--he more than deserves it!

Game:       Creatures/Creatures 2/Creatures 3/Creatures Adventures
Type:       Sim-life/Ant-farm/God Game                  
Publisher:  Warner Interactive
Developer:  Millennium Interactive 
Release:    Available Now

Details:

The Creatures series makes probably more use of Artificial Life technology--genetic algorithms, neural networks, etc.--than any other series of games on the market. This makes sense given the goal of the series--to simulate organic life. Similar to the Sim games and sharing much in common with the screen-saver game Dogz, Creatures takes the concept one step further. Each copy comes with six "eggs", each of which has a unique dog-like creature Milennium dubs a Norn. Milliennium guarantees that no two copies of Creatures will have the same sets of eggs, making every copy unique.

Once your Norns hatch the fun begins. You can begin interacting with your Norns, rewarding them for good behaviour, feeding them, introducing them to objects in their environment (a multiscreen world roughly 12 screens wide by 3 high), or punishing them. This environment is filled with interesting objects (which you can add or remove) with which the Norns will keep themselves busy, along with food, plants, and a predatorial race called Grendels. As your Norns grow, they progresses through adolescence and adulthood and (if you have two adult Norns) eventually will lay eggs of their own. In this way you build an ongoing "colony" of these creatures, much like an ant farm or aquarium on your computer.

The fun of the game lies in observing the development of the Norns over time and influencing that development. You do not have direct control over the Norns, so they will not necessarily respond to your stimuli in a predictable manner. Players will be able to "swap" eggs over the Internet, exchanging different character traits and gene pools as different strains of Norns evolve over time.

The interesting aspect of the game from an AI point of view is its use of its CyberLife technology. CyberLife is a combination of heterogeneous neural networks and a GA-ish winnowing process to push evolution of the creatures. This makes the CyberLife technology effectively a self-training neural network, with the AIs of the Norns learning over time what they like, what they're not supposed to do, what is "fun", etc. According to Anil Malhotra (Millenium Interactive's Director of the CyberLife technology), the AIs have evolved in ways that surprised even them. He reports that after a "litter" of baby Norns had hatched on Friday, when they came back in on the following Monday the babies had learned how to toss a ball back and forth for "fun". Fascinating.

The company supports the game in a variety of ways, including an online "Olympics" in which Norns from around the world can compete in various feats of prowess. In addition, since the natural lifespan of a typical Norn is a mere 40 hours, the company will also provide (for a small fee) a 'funeral kit' with which you can immortalize a beloved Norn's memory (I am not making this up).

Commentary:

Less of a game than an ant-farm, Creatures nevertheless seems to break new ground in the game AI field. The developers are quite frank about their intentions--Creatures is really more of a technology demonstration than anything else, and they'd really like to tap the commercial applications market.

Having said that, the game neatly solves the fundamental problems associated with online training by getting you to run the game constantly. Online realtime training of a neural network is (generally speaking) far too expensive for "normal" gameplay, but by marketing Creatures as an "organic simlator" which evolves over time Milliennium Interactive has side-stepped that problem. Even so, the program is still fairly expensive in terms of CPU...each Norn reportedly eats up approximately 5% of a typical Pentium P-100 CPU, a value probably far too expensive to use the Cyberlife technology in the next version of C&C.

The use of GA-ish technology to cross-breed the AI neural networks is solid, as is the ability of the player to trade eggs with others. There are tools within the game to look deeper into the 'brains' of the game and see what it's doing.

In trying to gain greater insight into how the AI in Creatures works, I recently asked the developers of to explain what they meant by their using "heterogeneous" neural networks. I was unsure what that meant, exactly, so I asked. Here's what developer Toby Simpson said in response:

Heterogeneous as in not harmonious. The neurones are divided up into
lobes which serve different purposes, although the neurones in each lobe
are the same. Things such as leakage rate, dendrite migration and so
forth can be set for particular lobes without simply having a collection
of the same old neurone as it would be in a "normal" net. This is the
way mother nature does it, etc. As for what they actually do, well, they
act like real living brains, only somewhat smaller than our own right
now. Hope that is of some help.

Game:       Close Combat
Type:       Tactical Realtime Wargame                   
Publisher:  Microsoft
Developer:  Atomic Games
Release:    Available Now        
Web:        http://www.atomic.com

Details:

Close Combat (CC) was to be the first game to use blackboard technology to guide both the player and computer squads throughout the game. Blackboard technology is a "cooperative" AI technology, in which a problem is made open to a group of AIs (which can be neural networks, genetic algorithms, whatever) and each AI "contributes" towards solving a piece of the problem. The analogy is a group of engineers sitting in front of a blackboard interatively solving a problem together. Each engineer contributes some idea which solves a part of the problem, and in solving that piece makes it possible for another engineer to solve a different piece. Eventually a solution is found.

Unfortunately, Atomic either couldn't make the blackboards work correctly or ran into some other problem with the implementation. In a recent flurry of emails between the Close Combat designers and fellow AI enthusiast Bryan Stout, one of the CC folks admitted that they'd scrapped the blackboard-based AI in favor of a more traditional hierarchial rules-based approach. Bryan was kind enough to provide me with an email summarizing the implementation that made it in, and it follows below (reprinted with permission of Atomic Games, of course):

The final implementation did not use blackboards. Here's a brief
description of the implementation.

The Strategic Artificial Intelligence (SAI) in Close Combat (CC) handles
the task of creating high and medium level orders for the teams under
direct computer control and medium level orders for the teams under
player control. Sometimes, a player's teams will be placed under
computer control (either by the player clicking the advance or retreat
buttons or if the player has not issued an order to a teams for a long
period of time) in which case the SAI will also generate high level
orders for the player's teams placed under computer control.

When creating  orders, the SAI reasons only about the teams as a
whole--It never reasons about individual soldiers in the teams. In
reasoning about locations on the CC map, the SAI divides the map into a
square grid. The squares in the grid are referred to as megatiles. The
size of a megatile is roughly the size of the smallest square building
which can be found on the CC maps (about 18 by 18 meters).

High level orders correspond to the orders in the pop-up menu that
appear when a player clicks on a team. An example of a high level order
would be "Move alpha team from megatile grid location (3,5) to megatile
grid location (6,11)." The SAI generates high level orders for the
computer controlled squads using information about the current game
situation and victory conditions. Player controlled units are given
high level orders by the player through the pop-up menu that appears
when the player clicks on units under his control.

Medium level orders are generated by the SAI to accomplish high level
orders at a team level. For the previous example of a high level order,
the medium level orders might include "Move alpha squad from megatile
grid location (3,5) to megatile grid location (4,6)," "Move alpha team
from megatile grid location (4,6) to megatile grid location (5,7)," and
so forth. The SAI always generates the medium level orders required to
accomplish high level orders regardless of whether the high level
orders were created by the SAI or the player.

The SAI is comprised of three main systems: the location selector, the
path planner, and the target selector. The location selector is used by
teams under SAI control for generating high level goals. It determines
where the team should be on the CC map. If the team is not at its
desired location, the path planner is invoked to determine the medium
level movement orders needed to get the team where it needs to be. The
path planner can generate paths based on speed, safety, or a
combination of the two. Once at its desired location, a team uses the
target selector to determine which enemy team (if any) it should
attack.

When a player gives a move order to a player controlled team, the
player's order becomes a high level movement order. The path planner is
then invoked to generate the medium level goals to get the team to the
specified location. Fire and smoke orders issued by the player are
converted to medium level orders indicating that the team should fire
at the specified location. A defend order issued by the player invokes
the target selector which picks an enemy target for the player's team
to attack.

The location selector uses a number of criteria for selecting a desired
location for a team. First, it determines if the team should just stay
where it is. When a tank first sees an enemy tank or anti-tank team,
it's most likely to try to get the first shot at the enemy rather than
moving on its way. Infantry under intense fire or in a good defensive
position tend to stay at their current location as do mortar teams
since they like to deploy and stay out of the enemy's line of sight.

If the location selector decides that a team should consider moving to
other megatiles, it begins a search for the best megatile in which to
position the team. Since each CC map contains hundreds of megatiles,
the location selector uses a number of heuristics to prune the search
space. For example, moving a rifle team into the middle of a large open
field is almost always a bad idea. The location selector uses a number
of factors to both prune and rank the list of megatile locations being
considered. The factors include the defensive benefit of the megatile,
the strategic importance  of the megatile (is it a victory location),
the amount of time it will take to reach the megatile, the amount of
danger involved to reach the megatile, and the number of friendly and
enemy causualties expected when the team attacks the enemy from the
megatile. When dealing with enemy teams, the location selector
hypothesizes the location of enemy units rather than cheating by
looking at the real positions of teams that it should not be able to
see.

Gary Riley
garyr@atomic.com

The designer's note article that mentioned blackboards was written at
the very earliest stages of development. At the time, it was not
apparent that handling interaction between the low level simulator and
the high level AI would be such a difficult task. A lot of effort went
into resolving these issues. For example, the simulator determines
line of sight to a team by tracing from individual soldier to
individual soldier, but the high level AI has to have some type of
abstraction which divides the map into locations and provides
information about whether a team can fire from one location at
another. If the abstraction didn't work well, you'd either have
teams moving to locations from which they couldn't attack the
enemy and moving out of location from which they could. The solution
we ended up with was to iterate over all locations on the map
deploying teams into the divided map locations and then have the
simulator determine whether a line of sight existed (which took a
considerable amount of time).

Anyway, we had a lot of issues like these that had to be worked
on through almost the entire development cycle, so as the AI was
being developed I just used very straight forward approaches to
determining team actions (i.e. if the AI can't figure out a location
to move to from which a team can attack the enemy, no one is going
to care whether the AI uses blackboards or not). In fact, because
so much time was spent on getting fundamental capabilities working,
a design for using blackboards never got beyond a very high level
hand waving stage.

It would be nice to chunk the current code and do a complete redesign
which utilized blackboards and made use of what I learned from the
first attempt since one of the AI's weaknesses is team coordination,
but I don't see that happening. So CC2 will use the same basic design
as CC1.

Gary

Game:       Close Combat 2
Type:       Tactical Realtime Wargame                   
Publisher:  Microsoft
Developer:  Atomic Games
Release:    Available Now 
Web:        http://www.atomic.com

Details:

Close Combat 2 (CC2) is the followup to the earlier Close Combat. It will use the same basic AI engine, though there are some refinements in the works that promise to significantly enhance the AI's overall intelligence.

John Anderson, one of the AI developers dedicated to the project (it's my understanding that there were three in total) provided the following when I asked him about the plans for CC2's AI:

AI will almost always be shirked by the software developers/producers, at
least in the initial release.  This I feel is because most AI cannot be
written to be effective until late in the development cycle as it needs a
functional game environment to see the effects of the AI.  Then the
developer is faced with a choice of spending several more months to get the
AI right, or release a fully functioning game with limited AI capability.
Most choose the latter.

We knew from the outset that with our design goal of having every soldier
be a living thinking entity on the screen, that we needed a lot of AI
resources.  I split my work between the TAI (Tactical AI) and the game
system because they were so closely related.  This allowed me to develop
the AI right along with the combat engine.  It still took 3 1/2 years of
work.

I use a fuzzy logic system which weighs in hundreds of variables through
dozens of formulas, eventually coming down to a probability of a particular
action that the soldier will perform.  To prevent inconsistant jumping of
actions (soldier decides to go prone, then the next instant decides to
stand up, then go prone, etc.) a series of weights are associated with good
behavior.  This helps reinforce a consistent set of good actions.  In the
case of bad behavior, the soldier is restricted from choosing a good
behavior action until certain conditions or time limit has been met.

This type of AI also exists for a team, or a collection of soldiers that
operate as 1 entity for the purposes of issuing them orders.  It helps to
reinforce the peer pressure greatly evident in stressful situations.

The biggest problem with this approach was the balancing of the engine.
Months and months of work went into getting appropriate behavior for any
given set of circumstances.  Often this just involved adding more
parameters to the engine to account for the new circumstance but several
times it required an adjustment to the current values, putting more
emphasis on one or more parameters, which then caused other behaviors to
get out of whack given slightly different circumstances.

In CC2, we have had even more time to balance and tune the engine making it
what I think is the best psychologically affected combat simulation around.

John

The path planner in CC2 uses an algorithm reminiscent of a
paint fill in a graphics program that allows it to calculate the
best path (as determined by a scoring function) from a starting
location to every other location on the map in one swoop.
So, for the price of determining one path you get all the
others as well. 

Game:       Colobot
Type:       Tactical Exploration and Combat                   
Publisher:  Self (mostly)
Developer:  Epsitec
Release:    Available Now 
Web:        http://www.colobot.com

Details:

A neat game that I need to add to the programmable list when I get a chance, Colobot (COLO) is a European game in which you must explore and colonize a world using robots. The neat thing from an AI perspective is, of course, that your robots are programmable. enhance the AI's overall intelligence.

According to the developer, COLO is a realtime game in which you head a space expedition in which you must explore and colonize various planets. Assisting you are a variety of robots, which you can program using a simple C-like scripting language to explore, find raw materials for future colonists, etc. Kidna neat.

The program is available in a variety of languages (French, Polish, and English at present) and tehre is an excellent forum at the site for discussing your Colobots and the game. You can download the game, new levels, and other Colobots as well. Very slick.

Commentary:

The scripting and programming engine for the robots is definitely where the game shines, and I'm impressed by just how C-like it is (as a C and C++ programmer, that is). That could intimidate some folks but it shouldn't--the game is far easier to pick up than some others in the same general genre. The programs themselves appear to form an AI which can be pretty straightforward in a finite state machine/fuzzy state machine kind of way but there's nothing wrong with that. Worth a download and a purchase, if you're looking for this kind of thing.

Game:       Dark Reign
Type:       Realtime Strategy
Publisher:  Activision       
Developer:  Auran
Release:    Available Now
Web:        http://www.activision.com

Details:

The designers of this game promised a lot of fascinating AI-related goodies:

• No cheating (strategy gamers hate cheating).
• Realtime adapatation to each player's strategy and tactics.
• Various pre-canned AIPs (Artificial Intelligence Personalities) that the players can select from that will play aggressively, defensively, etc. Players will be able to design their own missions and, within that mission, adjust the computer AI so it knows how to handle certain situations. Over 20 different game conditions will be provided together with a friendly user interface that will allow players to "roll their own" to suit the scenario in question.
• Some method of building your own AI; details on this are somewhat sketchy.

Game:       Dirt Track Racing
Type:       Action
Publisher:  ValuSoft
Developer:  RatBag
Release:    Available Now
Web:        http://www.ratbaggames.com

Details:

One of the few action titles on this page, Dirt Track Racing is much like any other racing game with one exception--it uses neural networks as part of its AI.

There's not much info about exactly how the AI uses NNs, but from what I can tell from the reviews and press releases the AI does seem up to the task.

Commentary:

There's honestly not much about the game's AI implementation on their pages. At a guess I imagine the NNs are used primarily for handling the "fuzzy" nature of driving around a racetrack more than anything else. There have been a number of NN-based AIs used in the public-domain RARS racing effort and they've all been rather successful. I'll post more when I know more.

Game:       Dungeon Keeper
Type:       First-Person Adventure Strategic
Publisher:  Electronic Arts
Developer:  Bullfrog
Release:    Available Now
Web:        http://www.interplay.com

Details:

Dungeon Keeper (DK) is an innovative twist on an old idea, in which you the player are the "keeper" of a dungeon filled with monsters, traps, and treasure. Your job is to keep the sweaky-clean good guys OUT. Sort of a "SimDungeon", you are placed in charge of a dungeon with a limited amount of resources and monsters and must build a dungeon room by room, trap by trap, and monster by monster. If you're successful, you'll be able to bring in new recruits and continue to fight off parties of adventurers foolish enough to come bug you.

The AI implemented in this game makes use of a process called "behavioural cloning" to learn from the human gamer's play. The brains of the monsters themselves come from hundred of hours of internal play by the designers; every time a particularly nasty trick or sneak attack by one of the players worked out, it was incorporated by the designers into the monster's AI database. In network games, one can even allow the game to run in the background and allow the AI to manage the hiring of monsters, placement of rooms and traps, etc., all based on information it has gleaned watching the player. Producer Peter Molyneux has also said that he'd even like the AI sophisticated enough to learn each player's playing style (in a networked game) and be able to mimic them if they leave, although whether or not this will actually be in the game remains unclear.

Dungeon Keeper claims to possess the "most sophisticated monster AI of any game yet", with each monster having roughly 1500 bytes dedicated to AI and personality stats and can have the senses of sight, hearing, and smell. (By comparison, the AIs in Populous used 48 bytes each.) Monsters that are hurt feel pain and try to run away; monsters which can smell use this ability to track players and lead other monsters to where the players are hiding.

Commentary:

The developer here is Bullfrog, probably best known for their "God Games" Populous and Powermonger, each of which garnered high praise for the "personalities" in each game. Peter Molyneux has a proven track record of building sophisticated and interesting AIs, and from the sound of things has taken everything he's ever wanted to do and put it into this game.

Molyneux is probably one of the most ardent supporters of better and more sophisticated AIs in the industry. To quote from the PC Review article:

"The technology is huge, and I think that up until now there 
hasn't been a lot of kudos associated with AI, but as graphics
accelerators become increasingly common there will be
processing power to spare."

Game:         Dynamic Gin
Type:         Gin Rummy Card Game
Publisher:    CyberSym Technologies
Developer:    CyberSym Technologie
Release:      Available Now
Web:          http://www.cybersym.com

Details:

Dynamic Gin is a Windows version of the classic card game. It is yet another mainstream, commercial game to make use of neural networks for its AI, this time in a learning and adaptive implementation.

According to the developers, the engine behind Dynamic Gin is (primarily) a temporal difference neural net from CyberSym's object-oriented AI library. This gives Dynamic Gin what the developers claim is a very competitive style of play as well as the capacity to adapt and evolve when playing against a human opponent (temporal difference NNs are unsupervised training NNs).

Dynamic Gin is available as shareware with a free 36 day/30 session evaluation period. You can find it at the CyberSym web site.

Commentary:

Yet another neural network based game hits the market! And this one learns as it plays no less. I can't vouch for how well it plays since I'm not a Gin fan, but I am impressed.

Obviously the slower pace of a card game like Gin is what makes the online learning possible, as well as the relatively low graphics requirements compared to, say, Dungeon Keeper. Still this is not to detract from CyberSym's accomplishment. They have implemented an online, adaptive, learning AI for a real, "mainstream" game. My hat is off to you, CyberSym.

Game:         Enemy Nations
Type:         Realtime Strategy
Publisher:    Head Studios
Developer:    David Thielen, Windward Studios
AI Programer: Eric Dybsand, Glacier Edge Technology
Release:      Available Now

Details:

Enemy Nations is intended to be a cross between Empire Deluxe, Command & Conquer, and Sim City. Announced at the 10th Computer Game Developer's Conference, the game features multi-player (up to 16 players, I believe) real-time combat in a race to be the first to develop a newly discovered planet.

The EN AI, or computer player, uses a network of cooperating intelligent agents, or managers that communicate via messages, finite and fuzzy state systems and a database of goals and tasks.

Functions such as path finding are provided via a separate pathing manager based on the A* algorithm and enhanced for four years. The routing and distribution of materials for the economics of the AI and human players relies on a separate route manager which provides for discrete event management of the economic and construction needs.

Evaluations and map based processes are performed by a separate map manager using a variety of specialized search algorithms that include (but are not limited to): breadth and depth searchs of the map space, directed and A* derived searchs of the map space, and a few very specialized searchs of the map space.

The AI uses a separate goal manager for strategic guidence and a low level task manager to assign and manage the functions carried out by the units of the game. The actual task functions operate as discrete intelligent agents themselves, reviewing the local data specific to the task assigned. With guidance from the goal/task/map manager state systems, these agents issue/receive messages to/from the game which results in an underlying behavior that produces the complex behavior that completes the tasks assigned. This collection of tasks performed for the goals as managed by the goal manager results in the operational behavior of the AI players.

Selected history is maintained at the goal manager, player, map and OpFor (Opposing Forces) levels.

The AI programer (Eric Dysband) invites all comments and suggestions for any enhancements to the AI and can be reached at edybs@ix.netcom.com.

Commentary:

Okay, first of all the confession. I first met Eric online in the Recognizing Strategic Dispositions and Influence Mapping threads during the summer of 1995, and since moving back to Denver have regularly met with him at the monthly Colorado Computer Game Developer's meetings. We're friends, have a scary number of ideas in common regarding AI, and ping ideas off each other all the time. So, if you're the type who thinks I might be biased just because I know the guy doing the work, you're so warned.

Okay, having said all that, I have to say this is one of the niftiest AIs out there. I know it's good because a.) I've seen it work and b.) Eric is doing it 99% the way I would. If it has any weaknesses it may be that it doesn't handle resource-poor starts and worlds very well...a minor complaint given how difficult it can be for a human player to compensate in such a situation. The EN AI really does do what it says.

Game:       Fields of Battle
Type:       Strategic Wargame
Publisher:  Bevelstone Productions (Denmark)
Developer:  Bevelstone Productions (Denmark)
Release:    Available Now

Details:
Fields of Battle (FoB) is a World War I strategic wargame, currently available for both the Amiga and the PC. In concept it's somewhat similar to the old SSI game Clash of Steel; players jostle for control of Europe, Russia, and the Middle East over a map divided into various sized regions. Production and sabotage of resources plays a key roles in the game.

What makes the game interesting from an AI point of view is its use of Neural Networks for the computer player AI. This makes FoB the first commercial game that I am aware of that uses neural networks of any kind (and definitely beating out BC3K for that honor).

I've asked the developer for more details, but I do know the following regarding the FoB AI:

• It uses stochastic Hopfield nets for analysis;
• It uses simulated annealing for unit movement decisions;
• I am unable to locate any kind of initialization file for the neural networks, leading me to believe they are stored internally.

   We are not training the neural nets during the game, and thats the 
problem of the AI at the moment.   One of the reasons it's running 
"slow" is that we have used our time to develop the AI, not to speed 
it up, it could probbbly run a bit faster using fixed-point floats, 
but we prefer using developing time on other subjects.

   Generally the AI was developed in a series of attempts. First the 
optimistic one - lets start programming - it failed :)

   We then developed a huge (40 pages) document describing how to 
determine where to move a unit, when to buy etc. but it still didn't 
work. We could however, use parts of this document to determine which 
areas to defend, and which to attack.

   That's when we changed to Hopfield nets. The reason it's a Hopfield 
is, that that's the way the game work. You take a unit, try to place 
it in all areas where it can be moved to and select the best (or randomly 
another place, thats the stocastic part, but you probebly know as much 
about this as I do) area. We just keeps during this until no units 
wants to be moved anymore.

   So, with the right utility function everything is as it should be. 
Unfortunately it's VERY difficult to write a utility function. We 
are currently taking a break with FoB, but here is some plans we 
have in store, if we take up the game again (they are developed as a 
project in an AI course at the University of Aarhus:

The problem is that the utility function is tactical: It adjusts 
itself to the current situation, without any overall planning. We 
would like to implement an strategic part which will be run first. 
This part will use Generic Algorithms (or programming, I can never 
remember, but the one where you are changing your data, not your code). 
First it will make a simplification of the situation (joining non-
importent areas, counting more units as "one" etc.) to allow decent 
processing time. We will then use the GA to determine which areas to 
"go for", and which to abbandon etc. 

We still have a problem with the process-time, but those who really 
wants to play, will probebly accept this if is gives them a strong 
opponent. We plan to make it possible to use multiple computers in a 
network to speed things up.  
 

   That's the plan, and if we get the time, it might even be 
implemented :). Right now we are planning a new project involving a 
different AI, so it will take our time for the next three or four 
years. This project's AI will be used to control various criminals 
syndicates moving around in a large world, actually carrying out there 
dirty work (like transporting and selling drugs, etc.).  

Game:       Fighting Wu-Shu (also known as Fighting Bujutsu)
Type:       Arcade Fighting
Publisher:  Konami
Developer:  Konami
Release:    Available Now
Web:        http://www.konami.com

Details:

Announced at the 1997 JAMMA arcade show in Tokyo, Japan, Fighting Wu-Shu (also called Fighting Bujutsu depending on whether you're in China or Japan) is a 3D arcade fighting game in the tradition of Sega's Virtua Fighter series of games. What makes it interesting from an AI point of view is that its developers are touting the AI's ability to learn the player's moves and adjust to them as the game progresses, thus "...creating a game that advances in difficulty as you advance in skill..."

Commentary:

There would appear to be a spate of games that attempt to "learn" what the player is doing. This game promises what VF2 promised but arguably failed to deliver...an opponent that truly adjusts itself to your fighting style.

There was no word in the various press releases of the type of technology being used in the game, whether it's a simple set of cascaded finite state machines, a min-max tree, or something else entirely. Knowing the Japanese it may prove difficult to get more information, but I'll work on it.

Game:       Formula One Grand Prix 2
Type:       Racing
Publisher:  Microprose
Developer:  Virgin Interactive
Release:    Available Now

Details:

According to PC Review, this racing game has artificially intelligent race track rivals based on real drivers from the sport. Each driver has a personality which determines their driving style. Cut off an aggressive driver and you'll get side-swiped in revenge. The intention is to give the game more of a feel for true racing strategy than can be obtained by using the "generic" drivers found in other racing games.

Commentary:

What I like about this is the use of real Formula One driver personalities to guide the game AI. I'd like to see more of this in other games. Imagine building AI "templates" based on real-world tank drivers for a tank game, pilots for an air-combat game, or squad leaders for a first-person tactical combat game.

Game:       FX Fighter              
Type:       Fighting 
Publisher:  Argonaut Software Limited 
Developer:  GTE
Release:    Available Now

Details:

One of the first 3D fighting games available on the PC, FX Fighter apparently makes use of a rules-based AI which "allows the computer opponents to recognise patterns in your attacks". This would seem to imply a learning process in which moves you are fond of are eventually noted and countered from a database of counter-move available to a given fighter.

Commentary:

That's about all I know about this one. The approach described would be fairly straightforward to implement given the nature of these games. Seems similar to the approach used in Virtua Fighter 2.

Game:       Galactic Civilization    
Type:       Strategic Space 
Publisher:  Stardock Systems                    
Developer:  Below Zero (now out of business)
Release:    Available Now (PC-OS/2 only)

Details:

Okay, you know the drill. You start with one tiny planet in the middle of the Vast Unknown. Expand, Explore, Exploit, and Exterminate your way to Galactic Domination!

Galactic Civilization is regarded in the strategy gaming circles as one of the best strategic space conquest games going, if not the best. Developed exclusively for OS/2, GalCiv makes use of OS/2's multi-process threading for the game AI.

Commentary:

I know very little about this game beyond what's listed above, as I don't have OS/2 so I never really investigated it. The ability to use multi-threading has got to be a boon, since the AI can continually reevaluate its strategy in the background.

If anybody else can tell me more I'd love to hear it.

Game:       Galapagos               
Type:       Adventure
Publisher:  Anark          
Developer:  Anark
Release:    Available Now
Web:        http://www.anark.com 

Details:

Galapagos uses a form of artificial life called Non-Stationary Entropic Reduction Mapping (NERM). This technology, a special form of feedback-based controller technology exclusive to Anark, is embodied in the game's protagonist. Mendel is a synthetic organism that adapts to his environment without the player's intervention or assistance. He sees radiation in several spectra, much like a bat, by emitting sound pulses and measuring the strength of the returning signal. In order to progress through the strange and exciting worlds found in Galapagos, you must activate objects near Mendel, affecting his environment in various ways. Thus, you coax him to solve the many puzzles that occlude his progress. Of course, Mendel is an independent thinker and may have other ideas.

To quote from NEXT Generation:

"More than any other title ever previewed in NEXT Generation, the
technologies pioneered in this title may significantly change the 
way we play games in the next several years." 
				--NEXT Generation, p.116 Dec 95

Game:       Half-Life               
Type:       First Person Shooter
Publisher:  Sierra          
Developer:  Valve Software
Release:    Available Now
Web:        http://www.sierra.com 
            http://www.valvesoftware.com/hocopus/halflife.htm
            http://www.planetquake.com/half-life

Details:

Promising to usher in a new era of realistic first-person gameplay, Valve Software's Half-Life (HL) has been gathering kudos since the first screen shots hit the Web. Yes, it looks gorgeous. Yes, the polygon counts are enormous. Yes, the lighting effects are outstanding.

But we don't care about that here. We want to know how smart it is. And the programmers at Valve were very careful to promise "a whole new level" of NPC and monster AI unmatched in any game of its genre:

• A "modular" AI system that provide "unparalleled" levels of monster AI, and which will be fully accessible by the player via a programming interface;
• Monsters which watch you, smell you, hear you, track you;
• Monsters smart enough to run from a fight that they're losing...and to fetch reinforcements;
• Squads of enemy soldiers that actually move in formation, just like the Army trains its squads to do, with covering fire and fallback positions;
• Monsters that use flanking moves, that will sneak up behind you, and which will lure you into traps and ambushes.
• Monsters that can tell dirty jokes and work calculus problems in their heads (okay, I made that one up).

   Traditionally, game AI is a set of hard-coded if-then decisions for every
possible situation that could confront a monster, such as, "If there is a bad
guy in this room then shoot at him." Valve took another tack, designing a
module-based AI system that provides practically infinite flexibility and
monster growth potential. Below are just a few of the ways that AI decision
modules work together to produce unprecedented monster intelligence. 

   Monster behavior based on player's actions moment by moment: In Half-Life,
monsters might advance only when it makes sense to. They assess how much
health the player may have, where the player is heading, how many of their own
kind are left in a room, and whether they have enough health themselves to
fight. Such conditions and others dictate whether a monster will chase,
attack, or retreat. While in other games monsters are basically suicide
squads, in Half-Life monsters don't want to die.

   Half-Life makes use of both squad- and flocking behavior to give its
monsters more lifelike responses.  Adversaries can make a threat assessment,
recruit others and then plan a coordinated attack against the player.
A detailed "Flocking Behavior Model" realistically depicts the organic 
movement of animals such as birds and fish.

   Schedules of behavior tell the character what it should be doing. If, for
example, you were picking off a baddie pretty good and it was two hits away
from being killed, it would be in a "panicked" state. The schedule of behavior
would be to take cover; the cool thing is that for every task (i.e. take
cover) there are several different schedules available to achieve that goal.
The upshot is that you won't have 10 baddies all do the same thing at the same
time. 

Game:       Heavy Gear               
Type:       Action-Combat
Publisher:  Activision          
Developer:  Activision
Release:    Available Now
Web:        http://www.activision.com 

Details:

Following up on Activision's Interstate '76 and MechWarrior 2, Heavy Gear (HG) is another 3D Mech-combat game. What makes it of unusual interest to the Game AI community is its use of neural networks as part of your Gears' (the name of the 'bot you drive) control mechanism.

According to an interview in the August, 1997 issue of Computer Games Strategy Plus, game associate designer Dustin Browder reveals that an individual Gear contains several "brains", each of which controls some aspect of the 'Mech and which uses a small, specialized neural network. These NNs obstensibly will learn as the game proceeds, learning from the player's actions.

   "They (the brains) learn as the game proceeds.  Your Gear might
    get faster, it might reload its weapons a little more quickly.
    Whatever you use more often, whatever you teach it, it will
    learn to enhance."

Game:       Interstate '76               
Type:       Action-Combat
Publisher:  Activision          
Developer:  Activision
Release:    Available Now
Web:        http://www.activision.com 

Details:

A 3D-action adventure combat game set in parallel '70s universe, Interstate 76 was in part built using scripts to control the enemy AI. AI programmer Karl Meissner was kind enough to provide me with a wealth of information concerning the design decisions that led to this implementation. I quote from these below, though you should note that since the discussion was spread over several emails I've combined them into one document so as to remove redundancy (and besides, I didn't think anybody else wanted to read our discussion of 70s pop music).

Hi Steve,

   I was reading your web page and I noticed your collection of posts 
about extendible AI.  Several people were speculating on using 
scripts. We found them to be very useful on Interstate 76.  
I wrote an AI language to control the cars and mission level behavior.  
The script specified high level behavior of the cars such as a attack, 
flee or race.  This allowed the designers to make missions with a lot 
of variety in them.  There are races, attacks on convoys, infiltrating 
bases and lots of other fun stuff.  It also allowed us to build a 
wingman AI car - "Taurus" who helps you out in the early missions.  

   The scripts were compiled to an op code language that was run on a 
virtual machine in the sim.  The overhead of the virtual machine is 
probably less than 1/2% of the CPU time so it is efficient.  The down 
side is that to get a complex mission, you need a complex script.  
This meant the designers spent a lot of the development time writing 
scripts.  But it was worth it in the end because it made each mission 
unique and a challenge.

   It was about month to write the high level language and the op code
language and build the VM interface in C.  Another guy on my team wrote 
about 80% of the high level compiler.

   The scripts define each mission and these are all come predefined 
with the game.  If Activision releases the compiler and documentation, 
players could make their own mission scripts.  However, management did 
not want to do this.  If Quake-C continues to be a success 
and there seems to be people who want the AI, we might release it.

   (I would like to see this happen.  On the game CD there is a level 
editor, which allows people to lay out terrain for multiplayer maps, 
but not AI.)

   The language uses a C like grammar but is very different from normal
function based programming.  The language, which is dubbed FSM 
(Finite State Machine),
is a concise way of setting up a bunch of concurrent AI processes that 
communicate with each other.  Each process is a finite state machine.  
Each state in a process makes an object in the game do some high level 
behavior.  The process changes state based on the changing conditions in 
the game or what the other processes are doing.

   For example:  a simple guard

    state1   sit(guard1){
       if (the player comes near)
          goto state2
       }

       state2  attack(guard1, player) { 
       if (isDead(guard1))
          stop
       }

   This works as follows:  The guard sits there.  When the player shows 
up, the guard attacks until dead.  This state change can trigger other 
things like setting an alarm variable that the other processes respond 
to, thus summoning  more AI guards, closing gates etc...

   The core part of the language specifies how to set up states, 
launch processes, and manage communication.  The "if, goto, {..}" 
stuff.

   The second part of the language is game specific functions that are
defined by a simple include file.  This is "sit, attack, the player 
comes near, isDead", etc.  This makes the language generic enough for 
any game.  The sim still has to do all the work of attack(guard, player).

   The mission editor calls the FSM compiler, which is a separate
program, to generate op code.  The op code is then included with the
mission for run time on the VM.



In response to a question regarding how difficult it was
to get the designers to use this tool:



   No, simplicity (as opposed to conciseness) is the inverse of 
generality and power.  Each mission had to do very different things.  
As far as training goes, it depends on the trainee.  Four of the 
designers had never programmed before.  The hardest thing was to 
actually teach them what a variable is.  I trained them for about two 
days and then answered a lot of questions and gave examples for about 
a month.   Two programmers who also had to do some stuff with mission 
just need documentation and examples and picked it up in about a day.

   I would say the key is good examples.  They did a lot of cut and 
paste, but then started to experiment.  Experience helps a lot.  It 
depends on your time constraints and the people involved.  Originally,  
there was much more fine grain control over the attacks.  As the 
deadline approached, this all got boiled down to the function, 
attack(me, him), and then the sim programmer (me) did all the work.  

Game:       The Chronicles of Jaruu Tenk 
Type:       Sim-life/Ant-farm/God Game                  
Publisher:  Gee Whiz! Entertainment
Developer:  Gee Whiz! Entertainment
Release:    Available Now

Details:

The Chronicles of Jaruu Tenk (JT) uses A-Life technologies to immerse the player in the virtual world of its hero, Jaruu Tenk. Described as "part virtual pet and part Artificial Intelligence companion", Jaruu is a Norn-like creature who lives on an island in the middle of the ocean. You the player interact with Jaruu and his companions as they and you explore their world.

The designers cite Activision's Little Computer People as one of the inspirations for JT. They're particularly excited about the realtime conversations the player can have with Jaruu and his companions; they will remember previous discussions and build up their vocabulary over time, modifying their behaviors based on what the player tells them.

Commentary:

Sort of a step above Petz but a notch below Creatures in complexity, Jaruu Tenk seems aimed mostly at the younger crowd (that's not a slam). The AI in the game really boils down to two interesting bits of technology to judge from the writeup...the first being the English language parser (named ALPS) and the second being the behavioral A-Life aspects of the game. My best guess it that, as with Petz, JT makes heavy use of cascaded fuzzy state machines to provide behavior for its critters, tied in with a modifiable conversational database to provide greater interaction and "memory".

One topic of discussion at the 1998 CGDC was the possibility of building "companion" AIs....non-player characters that would assist the player in some fashion. This style of technology is ideally suited to just such an application, and I expect we'll be seeing more of this kind of thing in RPGs over the next few years.

Game:       Mag-Ball 
Type:       Futuristic Sports
Publisher:  GreyStone Technology, Inc.
Developer:  GreyStone Technology, Inc.
Release:    Unknown
Web:        http://www.gstone.com 

Details:

Mag-Ball is a futuristic 3D sports game modelled somewhat like a technologically enhanced version of ice hockey.

Commentary:

Richard contacted me after seeing these pages and we began a very interesting exchange concerning AI in games. Richard pointed me at Pattie Mae's pages , upon which much of the AI in Mag-Ball is based.

GreyStone is making use of human behavioural modelling for the game, taking the playstyles of actual ice hockey players and mapping them into the computer players for Mag-Ball. This technique is essentially the same as the behavioural cloning approaches used by Bullfrog in many of their upcoming games.

By the way, GreyStone is a defense conversion company, so the parallels between them and the work we're doing here at Lockheed-Martin are interesting to me just from a personal point of view. More details as I have them.

Game:       Mindrover:  The Europa Project
Type:       Strategy/Action
Publisher:  CogniToy
Developer:  CogniToy
Release:    Available Now

Details:

Mindrover is a realtime strategy/action game with a difference--rather than depend on what the AI programmer who made the game thinks is good combat behavior, the player can "roll his own" AIs for his robotic units. In fact, the AI is just one of many components that make up a unit in Mindrover...the player must build his robotic army from parts both bought and salvaged.

The game uses a powerful scripting language named ICE to make all this as simple as possible. The player can control how various components are wired together and build the AI for his units either through direct file editing or through the program's fairly intuitive graphical user interface. (Note that direct editing of ICE files isn't supported just yet, but is coming in a future patch/release.) A series of tutorial walks the user through the potentially confusing aspects of programming and wiring together components in a slick fashion.

Of course none of this would be much fun if you couldn't pit your inventions in mortal combat agaisnt those created by others, and CogniToy has gone out of their way to make that fairly easy. There's a large support area on the Cognitoy web site, where users can trade files, swap information, download new components from Cognitoy, and organize combat sessions. Several indepedent web sites have sprung up around the Internet, and the company has been talking "sequel".

I've gone for a bit without mentioning that CogniToy also has an SDK of sorts for Mindrover named TWiki. It's pretty neat and has its own support page which answers all kinds of questions, and as with the robot pages has a big fan base supporting it. Pretty neat, CogniToy.

Commentary:

Based on the number of emails I've received since this game came out, I'd rate it as second only to The Sims as the current favorite of AI tinkerers. The ability to build what amounts to your own "virtual robots" and then program them for exploration and combat recalls the kind of fun programmers had with the venerable Omega.

Mindrover certainly does have a lot going for it. The ICE interface is clean and easy to use, and it's a snap both to build your robots and to trade them with others.

From an AI perspective Mindrover isn't doing anything terribly sophisticated. The ICE interface is essentially a fancy Finite State Machine editor with some options for random decision making tossed in. The challenge lies in learning what various components can do when wired to various other components, and in designing your robot's FSM flexibly enough to handle unexpected circumstances.

Still, the game is fun and a great way to fiddle with "roll your own" AI concepts. A lot of other games could learn a lesson or two from Mindrover. Hats off to CogniToy for both their SDK and the openess of their AI in general.

Game:       Minions
Type:       Online Mutiplayer 3D Fighter and "Pet Simulator"
Publisher:  HardCoded Games
Developer:  HardCoded Games
Release:    Not Announced

Details:

Minions is an upcoming online multiplayer game with elements that might be familiar to anybody who played Black and White. According to the web site, players advance in the game by developing a team of "pets" (similar to the main Creature in Black and White) and the pitting them in battle against other player's pets. Key to the game is that your pets can grown and learn over time. Each pet is unique because of it receives personalized training from it's owner.

Commentary:

That's about all I know, actually. There haven't been any announcements on when this game is coming out that I know of. Of interest is that one of the HardCoded Game folks did recently do a chapter for the upcoming Game Programming Gems 3 on machine learning that was quite interesting.

More when I know it. Could be kinda fun, especially for those who were big fans of Black and White.

Game:       Mission:  Impossible
Type:       Adventure
Publisher:  Ocean
Developer:  Ocean
Release:    Available Now

Details:

Aimed at the Nintendo-64 marketplace, Mission: Impossible promised to include a "...new AI language to produce social interaction in a way gamers haven't seen before...". Every NPC in the game would have their own motives and agendas which would (it was said)provide a highly realistic adventure.

Commentary:

Didn't happen. Given the AI's promised design being somewhat similar in concept to ideas discussed in the Extensible Game AI thread, I had decided to go ahead and add it (on a provisional basis) to this page. As it turns out, it didn't really deliver what was promised, showing no particularly interesting AI on the part of the NPCs and nothing different from, say, GoldenEye.

Game:       Myth
Type:       Realtime Tactical
Publisher:  Bungie Software  
Developer:  Bungie Software
Release:    Available Now
Web:        http://www.bungie.com/

Details:

Another entry in the Command & Conquer/Warcraft II genre of games, Myth takes a look at realtime tactical combat rather than realtime strategic. What makes it interesting from an AI point of view was the intent (didn't happen, but might with a patch) to add an extensible AI capability.

According to an interview in the March, 1997 issue of Computer Games Strategy Plus, the game was to sported a Java-like scripting language that would let the user write their own AI for their characters/armies. These scripts would have been written ahead of time by the player, then loaded into the game at startup.

An example cited was that of an Archer unit, which might normally split its fire across enemy units as they charge your armies. With a script, however, you would have been able to modify their behavior so that they might instead focus their fire on Dwarves. Great flexibility was promised in the scripts, outlining everything from ambush tactics to flanking maneuvers. Bungie also planned to provide a web page upon which players of the game ould exchanges scripts that they've written--a great idea long overdue.

Commentary:

Unfortunately, Bungie was unable to put the scriptable, extensible AI capability into the game (and they apparently had trouble getting their pathfinding right as well). From various posts and tidbits scattered about the Net, I understand that the AI developer quit midway through the project, putting a definite cramp on Bungie's plans. As of this writing Bungie hasn't said much about the deleted features, though there are rumors they will resurface in a patch in early '98.

Extensible and modifiable AI scripts are an idea which I'm terribly glad to see somebody putting into actual practice. Several of us on the comp.ai.games newsgroup touched on this very approach a few months ago in the Extensible Game AI thread; scripting was one of the methods we kicked around at some length. It's a pity Bungie had to drop the idea, and I hope they're able to resurrect it in a patch later.

Game:       Nooks and Crannies
Type:       Realtime Strategy
Publisher:  TBA  
Developer:  And Now
Release:    Available Now

Details:

An odd cross between Command and Conquer and Creatures, Nooks and Crannies (N&C) builds on genetic-based artificial life technology to provide an interesting game.

In the game, players must 'breed' lifeforms they find on a desolate planet (one species are the 'Nooks'; the others are the 'Crannies') into better war machines. Each individual creature has a strand of DNA that controls nearly every characteristic of the animal, from its extenal appearance to the way it reacts to stress. Each creature reproduces by splitting itself when it has enough food, but this split introduces some random mutations to the creature's DNA. By carefully selecting which creatures survive, the player can shape their evolution and tailor them to the needs of the war effort.

As the preview article in the July, 1998 issue of Next Generation magazine points out, there's tremendous potential here for both A-Life enthusiasts and wargamers. You can play the game Creatures-style, taking care of your critters and just generally helping them to evolve, or you can focus on the gaming aspects to breed all kinds of specialized organic killing machines. There will also be multiplayer options, which opens up possibilities for trading of particularly interesting DNA strands over the Internet.

Commentary:

As a big proponent of A-Life technologies, as well as a fan of the advancements made in the field by games such as Creatures, I find this upcoming game particularly fascinating. If the developers of N&C can deliver Creatures-class A-Life in a C&C style environment, they'll have advanced the state of the art another notch.

I'll definitely be watching this one with great curiosity.

Game:       Petz (Dogz, Catz, Oddballz)
Type:       Sim-life/Ant-farm/God Game                  
Publisher:  PF.Magic
Developer:  PF.Magic
Release:    Available Now

Details:

The Virtual Petz products are games in the vein of Creatures in that they're as much "electronic pets" as they are games. The Petz line covers a variety of critters, including Catz, Dogz, and Oddballsz (you need to see them to understand).

What's interesting about the Petz products is, like Creatures, it attempts to provide very realistic behavior from its electronic animals. They do this using a variety of techniques and an adaptive AI (presumably some form of self-modifying fuzzy state machines) that reacts to the user's actions...how much he plays with one pet vs. another, how he punishes an animal, etc. Apparently you can drive your pets quite insane if you really want to.

Developer Andrew Stern, self-professed fan of these pages and the behavior/AI designer at PF.Magic behind much of the Petz AI, recently provided me with the following information about Petz. He discusses some of the new features in their latest games, DogzII and CatzII, and the types of behavioral AI that he implementated for these games:

   The biggest new feature is full-on 3D multiple character interaction.
Dogz and catz can now play with each other, play with toys together, wrestle 
each other, chase, carry each other, tug-of-war, follow-the-leader, groom 
each other, etc.  Petz can now form dramatic relationships with each 
other (e.g., enemies, buddies, parent-child nurturing, etc.) which develop 
over time as the pets age, influenced by how the user interacts and trains 
them.  Specifically users can give positive and negative reinforcement to 
any behavior in the system, allowing users to not just train petz to do 
tricks but to train them to like or dislike certain toys, to stop rubbing 
their butt on the ground, to start behaving nicely with one another, etc.  
The Petz have a simple adaptive AI which pays attention to how much the 
user likes or dislikes other petz, activities and toys, and modifies its 
weights and attitudes towards them appropriately.  They also remember 
their history of interaction with the user and can express varying stages of 
neglected feelings, up to the point where they may "runaway", our equivalent 
of death.  

   Do these projects matter to the traditional gaming AI community?  I 
feel it should, and if it doesn't now, it will.  Once computer opponents 
begin expressing more lifelikeness, feeling and emotion (which should 
greatly enhance the gaming experience), there will be no turning back.  
All of the techniques we are using will need to be used.  

   But what I think is interesting, and what we've been talking about at 
conferences, is not the details of the AI itself, but how we are striving to 
create an illusion of life, of which AI is a major component.  
We concentrate equally on intelligence, personality, interactivity, and 
reactiveness.  In fact we would argue that a broader definition of 
"intelligence" really means "the illusion of life".  At least that's 
the way the term AI is often used by people, or what it implies --
they really mean "alive" or "lifelike".  

Game:       Platoon Leader
Type:       Turn-based Tactical 
Publisher:  Unknown
Developer:  Brainstorm Entertainment
Release:    Unknown

Details:

Briefly mentioned in a recent issue of Computer Gaming World, Platoon Leader promises to provide small unit WW-II tactical conflict similar in many respects to Close Combat. In addition to the standard psychological-level AI modelling that has become rather standard for any WW-II tactical level game (thank Avalon Hill's Squad Leader for that--I used to play that game a lot), it will also offer an AI using a fuzzy logic system.

Commentary:

The article in CGW really didn't offer any more detail on the game, and (so far as I know) it doesn't have a publisher yet, so I don't know any more beyond the simple claim above. Fuzzy logic AIs have not been much used in games to date, however (leaving aside the debate over whether 'fuzzy state machines' are really fuzzy logic or not), which makes PL rather interesting from an AI point of view. Only BC3K and Close Combat have offered similar AIs, with somewhat mixed results to date. I do think that a fuzzy logic AI might make a lot of sense for psychological profiling and action/reaction situations.

More on this as I know it.

Game:       Return Fire II
Type:       Action Strategy (?)
Publisher:  MGM Interactive
Developer:  Prolific Publishing, Inc.
Release:    Available Now
Web:        http://www.returnfire2.com/

Details:

Return Fire 2 (RF2) is the followup to a fairly popular action shooter game released in 1996. The new game is promising larger maps, more vehicles, and a true 3D engine.

What makes this game interesting from an AI point of view is that it claims to use genetic algorithms for the game AI. According to a preview article in the July, 1998 issue of Next Generation magazine, the AI "...was designed by analyzing thousands of games, so enemies are finally capable of developing strategies worthy of human opponents."

Commentary:

That's really all I know so far. There's no information of any kind about the AI (or much of the game itself, actually) on the web site, nor have I heard much of interest from players. I'll keep an eye on it and see what comes out. It's interesting to see another game claiming to use genetic algorithms, though I wonder if they're truly being used within the game engine proper or as a "tuning" device through which a stronger, though static, AI opponent can be "evolved". If the former, I'm very impressed; if the latter, I'm still impressed, though it's not the only game coming out that makes use of GAs for tuning purposes.

Game:       Roboforge
Type:       Action
Publisher:  Liquid Edge Games
Developer:  Liquid Edge Games
Release:    Available Now
Web:        http://www.roboforge.net/

Details:

Roboforge (RF) is an interesting game that's part shoot'em up, part action extravaganza, and part AI programming exercise. Roboforge lets you construct giant robots, train them to think and fight, and them pit them against each other in massive tourneys. You can actually even win money and prizes...who says you'll never amount to anything playing games?

Your robots are built using a variety of components. Many ship with the game, together with a variety of textures (or "skins") to further customize your warriors. A 3D interface makes assembling the components into a robot pretty easy, and the only limitations are your imagination and the limits, if any, of the tournament you want to enter.

AI comes in when you move to train your robot. There's an "AI Wizard" of sorts that can help you build your AI or select from some existing approaches, but you can customize anything it comes up with yourself. You can also download AIs from a variety of web sites if you've found one from somebody else that you like better. The robot can process any information it receives from its sensors (one of the components mentioned above) and then make decisions bout what to do. The AI is generally rules-based; i.e., "If my opponent is in front of me and within 3 meters, throw a punch". You can of course test your robot and its AI before you send it away to actual combat.

Once in combat you can either play for fun, or for prizes. The Roboforge folks sponsor regular tourneys and the like (rather like a sports circuit) and there's always something new to try. The big tourneys have a variety of prizes and, if you make it all the way to the top, a $10K prize. Not bad, all things considered.

Commentary:

The whole approach is more rules-based than anything else, but that doesn't lessen the impact of the game itself. Running regular tournaments and handing out substantial prizes has attracted quite a largish following, and there are a variety of web pages dedicated to the game, playing it smarter, descriptions of robotic construction and AI, etc. The AI is specialized for fighting games, making it somewhat more restrictive than some others, but the basic approach is useful and the whole package is a great way to learn more about rules-based AIs in general.

Nothing hugely ground-breaking, but very good to see it all put together in a slick package like this. Well worth looking at (and visiting the link above).

Game:       7th Legion
Type:       Realtime Strategy
Publisher:  Epic Megagames
Developer:  Epic Megagames
Release:    Available Now

Details:

7th Legion looked like it would be a one of the first realtime strategy games to implement a realtime learning neural network (NN). In early press Epic was making the following claims:

"This means that rather than use a pre-determined set of choices 
in reaction to what a player might do, it will watch what the player 
does, and learn from past actions."  

The AI in 7th legion, works on a weighting system. There are certain 
"jobs" that units can do, and what they are doing at any one stage 
depends on whats going on elsewhere in the game. The AI will create 
base defenses to protect vital areas, set up guard points around the 
map at areas of conflict and potential infiltration and reinforce 
those areas. It will look for safe ways into your base and exploit 
them. Of course, support of other units/buildings under attack is 
done. The AI will measure up what units to send to the aid of an ally 
based on strength, current powerups etc and whether that unit is 
"worth" saving or can be saved in the time it will take support to 
help it etc. Other smaller things like picking up powerup crates as 
the AI moves around are also supported. One thing that really makes 
the AI seem "smart" is the playing of the cards in the game. 7th 
legion has magic which can be used to kill the enemy, or help you 
out. The AI weighs up conflicts, dangers etc and plays cards 
appropriately. Its looks cool, when you see a few soldiers coming 
toward your hordes, only to have the computer play BATTLE PSYCHOSIS 
on its units, and run into the fray, taking out all your troops. :-)

Game:       Seaman
Type:       A-Life Simulator
Publisher:  Sega
Developer:  Sega
Release:    Available Now (on Dreamcast)

Details:

A strange game if there ever was one, Seaman is a Sega entry in the A-Life field...and it's an odd one. More of a pet simulator of sorts than a "game" per se (even more so than Creatures or Petz), Seaman gives the user the opportunity to raise, train, and even talk to a strange race of half-fish, half-man creatures called...well, Seamen.

The main point of this game is to raise