The Game Design Mailing List "Learning AI" Thread
Hello All: 9/25/97
For those who don't subscribe, there's a fairly good mailing list
aimed at game developers called, not surprisingly, the
Game Design list.
Around the middle of 1997
Steve Schonberger
hopped onto the list, mentioned that he'd been to our CGDC AI Roundtables,
and asked the group in general if they thought it was a good idea for
a game's AI to "learn" and adapt itself to a player. This kicked off
a very interesting discussion which is presented here.
The posts from this thread are presented essentially
as is, with some *minor* editing on my part for formatting. I left the
headers and most .sigs intact. Alert readers will note that some of
the info here also made its way to the Current
and Upcoming Games with Interesting AI page, since there was quite
a bit of discussion of the learning AI built into
Age of Empires on the part of the
designers and playtesters.
Here are the e-mail addresses for those contributors as best I could
pluck them out. My profound apologies if I missed anybody; please
let me know and I'll correct this list forthwith:
Mark Atkinson
Robert Blum
Rick Cronan
Ryan T. Drake
Eric Dybsand
John Judd
Orlando Llanes
Paul Nash
David Pottinger
Steve Schonberger
Nick Shaffner
John Vanderbeck
Steven Woodcock
Hui Ka Yu
Enjoy!
Steven
From stevesch@csealumni.UNL.edu Fri Jul 11 16:43:09 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id QAA26414; Fri, 11 Jul 1997 16:43:08 -0400
Received: from smtp.gte.net by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id QAA21718; Fri, 11 Jul 1997 16:43:07 -0400
Received: from gateway-pp200 (1Cust38.Max39.Seattle.WA.MS.UU.NET [153.34.126.166])
by smtp.gte.net (SMI-8.6/SMI-SVR4) with ESMTP id PAA11409
for ; Fri, 11 Jul 1997 15:43:09 -0500 (CDT)
Message-Id: <199707112043.PAA11409@smtp.gte.net>
From: "Steve Schonberger"
To:
Subject: Re: Game AI
Date: Fri, 11 Jul 1997 13:41:40 -0700
X-MSMail-Priority: Normal
X-Priority: 3
X-Mailer: Microsoft Internet Mail 4.70.1161
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Length: 2777
Status: RO
At the Computer Game Developers' Convention, I attended a game AI
round-table (one of Steven Woodcock's sessions). An interesting topic
that came up was whether it is a good idea for game AI to learn while
playing a single human player. It seemed to me that there were more people
advocating non-learning AI models, but a few people did think learning
was a good thing.
Note: By "non-learning", I don't mean that the computer opponent
wasn't showing the appearance of getting smarter as it played the game. I
just mean that if it did get smarter, it did so by adaptively selecting
stronger AI settings, rather than actually evaluating the results it got and
adjusting its AI settings based on its evaluation.
Since this list has been pretty quiet lately, and overall pretty quiet
when things we've agreed are off topic are counted out, I figured it
deserved a new, on-topic subject, now that the AI scripting topic seems to be
dying down. There were plenty of other good AI topics in that round-table,
which I'll hold in reserve for now.
Steve Schonberger
From woodcock@real3d.com Fri Jul 11 16:17:37 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id QAA25524; Fri, 11 Jul 1997 16:17:19 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id QAA21596; Fri, 11 Jul 1997 16:17:17 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id QAA26308
for woodcock@real3d.com; Fri, 11 Jul 1997 16:16:23 -0400 (EDT)
Resent-Date: Fri, 11 Jul 1997 16:16:23 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
From: woodcock@real3d.com
Message-Id: <9707112016.AA02698@stargazer.real3d.com>
Subject: Re: Game AI
To: gamedesign@mail.digiweb.com
Date: Fri, 11 Jul 1997 16:16:10 -0400 (EDT)
In-Reply-To: <199707111920.OAA20395@smtp.gte.net> from "Steve Schonberger" at Jul 11, 97 12:18:55 pm
X-Mailer: ELM [version 2.4 PL25]
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Resent-Message-ID: <"-2SpWD.A.cXG.dRpxz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/307
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 2697
Status: RO
> At the Computer Game Developers' Convention, I attended a game AI
> round-table (one of Steven Woodcock's sessions).
Hi Steve! It was good to meet you there.
> An interesting topic that
> came up was whether it is a good idea for game AI to learn while playing a
> single human player. It seemed to me that there were more people
> advocating non-learning AI models, but a few people did think learning was
> a good thing.
That was on what, the first day?
I thought it was interesting how many developers did *not* think it was
a good idea, vs. those who thought it might be but just didn't have the time
to do it or who didn't think it was appropriate for their current
project. Many of the developers present (as you say) thought learning
and adjusting could be a good thing (and in fact it was a major topic
on a second day of discussions when we talked about A-life and the new
game Creatures). A few, however, thought it was not a good idea at any
time...that it would either make the game too difficult for the player or that
(in a multiplayer family) it would train against Player A but become
worse against Player B.
There were definitely some interesting points made on both sides.
> Since this list has been pretty quiet lately, and overall pretty quiet when
> things we've agreed are off topic are counted out, I figured it deserved a
> new, on-topic subject, now that the AI scripting topic seems to be dying
> down.
Yes....I had hoped it would run longer. ;(
> There were plenty of other good AI topics in that round-table, which
> I'll hold in reserve for now.
I'm glad you enjoyed them. We're trying to continue with them over
on the new Gamasutra site (www.gamasutra.com).
Steve
+=============================================================================+
| _ |
| Steven Woodcock _____C .._. |
| Senior Software Engineer, Gameware ____/ \___/ |
| Lockheed Martin Real3D <____/\_---\_\ "Ferretman" |
| Phone: 719-597-5413 |
| E-mail: woodcock@real3d.com |
| Web: http://www.cris.com/~swoodcoc/ai.html (Games AI page) |
| Disclaimer: My opinions in NO way reflect the opinions of |
| Lockheed Martin Real3D--get *serious* |
+=============================================================================+
From drake@cse.psu.edu Fri Jul 11 16:24:47 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id QAA25754; Fri, 11 Jul 1997 16:24:46 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id QAA21621; Fri, 11 Jul 1997 16:24:44 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id QAA27410
for woodcock@real3d.com; Fri, 11 Jul 1997 16:23:50 -0400 (EDT)
Resent-Date: Fri, 11 Jul 1997 16:23:50 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Date: Fri, 11 Jul 1997 16:23:35 -0400 (EDT)
From: Ryan T Drake
To: gamedesign@mail.digiweb.com
Subject: Re: Game AI
In-Reply-To: <199707111920.OAA20395@smtp.gte.net>
Message-ID:
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Resent-Message-ID: <"OZvfIC.A.VqG.aYpxz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/308
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 2938
Status: RO
On Fri, 11 Jul 1997, Steve Schonberger wrote:
> At the Computer Game Developers' Convention, I attended a game AI
> round-table (one of Steven Woodcock's sessions). An interesting topic that
> came up was whether it is a good idea for game AI to learn while playing a
> single human player. It seemed to me that there were more people
> advocating non-learning AI models, but a few people did think learning was
> a good thing.
I would say that adaptive AI would make a really good option in most
cases, but it also would depend on the game. Something else that would
need consideration: Does the game remember what it learned when you quit
the game and come back in?
My feeling is, you can tell what is good AI if the computer's actions
look like the way a human would play. For instance: If I am playing
Quake, and i see a monster trying to follow me around the corner and
hitting a wall, i would not consider that good AI. On the other hand, if
i had a monster after me that hits every shot and follows me perfectly, i
would also not consider that good AI. Reason being, I know a human cannot
possibly be dumb enough to try to follow me through walls, and I also know
a human cannot possibly be good enough to hit each shot and follow
perfectly. In essence, to have good artificial intellegence, you also
need artificial stupidity.
One idea of implementing this kind of AI is to start by giving the
computer only as much information as a human would get. For example,
sticking with the Quake idea. Only give the computer information about
its surroundings, and not give it access to where everything is on the
map. If a normal player would get sound cues give the computer ai sound
cues.
Another important thing to give any AI is reaction time. No human being
has perfect 0ms reaction time. In a ``deathmatch'' for instance, a human
being could take anywhere between 100-500ms to react to events on the
screen. Add this factor in when calculating how the computer reacts to
things. Reasonable AI can vary given a user-selected ``skill level'' but
it should always be beatable.
Now to your original point... Yes, I think ideally AI should be adaptive,
so long as its adaptiveness tops off at a certain point. Anyone who
considers themself very good at a certain game can relate to what I mean.
Eventually the learning curve just stops and you are at a point that
you can't really get much better...Usually this is when you can beat most
other players and there is no one better to challenge you. This should
also happen with AI. There should be a point where the AI stops getting
better. As soon as it looks like the AI is doing a certain amount better
than the human player, it should shut off and not learn anything new.
Then as the human player starts getting better you can turn the learning
on again.
= Ryan Drake =
= drake@cse.psu.edu =
= http://www.cse.psu.edu/~drake =
From woodcock@real3d.com Fri Jul 11 16:42:53 1997
Return-Path:
Received: from stargazer.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id QAA26399; Fri, 11 Jul 1997 16:42:52 -0400
Received: by stargazer.real3d.com (4.1/1.34.a)
id AA02715; Fri, 11 Jul 97 16:42:51 EDT
From: woodcock@real3d.com
Message-Id: <9707112042.AA02715@stargazer.real3d.com>
Subject: Re: Game AI
To: gamedesign@mail.digiweb.com
Date: Fri, 11 Jul 1997 16:42:51 -0400 (EDT)
In-Reply-To: from "Ryan T Drake" at Jul 11, 97 04:23:35 pm
X-Mailer: ELM [version 2.4 PL25]
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Content-Length: 2958
Status: RO
> I would say that adaptive AI would make a really good option in most
> cases, but it also would depend on the game. Something else that would
> need consideration: Does the game remember what it learned when you quit
> the game and come back in?
If I built it it would. I'd also give the user a chance to either
play with the "adapted" or "evolved" AI vs. the "out of the box" setting.
>
> (discussion of the need for "artificial stupidity" deleted)
>
Those are all good points Ryan. One reason the Quake-bots are as good
as they are is that they can in fact react FAR faster than any human
can. A good design should always allow a player to select the AI
difficulty (much like Enemy Nations does and which (sadly) Dungeon
Keeper does NOT), which would of course mean different things for different
games. For a game like Quake or Doom reaction times are definitely
part of it.
> Now to your original point... Yes, I think ideally AI should be adaptive,
> so long as its adaptiveness tops off at a certain point. Anyone who
> considers themself very good at a certain game can relate to what I mean.
> Eventually the learning curve just stops and you are at a point that
> you can't really get much better...Usually this is when you can beat most
> other players and there is no one better to challenge you. This should
> also happen with AI. There should be a point where the AI stops getting
> better. As soon as it looks like the AI is doing a certain amount better
> than the human player, it should shut off and not learn anything new.
> Then as the human player starts getting better you can turn the learning
> on again.
I'm curious how you might spot that the human player has "leveled out"?
Taking Quake as an example, what criteria might you use? Kills vs.
shots? Average time playing a level? Some combination of the two might
suffice....that's an interesting problem.
Steve
+=============================================================================+
| _ |
| Steven Woodcock _____C .._. |
| Senior Software Engineer, Gameware ____/ \___/ |
| Lockheed Martin Real3D <____/\_---\_\ "Ferretman" |
| Phone: 719-597-5413 |
| E-mail: woodcock@real3d.com |
| Web: http://www.cris.com/~swoodcoc/ai.html (Games AI page) |
| http://www.cris.com/~swoodcoc/software.html (AI Software page) |
| Disclaimer: My opinions in NO way reflect the opinions of |
| Lockheed Martin Real3D--get *serious* |
+=============================================================================+
From stevesch@csealumni.UNL.edu Fri Jul 11 16:43:09 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id QAA26414; Fri, 11 Jul 1997 16:43:08 -0400
Received: from smtp.gte.net by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id QAA21718; Fri, 11 Jul 1997 16:43:07 -0400
Received: from gateway-pp200 (1Cust38.Max39.Seattle.WA.MS.UU.NET [153.34.126.166])
by smtp.gte.net (SMI-8.6/SMI-SVR4) with ESMTP id PAA11409
for ; Fri, 11 Jul 1997 15:43:09 -0500 (CDT)
Message-Id: <199707112043.PAA11409@smtp.gte.net>
From: "Steve Schonberger"
To:
Subject: Re: Game AI
Date: Fri, 11 Jul 1997 13:41:40 -0700
X-MSMail-Priority: Normal
X-Priority: 3
X-Mailer: Microsoft Internet Mail 4.70.1161
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Length: 2777
Status: RO
> > At the Computer Game Developers' Convention, I attended a game AI
> > round-table (one of Steven Woodcock's sessions).
>
> Hi Steve! It was good to meet you there.
Yeah, that's part of the fun of conferences.
> > An interesting topic that
> > came up was whether it is a good idea for game AI to learn while
playing a
> > single human player. It seemed to me that there were more people
> > advocating non-learning AI models, but a few people did think learning
was
> > a good thing.
>
> That was on what, the first day?
I'm not sure. It was the day when you and the other AI session leaders
talked about how one of the sessions had all of the women (one!)
> I thought it was interesting how many developers did *not* think it was
> a good idea, vs. those who thought it might be but just didn't have the
time
> to do it or who didn't think it was appropriate for their current
> project. Many of the developers present (as you say) thought learning
> and adjusting could be a good thing (and in fact it was a major topic
> on a second day of discussions when we talked about A-life and the new
> game Creatures). A few, however, thought it was not a good idea at any
> time...that it would either make the game too difficult for the player or
that
> (in a multiplayer family) it would train against Player A but become
> worse against Player B.
>
> There were definitely some interesting points made on both sides.
If I think of more to say, I'll say them on the list. Might as well see
what the rest of the list has to say about it.
> > Since this list has been pretty quiet lately, and overall pretty quiet
when
> > things we've agreed are off topic are counted out, I figured it
deserved a
> > new, on-topic subject, now that the AI scripting topic seems to be
dying
> > down.
>
> Yes....I had hoped it would run longer. ;(
But it didn't seem to be going much of anywhere. It seemed more like a
discussion of "what programming language do you script in?" than "what do
you do with the scripts?". The latter is more interesting.
> > There were plenty of other good AI topics in that round-table, which
> > I'll hold in reserve for now.
>
> I'm glad you enjoyed them. We're trying to continue with them over
> on the new Gamasutra site (www.gamasutra.com).
I suppose most of the credit for the session being interesting goes to the
other people attending, but without moderators there wouldn't be any
session at all! Also, I suppose you would have kicked things in the right
direction if there hadn't been lots of people speaking up on their own. A
lot of the session went over my head, being on the fringes of
research-grade AI, but what I understood was sure interesting.
Steve Schonberger
From DPottinger@Ensemble-Studios.com Fri Jul 11 17:01:41 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id RAA27151; Fri, 11 Jul 1997 17:01:41 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id RAA21837; Fri, 11 Jul 1997 17:01:37 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id RAA02992
for woodcock@real3d.com; Fri, 11 Jul 1997 17:00:43 -0400 (EDT)
Resent-Date: Fri, 11 Jul 1997 17:00:43 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-ID: <802B50C269DECF11B6A200A0242979EF33CFA7@consulting.ensemble.net>
From: David Pottinger
To: "'gamedesign@mail.digiweb.com'"
Subject: RE: Game AI
Date: Fri, 11 Jul 1997 16:02:14 -0500
X-Priority: 3
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.0.1457.3)
Content-Type: text/plain
Resent-Message-ID: <"xajq3.A.Po.Y6pxz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/310
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 3478
Status: RO
I guess I'd chime in with the thought that, if applied correctly, a
learning AI is a great thing.
I love cool AI, and you can't get much cooler than an AI that learns how
to beat you by watching you and other people play the game. That's an
AI I'd love to develop and love to play against. Playing humans (if
they're any good:) constantly forces you to either refine/optimize or
reevaluate/replace your strategies. If you always win easily, human
nature will tend to make you want to play something else. A good
learning AI helps fill that void when you aren't playing against other
humans.
You do have to consider the game (as Ryan Drake already mentioned). No
AI should ever make the game unfun for people to play. Does that mean
that you shouldn't do learning AI then? No way! There is a lot of room
for learning AIs to be applied in ways that still make games fun.
For Age of Empires (Shameless plug: The recent winner of best realtime
strat at E3:), we did do limited learning to augment a pretty
straightforward approach to the AI. When you play any of our campaign
scenarios the first time, the game is even. However, as a human, you
carry over information about the scenario when you replay it the next
time. So, we let the CPs do the same thing. They remember where you
attacked them or they attacked you, etc. We also let the CPs remember
your general playing tendencies so that they can improve playing against
you in the randomly generated games. This has helped the quality of the
AI out a lot. Well enough, in fact, that we'll be able to ship the game
w/ an AI that doesn't cheat. Though, we may do a "Doom-style Nightmare
mode" where the AI overtly cheats (by way of getting a resource boost at
the start) just to pound on people who like that kind of thing.
I guess I'd have to say that I've yet to see an AI that can't be beat by
some strategy that the developers either didn't foresee or didn't have
the time to code against. Learning is a great way to help alleviate
that problem and thus create a better playing experience.
dave
Dave C. Pottinger
Engine Lead and AI Guy
Ensemble Studios, Inc.
> -----Original Message-----
> From: Steve Schonberger [SMTP:stevesch@csealumni.UNL.edu]
> Sent: Friday, July 11, 1997 2:19 PM
> To: gamedesign@mail.digiweb.com
> Subject: Game AI
>
> At the Computer Game Developers' Convention, I attended a game AI
> round-table (one of Steven Woodcock's sessions). An interesting topic
> that
> came up was whether it is a good idea for game AI to learn while
> playing a
> single human player. It seemed to me that there were more people
> advocating non-learning AI models, but a few people did think learning
> was
> a good thing.
>
> Note: By "non-learning", I don't mean that the computer opponent
> wasn't
> showing the appearance of getting smarter as it played the game. I
> just
> mean that if it did get smarter, it did so by adaptively selecting
> stronger
> AI settings, rather than actually evaluating the results it got and
> adjusting its AI settings based on its evaluation.
>
> Since this list has been pretty quiet lately, and overall pretty quiet
> when
> things we've agreed are off topic are counted out, I figured it
> deserved a
> new, on-topic subject, now that the AI scripting topic seems to be
> dying
> down. There were plenty of other good AI topics in that round-table,
> which
> I'll hold in reserve for now.
>
> Steve Schonberger
From t-pauln@microsoft.com Fri Jul 11 17:13:12 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id RAA27587; Fri, 11 Jul 1997 17:13:11 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id RAA21889; Fri, 11 Jul 1997 17:13:10 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id RAA05616
for woodcock@real3d.com; Fri, 11 Jul 1997 17:12:17 -0400 (EDT)
Resent-Date: Fri, 11 Jul 1997 17:12:17 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-ID: <4152F7B641AFCF11A49800805F680B3F3F0943@RED-36-MSG.dns.microsoft.com>
From: Paul Nash
To: "'gamedesign@mail.digiweb.com'"
Subject: RE: Game AI
Date: Fri, 11 Jul 1997 14:11:33 -0700
X-Priority: 3
X-Mailer: Internet Mail Service (5.0.1458.49)
Resent-Message-ID: <"rwreOC.A.2SB.gFqxz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/311
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 4351
Status: RO
(Comments inline. Damn MS-mail screwed up reply format...)
-Paul R. Nash, Multimedia Developer At Large
Microsoft Multimedia Dev. Intern http://www.uiuc.edu/ph/www/pr-nash/
> -----Original Message-----
> From: Ryan T Drake [SMTP:drake@cse.psu.edu]
> Sent: Friday, July 11, 1997 1:24 PM
> To: gamedesign@mail.digiweb.com
> Subject: Re: Game AI
>
> On Fri, 11 Jul 1997, Steve Schonberger wrote:
>
> > At the Computer Game Developers' Convention, I attended a game AI
> > round-table (one of Steven Woodcock's sessions). An interesting
> topic that
> > came up was whether it is a good idea for game AI to learn while
> playing a
> > single human player. It seemed to me that there were more people
> > advocating non-learning AI models, but a few people did think
> learning was
> > a good thing.
>
> I would say that adaptive AI would make a really good option in most
> cases, but it also would depend on the game. Something else that
> would
> need consideration: Does the game remember what it learned when you
> quit
> the game and come back in?
[Paul Nash]
Most definitely it should remember that. The problem of
training on player B and wiping out player A is not a problem at all,
either. Don't a lot of games have saved game information? Is it so
hard for the AI to save it's brain to disk? Not in a properly designed
AI, I say. For instance, genetic algortihms should allow the saving of
the genes to prescribe a persitent state, I would think. So save that
for each player or each saved game.
> perfectly. In essence, to have good artificial intellegence, you also
> need artificial stupidity.
>
[Paul Nash]
Yes, AI should definitely "make mistakes."
> One idea of implementing this kind of AI is to start by giving the
> computer only as much information as a human would get. For example,
> sticking with the Quake idea. Only give the computer information
> about
> its surroundings, and not give it access to where everything is on the
> map. If a normal player would get sound cues give the computer ai
> sound
> cues.
>
[Paul Nash]
...and then provide some sort of proficiency function for
understanding those sound cues.
> Now to your original point... Yes, I think ideally AI should be
> adaptive,
> so long as its adaptiveness tops off at a certain point. Anyone who
> considers themself very good at a certain game can relate to what I
> mean.
> Eventually the learning curve just stops and you are at a point that
> you can't really get much better...Usually this is when you can beat
> most
> other players and there is no one better to challenge you. This should
> also happen with AI. There should be a point where the AI stops
> getting
> better. As soon as it looks like the AI is doing a certain amount
> better
> than the human player, it should shut off and not learn anything new.
> Then as the human player starts getting better you can turn the
> learning
> on again.
>
[Paul Nash]
Sorry Ryan, but I 100% disagree. You've just shot down your own
argument here, because you say that AI should level off to match the
player, but you say that the player levels off because there's noone
left to challenge them. BUT, the AI IS the challenger. If the AI is
always challenging a person, then they won't necessarily level off,
right? I don't consider this to be like weightlifting, because it's the
human brain, which is an entirely different kind of muscle. :)
That said, the AI should definitely *track* the player, and not
necessarily always be better than the human. If the AI sense the human
is thoroughly getting his butt kicked regularly, the AI is too strong
and it should back off a little to let the human "catch up."
I think it's amazing how Mavis Beacon can tell that you're
mising certain fingers and can intuitively KNOW what you're doing wrong.
The thing there is that the program has so much understanding about the
act of typing that it can diagnose your problems. Some games are like
this: "you're need to buy more armor more often" or "explore in larger
parties" might be cures for certain problems in strategy games. If you
could design an AI that can detect specific gameplay deficiencies in the
human and somehow adapt to them, that would be very cool.
Of course, I am not suggesting that any of this is really easy,
but all of it has potential.
>
From nshaf@intur.net Fri Jul 11 17:31:38 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id RAA28041; Fri, 11 Jul 1997 17:31:37 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id RAA21933; Fri, 11 Jul 1997 17:31:36 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id RAA10630
for woodcock@real3d.com; Fri, 11 Jul 1997 17:30:43 -0400 (EDT)
Resent-Date: Fri, 11 Jul 1997 17:30:43 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-ID: <33C6A704.104D@intur.net>
Date: Fri, 11 Jul 1997 16:35:00 -0500
From: Nick Shaffner
Reply-To: nshaf@intur.net
Organization: DigiFX Interactive (http://www.digifx.net)
X-Mailer: Mozilla 3.0Gold (Win95; I)
MIME-Version: 1.0
To: gamedesign@mail.digiweb.com
Subject: Re: Game AI
References: <802B50C269DECF11B6A200A0242979EF33CFA7@consulting.ensemble.net>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Resent-Message-ID: <"b0FHtB.A.6eC.xWqxz"@mail>
Resent-From: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/312
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 1317
Status: RO
Hey David, remember me? :)
> You do have to consider the game (as Ryan Drake already mentioned). No
> AI should ever make the game unfun for people to play. Does that mean
> that you shouldn't do learning AI then? No way! There is a lot of room
> for learning AIs to be applied in ways that still make games fun.
Agreed, in addition, one could design the AI with the primary purpose
of making the game fun, rather than victory. This certainly be a more
difficult task, but (for example) by assessing the users levels of
interaction with the game, the AI could forcibly create incidents to
break up 'slow' periods - or perhaps not attack the users when he's down
or running low on resources, etc...
> the time to code against. Learning is a great way to help alleviate
> that problem and thus create a better playing experience.
Agreed, so long as it is used properly - it could possibly be used to
help plug unforseen gaps in game mechanics (perhaps like the sandbag
thing in C&C) ... It can also help make the game more user extensible,
for example in Mission to Nexus Prime (plug, plug) - the user can
actually design/create completely new types of units, and learning is
essential in order to get the AI to the point where it can use them
effectively...
Nick Shaffner
http://www.digifx.net/
From drake@cse.psu.edu Fri Jul 11 17:48:42 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id RAA28424; Fri, 11 Jul 1997 17:48:41 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id RAA21992; Fri, 11 Jul 1997 17:48:40 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id RAA12833
for woodcock@real3d.com; Fri, 11 Jul 1997 17:47:47 -0400 (EDT)
Resent-Date: Fri, 11 Jul 1997 17:47:47 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Date: Fri, 11 Jul 1997 17:48:53 -0400 (EDT)
From: Ryan T Drake
To: gamedesign@mail.digiweb.com
Subject: Re: Game AI
In-Reply-To: <9707112042.AA02715@stargazer.real3d.com>
Message-ID:
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Resent-Message-ID: <"Hi7sC.A.bGD.Enqxz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/313
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 3932
Status: RO
On Fri, 11 Jul 1997 woodcock@real3d.com wrote:
> I'm curious how you might spot that the human player has "leveled out"?
> Taking Quake as an example, what criteria might you use? Kills vs.
> shots? Average time playing a level? Some combination of the two might
> suffice....that's an interesting problem.
There are certain things that make a Quake player (or any other game
player) ``good.'' I will break it down into 3 categories for now: skill,
tactics, and strategy. Although they may sonund the same they have very
different meanings when you look at them from an AI perspective...
SKILL
I consider ``skill'' to be mastery of a game's physical interaction. For
instance reaction time would be a skill. The ability to perform complex
maneuvers with the joystick or keyboard would also be a skill. Think
about Command and Conquer--the ability to keep two different sets of tanks
driving in two different directions (with keyboard shortcuts or by
flipping back and forth between the two groups) would also be a skill.
This is the easy part of programming AI. We start with the assertion that
a computer has 100% skill. Normally there is no reaction time and the
computer has as many fingers as it needs to control itself and its
character. By adding code to modify the computer's reaction time, or
limiting the number of things it can keep track of, you are lowering the
computer's SKILL to a human's level.
TACTICS
Tactics describe a player's specific reactions to the game environment and
to his or her opponents. In Command and Conquer, if my opponent starts
driving 20 tanks in the direction of my base, I will react by erecting a
few turrets and sending a force of my own out. Tactics vary considerably
from player to player. They can be thought of as a person's style of
gameplay. In Quake if I make a habit of going for health boxes whenever I
drop below 30%, that would be a tactic. Tactics are also rather trivial
to program, and a rudimentary set of tactics can be programmed with a
bunch of if (player does this) then (react with this action)
STRATEGY
Strategy is a little harder to pin down. I would say this is the overall
flow of your game. Another way of putting it is strategy is "A player's
guess at what set of tactics will work to defeat the opponent" How am I
going to win the game? Some are conservative and usually work fairly
well, some are risky, but can really hammer an unsuspecting opponent.
From an AI perspective, the first part of the problem is figuring out an
optimal strategy given an opponent's apparent tactics. This is an
EXTREMELY complex problem, and something I consider a very exciting part
of computer science. They couldn't even do it with Deep Blue, because
Kasparov's _tactics_ were designed to fool DB, and DB then chose a bad
strategy. I believe if anyone can really find a way to program a computer
so that it looks at a game, analyzes the opponont's moves and trys to
figure out what he is thinking--the person that can program this will
become a legend in the gaming industry ;-) Think about how many factors
you would have to consider. You would have to be able to quantitatively
describe how ``agressive'' an opponent is. How do you calculate that?
How do you decide what will fool a human being?
After your AI decides on a certain strategy, then you have to be able to
implement it. This would be done by deploying a number of tactics that
make up this strategy, but which ones? and when?
A truly successful AI would be able to evaluate its opponent's tactics,
come up with an overall strategy, and translate that strategy into tactics
of its own. A game that can do all this would be incredible.
I hope this sorta explained what I was talking about. Heh i didn't mean
for it to get this long, but once i start writing.... ;-)
= Ryan Drake =
= drake@cse.psu.edu =
= http://www.cse.psu.edu/~drake =
From woodcock@real3d.com Fri Jul 11 18:48:39 1997
Return-Path:
Received: from stargazer.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id SAA29540; Fri, 11 Jul 1997 18:48:38 -0400
Received: by stargazer.real3d.com (4.1/1.34.a)
id AA02847; Fri, 11 Jul 97 18:48:38 EDT
From: woodcock@real3d.com
Message-Id: <9707112248.AA02847@stargazer.real3d.com>
Subject: Re: Game AI
To: gamedesign@mail.digiweb.com
Date: Fri, 11 Jul 1997 18:48:37 -0400 (EDT)
In-Reply-To: <802B50C269DECF11B6A200A0242979EF33CFA7@consulting.ensemble.net> from "David Pottinger" at Jul 11, 97 04:02:14 pm
X-Mailer: ELM [version 2.4 PL25]
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Content-Length: 4567
Status: RO
> I guess I'd chime in with the thought that, if applied correctly, a
> learning AI is a great thing.
Cool....we agree on that...
> I love cool AI, and you can't get much cooler than an AI that learns how
> to beat you by watching you and other people play the game. That's an
> AI I'd love to develop and love to play against. Playing humans (if
> they're any good:) constantly forces you to either refine/optimize or
> reevaluate/replace your strategies. If you always win easily, human
> nature will tend to make you want to play something else. A good
> learning AI helps fill that void when you aren't playing against other
> humans.
One topic brought up at the AI roundtables was how many developers
were being driven to put in multiplayer *instead of* good game AI.
Though in the minority (thankfully), some folks reported that their
management was far more interested in making sure a game was Internet
capable than in making it play a challenging solitaire game. IMO,
that's a poor vision of the future...as much as I enjoy online gaming
I like to *learn* the game alone for a bit first. Fortunately as
I said, that viewpoint didn't seem to be in the majority.
> For Age of Empires (Shameless plug: The recent winner of best realtime
> strat at E3:), we did do limited learning to augment a pretty
> straightforward approach to the AI. When you play any of our campaign
> scenarios the first time, the game is even. However, as a human, you
> carry over information about the scenario when you replay it the next
> time. So, we let the CPs do the same thing. They remember where you
> attacked them or they attacked you, etc. We also let the CPs remember
> your general playing tendencies so that they can improve playing against
> you in the randomly generated games. This has helped the quality of the
> AI out a lot. Well enough, in fact, that we'll be able to ship the game
> w/ an AI that doesn't cheat. Though, we may do a "Doom-style Nightmare
> mode" where the AI overtly cheats (by way of getting a resource boost at
> the start) just to pound on people who like that kind of thing.
This is *very* interesting Dave. Might I impose upon you for a more
detailed writeup so I can add it to my Games AI page (address below)?
If so, please email me at my "home" address (swoodcoc@concentric.net).
More on subject, I'm surprised that you actually found strategies to
be similar enough from game to game to make saving such information
useful. If I replay a given scenario (and I'll admit it has to
be pretty compelling for me to do so) I usually try something different
than I did the time before.
Your solution of saving general playing tendencies (which I presume
are things like types of units the player likes to build, battle
formations they prefer, etc.) for randomly generated games seems very
clever. The AI in C&C, for example (just to pick on the game we
picked on in the roundtable discussions) seems to be especially tuned
for the "canned" scenarios and flails somewhat when presented with
a new user-designed map.
> I guess I'd have to say that I've yet to see an AI that can't be beat by
> some strategy that the developers either didn't foresee or didn't have
> the time to code against. Learning is a great way to help alleviate
> that problem and thus create a better playing experience.
Agreed. Adaptation is one of the things promised in the large
online games (such as Ultima Online) and seems to me to be a natural
next step for AI in games.
Steve
+=============================================================================+
| _ |
| Steven Woodcock _____C .._. |
| Senior Software Engineer, Gameware ____/ \___/ |
| Lockheed Martin Real3D <____/\_---\_\ "Ferretman" |
| Phone: 719-597-5413 |
| E-mail: woodcock@real3d.com |
| Web: http://www.cris.com/~swoodcoc/ai.html (Games AI page) |
| http://www.cris.com/~swoodcoc/software.html (AI Software page) |
| Disclaimer: My opinions in NO way reflect the opinions of |
| Lockheed Martin Real3D--get *serious* |
+=============================================================================+
From woodcock@real3d.com Fri Jul 11 19:05:12 1997
Return-Path:
Received: from stargazer.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id TAA29901; Fri, 11 Jul 1997 19:05:10 -0400
Received: by stargazer.real3d.com (4.1/1.34.a)
id AA02884; Fri, 11 Jul 97 19:05:08 EDT
From: woodcock@real3d.com
Message-Id: <9707112305.AA02884@stargazer.real3d.com>
Subject: Re: Game AI
To: gamedesign@mail.digiweb.com
Date: Fri, 11 Jul 1997 19:05:07 -0400 (EDT)
In-Reply-To: <4152F7B641AFCF11A49800805F680B3F3F0943@RED-36-MSG.dns.microsoft.com> from "Paul Nash" at Jul 11, 97 02:11:33 pm
X-Mailer: ELM [version 2.4 PL25]
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Content-Length: 4677
Status: O
> (Comments inline. Damn MS-mail screwed up reply format...)
I hate when that happens. That's why I only use Unix Elm (tm).
Command line is where it's at! ;)
> > On Fri, 11 Jul 1997, Steve Schonberger wrote:
> >
> > Does the game remember what it learned when you quit
> > the game and come back in?
> [Paul Nash]
> Most definitely it should remember that. The problem of
> training on player B and wiping out player A is not a problem at all,
> either. Don't a lot of games have saved game information? Is it so
> hard for the AI to save it's brain to disk? Not in a properly designed
> AI, I say. For instance, genetic algortihms should allow the saving of
> the genes to prescribe a persitent state, I would think. So save that
> for each player or each saved game.
In fact GAs were one of the methods of building a learning AI
discussed in the sessions, mostly with regards to the Creatures game
and its A-Life techniques.
One side aspect of this that somebody brought up (it might have
been me, but I'm not sure) is that by having the AI "brain" loaded
rather than coded you can release add-on expansion packs for the game
containing additional AIs. Better yet (separate topic), if the AI
code is accessible to the player (ala Quake-C), then you can sponsor
contests amongst players to see who can develop the most devious
AIs. The best 10 get released on a CD, or posted to a web site.
> >
> > (Ryan says games should throttle back to match the players' level)
> >
> [Paul Nash]
> Sorry Ryan, but I 100% disagree. You've just shot down your own
> argument here, because you say that AI should level off to match the
> player, but you say that the player levels off because there's noone
> left to challenge them. BUT, the AI IS the challenger. If the AI is
> always challenging a person, then they won't necessarily level off,
> right? I don't consider this to be like weightlifting, because it's the
> human brain, which is an entirely different kind of muscle. :)
I don't know about that, Paul. I can see value in adjusting the AI
so that it's always a bit tougher than the player is. I remember the
first few times I played C&C I was sweating bullets over the AI, but
once my expertise got high enough he ceased to become a threat in
all but the most unbalanced scenarios.
The trick is capturing parameters which you can use to accurately
judge how experienced a player is.
> That said, the AI should definitely *track* the player, and not
> necessarily always be better than the human. If the AI sense the human
> is thoroughly getting his butt kicked regularly, the AI is too strong
> and it should back off a little to let the human "catch up."
Er....isn't that what Ryan said?
> If you
> could design an AI that can detect specific gameplay deficiencies in the
> human and somehow adapt to them, that would be very cool.
Genetic algorithsm have a possibility here, provided the game itself
is "big enough" to permit them time to evolve (they can be notoriously
slow). Consider a space strategy game ala MOO, in which individual
ship designs are modified by the AI over time based on those ships
which do well against the player. If you the player tend to build
lots of fighters and carriers, then gradually over time the AI will
adapt to that by building ships based on surviving ship types
(those that have more anti-fighter defenses).
> Of course, I am not suggesting that any of this is really easy,
> but all of it has potential.
That's why they pay us the big bucks! (Well, it's rumored some folks
get big bucks...I wouldn't know..... ;).
Steve
+=============================================================================+
| _ |
| Steven Woodcock _____C .._. |
| Senior Software Engineer, Gameware ____/ \___/ |
| Lockheed Martin Real3D <____/\_---\_\ "Ferretman" |
| Phone: 719-597-5413 |
| E-mail: woodcock@real3d.com |
| Web: http://www.cris.com/~swoodcoc/ai.html (Games AI page) |
| http://www.cris.com/~swoodcoc/software.html (AI Software page) |
| Disclaimer: My opinions in NO way reflect the opinions of |
| Lockheed Martin Real3D--get *serious* |
+=============================================================================+
From stevesch@csealumni.UNL.edu Fri Jul 11 18:37:04 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id SAA29367; Fri, 11 Jul 1997 18:37:04 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id SAA22083; Fri, 11 Jul 1997 18:36:59 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id SAA20046
for woodcock@real3d.com; Fri, 11 Jul 1997 18:36:02 -0400 (EDT)
Resent-Date: Fri, 11 Jul 1997 18:36:02 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-Id: <199707112235.RAA11449@smtp.gte.net>
From: "Steve Schonberger"
To:
Subject: Re: Game AI
Date: Fri, 11 Jul 1997 15:31:33 -0700
X-MSMail-Priority: Normal
X-Priority: 3
X-Mailer: Microsoft Internet Mail 4.70.1161
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Resent-Message-ID: <"JpIIqC.A.30E.8Trxz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/314
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 7858
Status: RO
I wrote:
> > At the Computer Game Developers' Convention, I attended a game AI
> > round-table (one of Steven Woodcock's sessions). An interesting topic
that
> > came up was whether it is a good idea for game AI to learn while
playing a
> > single human player. It seemed to me that there were more people
> > advocating non-learning AI models, but a few people did think learning
was
> > a good thing.
From: Ryan T Drake
Date: Friday, July 11, 1997 1:23 PM:
> I would say that adaptive AI would make a really good option in most
> cases, but it also would depend on the game. Something else that would
> need consideration: Does the game remember what it learned when you quit
> the game and come back in?
This is a point Steven Woodcock brought up. One answer is to have a "who
are you?" page as part of the startup, so that the computer doesn't
instantly kill player two because player one has worked up to a high skill
level. That's annoying, but fixes the problem. An "I'm the only one who
plays this game on this computer" option could be used to avoid asking the
problem unnecessarily.
This isn't really an AI issue, though, because any program that has
multiple levels (game or not) has to deal with multiple users unless
there's only one meaningful configuration. It's just as important for a
word processor to keep track of user one wanting technical manual mode and
user two wanting personal letter mode as it is for a game to know user one
is on level 99 and user two is on level 2.
So, new topic (please change the subject line if you run with it): What's
the best way to manage keeping track of user settings (preferences,
difficulty, etc.) on a computer shared by more than one user?
> My feeling is, you can tell what is good AI if the computer's actions
> look like the way a human would play. For instance: If I am playing
> Quake, and i see a monster trying to follow me around the corner and
> hitting a wall, i would not consider that good AI. On the other hand, if
> i had a monster after me that hits every shot and follows me perfectly, i
> would also not consider that good AI. Reason being, I know a human
cannot
> possibly be dumb enough to try to follow me through walls, and I also
know
> a human cannot possibly be good enough to hit each shot and follow
> perfectly. In essence, to have good artificial intellegence, you also
> need artificial stupidity.
Although this is very important, this is more a play balance issue than an
AI issue. How do you deal with this? Maybe by starting with a few enemies
and adding more at higher levels. Maybe by giving the enemies tougher
equipment or attributes. Maybe by giving them smarter AI.
> One idea of implementing this kind of AI is to start by giving the
> computer only as much information as a human would get. For example,
> sticking with the Quake idea. Only give the computer information about
> its surroundings, and not give it access to where everything is on the
> map. If a normal player would get sound cues give the computer ai sound
> cues.
I think this was another topic in the AI round-table. Do we make the AI
enemies play by exactly the same rules and information a player uses? (In
chess, of course, other games, maybe.) Or do we make up for weaknesses in
the enemies' AI by giving them "hints from God"?
In short, does computer-controlled player cheat or play fair?
> Another important thing to give any AI is reaction time. No human being
> has perfect 0ms reaction time. In a ``deathmatch'' for instance, a human
> being could take anywhere between 100-500ms to react to events on the
> screen. Add this factor in when calculating how the computer reacts to
> things. Reasonable AI can vary given a user-selected ``skill level'' but
> it should always be beatable.
Play balance.
> Now to your original point... Yes, I think ideally AI should be adaptive,
> so long as its adaptiveness tops off at a certain point. Anyone who
> considers themself very good at a certain game can relate to what I mean.
> Eventually the learning curve just stops and you are at a point that
> you can't really get much better...Usually this is when you can beat most
> other players and there is no one better to challenge you. This should
> also happen with AI. There should be a point where the AI stops getting
> better. As soon as it looks like the AI is doing a certain amount better
> than the human player, it should shut off and not learn anything new.
> Then as the human player starts getting better you can turn the learning
> on again.
Again, play balance.
What I was getting at in my note was not about how to achieve play balance.
I was talking about whether an AI enemy "learns". Consider these examples
for a computer chess player:
1 (non-learning). At level 1, the computer plays one of a few well-known
openings, and analyzes moves only a few levels deep, with a weak set of
pre-defined move-evaluation settings. At level 2, it expands its set of
openings, analyzes moves a little deeper, and uses a stronger set of
pre-defined move-evaluation settings. At the top level, it uses a huge
library of openings, analyzes moves as deeply as time allows, and uses the
strongest set of move-evaluation settings the developer has been able to
pre-calculate.
2 (learning). Initially, the computer plays with a complete library of
openings, but has settings that tell it to choose the most familiar
openings. It always analyzes moves as deeply as time allows, but starts
with a weak set of move-evaluation settings. Where its moves produce
favorable results, it adjusts its move-evaluation settings to do more of
the same, and where its moves produce unfavorable results, it adjusts its
settings to try something different.
Example 1 increases in difficulty, but doesn't "learn". It can play at a
user-selected difficulty level, or advance in level when it counts the
player winning a lot. But the way it increases difficulty is by
substituting a _pre-defined_ set of stronger move-evaluation settings.
Example 2, by contrast, also increases in difficulty, by "learning". If it
finds that the player usually wins when it opens P-K4, but it usually wins
when it opens P-Q4, it plays P-Q4 more, and vice-versa. If it usually wins
by trading lots of pieces, it tries to trade pieces more often. Basically,
it adapts to what works.
Each has good and bad points. Example 1 is likely to be easier to build,
which is a big plus. On the down-side, if its pre-defined settings have a
critical flaw that's hard enough to find that play-testers miss it, but
players find it, then any player that discovers (or hears about it) that
flaw can beat the computer with a shortcut. Example 2 can deal with that
sort of flaw by learning that it doesn't want to leave that sort of hole
open.
But example 2 has its weaknesses too, most notably being harder to program.
A more subtle one is that the human player can teach the computer to play
worse, by intentionally playing badly in response to the computer player's
mistakes. If the human chess player dropped pieces or resigned every time
the computer player left the queen unguarded, the computer player might
start to think that leaving the queen unguarded was a good strategy.
Another complication of the learning model is that it may be difficult to
measure what results are good and what aren't. Did the computer player win
game 1 and lose game 2 because it played smarter in game 1? Or was it
because the human player played smarter in game 2? What did it do
differently that made it play worse, and what made it play better?
So, what's good about using pre-calculated strategy settings? What's good
about measuring results, and ajusting the strategy settings during or
between games?
Steve Schonberger
From woodcock@real3d.com Fri Jul 11 19:13:24 1997
Return-Path:
Received: from stargazer.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id TAA00042; Fri, 11 Jul 1997 19:13:19 -0400
Received: by stargazer.real3d.com (4.1/1.34.a)
id AA02904; Fri, 11 Jul 97 19:13:19 EDT
From: woodcock@real3d.com
Message-Id: <9707112313.AA02904@stargazer.real3d.com>
Subject: Re: Game AI
To: nshaf@intur.net
Date: Fri, 11 Jul 1997 19:13:18 -0400 (EDT)
In-Reply-To: <33C6A704.104D@intur.net> from "Nick Shaffner" at Jul 11, 97 04:35:00 pm
X-Mailer: ELM [version 2.4 PL25]
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Content-Length: 2385
Status: O
> Agreed, in addition, one could design the AI with the primary purpose
> of making the game fun, rather than victory. This certainly be a more
> difficult task, but (for example) by assessing the users levels of
> interaction with the game, the AI could forcibly create incidents to
> break up 'slow' periods - or perhaps not attack the users when he's down
> or running low on resources, etc...
I like the idea of the AI somehow gauging the "pace" of the game,
provided it's a game that makes that a sensible thing to do. For
Dungeon Keeper, for example, it might be difficult to launch an assault
just to keep things moving, especially if the AI and the player haven't
had contact yet. Similarly, one can easily imagine a creature or
two in Quake leaping out to harass the player when he's been sitting in
one place too long.
Question though: Is that "cheating"? One can argue it is, or one
can argue it provides for better gameplay.
> It can also help make the game more user extensible,
> for example in Mission to Nexus Prime (plug, plug) - the user can
> actually design/create completely new types of units, and learning is
> essential in order to get the AI to the point where it can use them
> effectively...
Nick, I ask you the same thing I asked Dave in an earlier post.
I'd love to hear more info on this so I can add it to my Games AI
page. I think a lot of folks would be interested.
Steve
+=============================================================================+
| _ |
| Steven Woodcock _____C .._. |
| Senior Software Engineer, Gameware ____/ \___/ |
| Lockheed Martin Real3D <____/\_---\_\ "Ferretman" |
| Phone: 719-597-5413 |
| E-mail: woodcock@real3d.com |
| Web: http://www.cris.com/~swoodcoc/ai.html (Games AI page) |
| Disclaimer: My opinions in NO way reflect the opinions of |
| Lockheed Martin Real3D--get *serious* |
+=============================================================================+
From stevesch@csealumni.UNL.edu Fri Jul 11 19:29:22 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id TAA00266; Fri, 11 Jul 1997 19:29:20 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id TAA22195; Fri, 11 Jul 1997 19:29:18 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id TAA26723
for woodcock@real3d.com; Fri, 11 Jul 1997 19:28:25 -0400 (EDT)
Resent-Date: Fri, 11 Jul 1997 19:28:25 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-Id: <199707112328.SAA24642@smtp.gte.net>
From: "Steve Schonberger"
To:
Subject: Re: Game AI
Date: Fri, 11 Jul 1997 16:27:11 -0700
X-MSMail-Priority: Normal
X-Priority: 3
X-Mailer: Microsoft Internet Mail 4.70.1161
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Resent-Message-ID: <"WyejfB.A.RfG.mFsxz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/317
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 2826
Status: RO
> From: woodcock@real3d.com
> Date: Friday, July 11, 1997 3:48 PM
> One topic brought up at the AI roundtables was how many developers
> were being driven to put in multiplayer *instead of* good game AI.
> Though in the minority (thankfully), some folks reported that their
> management was far more interested in making sure a game was Internet
> capable than in making it play a challenging solitaire game. IMO,
> that's a poor vision of the future...as much as I enjoy online gaming
> I like to *learn* the game alone for a bit first. Fortunately as
> I said, that viewpoint didn't seem to be in the majority.
Since I'm working on a massively multi-user game, my AI opponents are there
just to train you so that you can handle the controls when you come upon
human opponents. There may be some AI enemies that are so nasty that you
can only hope to beat them by rounding up several human players and ganging
up on them, but for the most part I just want the AI to do interesting
stuff, not necessarily to make them extra-tough. A common question is,
"How do you make an AI player tough?" Well, I also want to ask, "How do
you give an AI player 'personality'?"
> > I guess I'd have to say that I've yet to see an AI that can't be beat
by
> > some strategy that the developers either didn't foresee or didn't have
> > the time to code against. Learning is a great way to help alleviate
> > that problem and thus create a better playing experience.
>
> Agreed. Adaptation is one of the things promised in the large
> online games (such as Ultima Online) and seems to me to be a natural
> next step for AI in games.
For a game like Ultima Online, adaptation doesn't have to be done by
"learning". It can be done by the game operators noticing that the
werewolves always kill human players lacking silver weapons, and always die
against silver-armed humans, and either pulling werewolves out of the game
or resetting their programming (AI, attributes, and where they show up).
In other words, adaptation can be accomplished by just changing something
that's broken on the server, so as soon as the operators see something
wrong, they can make the server "adapt". A stand-alone game doesn't have
that option, except by offering updates on the publisher's web site.
That suggests a contrast between "smart" AI and AI with "personality".
"Smart" werewolves would go into towns in human form and find another town
if anyone had silver (possibly testing that by sending in a "dumb"
werewolf), but declare dinner time if no one had silver. But that would
just massacre such towns, which isn't fun. A werewolf with "personality"
might seek out towns with lots of silver weapons, either for the challenge,
or seeking release from their curse. Which is more fun?
Steve Schonberger
From nshaf@intur.net Fri Jul 11 23:19:35 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id XAA04050; Fri, 11 Jul 1997 23:19:34 -0400
Received: from www.intur.net by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id XAA22614; Fri, 11 Jul 1997 23:19:19 -0400
Received: from grim by www.intur.net via ESMTP (940816.SGI.8.6.9/940406.SGI)
for id WAA06093; Fri, 11 Jul 1997 22:20:47 -0500
Message-Id: <199707120320.WAA06093@www.intur.net>
From: "Nick Shaffner"
To:
Subject: Re: Game AI
Date: Fri, 11 Jul 1997 22:18:10 -0500
X-MSMail-Priority: Normal
X-Priority: 3
X-Mailer: Microsoft Internet Mail 4.70.1161
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Length: 3479
Status: RO
I knew I knew you from somewhere, you've got that nifty 'You know your
game is in trouble when' with a couple of my quotes on it :-)
> I like the idea of the AI somehow gauging the "pace" of the game,
> provided it's a game that makes that a sensible thing to do. For
> Dungeon Keeper, for example, it might be difficult to launch an assault
> just to keep things moving, especially if the AI and the player haven't
> had contact yet. Similarly, one can easily imagine a creature or
> two in Quake leaping out to harass the player when he's been sitting in
> one place too long.
> Question though: Is that "cheating"? One can argue it is, or one
> can argue it provides for better gameplay.
I'll choose gameplay instead of "not cheating" if I think 90% of my
players aren't going to notice or care about the cheating (given it doesn't
actually hurt gameplay).
For example:
Letting the enemy units pathfind around the exact edge of the known human
units' visible range would be fine with me ( in order to enable some of my
eakiness' algo - which makes the game significantly more fun )
Letting the AI's have a much higher probability of allying with each other
whenever they notice human player's allying seems acceptable, as it would
keep the game more even/challenging
Giving the AI a cash boost when it's loosing, or allowing it to rebuild
buildings might allow the AI to last longer, but I think this sort of
visible cheating is what most annoys players and is quite detrimental to
gameplay...
> > It can also help make the game more user extensible,
> > for example in Mission to Nexus Prime (plug, plug) - the user can
> > actually design/create completely new types of units, and learning is
> > essential in order to get the AI to the point where it can use them
> > effectively...
>
> Nick, I ask you the same thing I asked Dave in an earlier post.
> I'd love to hear more info on this so I can add it to my Games AI
> page. I think a lot of folks would be interested.
Well, the unit utilization AI uses a couple of techniques, I use the
"boxes" techniques for determining which types units are most effective
against other types units (essentially genetic programming without the
genes :) ) - this way new units can be created and the AI can use them and
learn their effective value, as well as adapt to their altered value with
different players' 'styles' (concerning overall unit selection and usage)
The original value for each unit is computed from the unit's cost,
mobility, firepower, etc, - and originally initialized to the 'economically
optimal' state - ex. the computer will tend to build the units that give it
the most bang/mobility for the buck... It is incrementally modified as the
AI learns about the units effective average lifespan, offensive/defensive
%, % utilization, effected influence, effective influence and actual
influence, etc against each of the other types of units... This works out
suprisingly well (for such a simple algo), and adapts extremely quickly in
our game ( which has a high unit turnover ). It also allows for some quite
useful comparisons against 'factory norms' which allow the AI to make
guesses as to when it might be getting too hard for the player... It also
has the extremely visible side effect of allowing the AI to pull back into
a 'fuzzy' defensive position.
:) Nick
--
Nick Shaffner
Technical Director
DigiFX Interactive.
http://users.intur.net/~nshaf/
http://www.digifx.net/
From ollanes@accesspro.net Sat Jul 12 00:59:50 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id AAA05581; Sat, 12 Jul 1997 00:59:49 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id AAA22857; Sat, 12 Jul 1997 00:59:48 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id AAA25212
for woodcock@real3d.com; Sat, 12 Jul 1997 00:58:55 -0400 (EDT)
Resent-Date: Sat, 12 Jul 1997 00:58:55 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-Id: <199707120457.AAA24922@mail.digiweb.com>
Reply-To:
From: "Orlando Llanes"
To:
Subject: Re: Game AI
Date: Sat, 12 Jul 1997 00:49:00 -0400
X-MSMail-Priority: Normal
X-Priority: 3
X-Mailer: Microsoft Internet Mail 4.70.1155
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Resent-Message-ID: <"IY_SYB.A.wFG.T7wxz"@mail>
Resent-From: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/318
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 1415
Status: RO
I like the AI from X-Com. I'm not AI expert, but the X-Com AI looks to be a
bit complicated. For example, if you shoot at the alien, they move out of
the way after the shots are fired. If you were spotted by them, and then
you run back for cover, they will look for you. They leave their
fallen/landed spacecraft and seek buildings from which to shoot from high
above. Etc.
Another cool thing about X-Com is that the soldiers don't fire accurately
on the first mission, they gradually get more accurate.
I also like the AI from DOOM where if you're seen, the aliens will pursue
you and open doors if they can. One time, I heard a bulldemon (or whatever
they're called), I was going crazy trying to figure out where it was! As it
turns out, it poped up right in front of me while I was waiting for the
elevator. I shot at it from a split-second reaction.
The kind of AI I like is one where the enemy can surprise you, even if you
know their pattern of attack. The other thing that's cool is how the
character interacts with the environment. For Example, I was playing X-Com
today, and I positioned a couple of my soldiers behind some trees, on the
alien's turn, one of my soldiers spotted them and was shot at, but the
soldier was safe because the shot hit the log.
Sorry if I'm off-topic, or if I've mentioned something already said, but I
was just writing down some random thoughts :)
See ya!
Orlando Llanes
From kyhui@netvigator.com Sat Jul 12 04:36:53 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id EAA09176; Sat, 12 Jul 1997 04:36:52 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id EAA23230; Sat, 12 Jul 1997 04:36:50 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id EAA27309
for woodcock@real3d.com; Sat, 12 Jul 1997 04:35:30 -0400 (EDT)
Resent-Date: Sat, 12 Jul 1997 04:35:30 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-ID: <33C740C2.96FF813C@netvigator.com>
Date: Sat, 12 Jul 1997 16:30:58 +0800
From: Hui Ka Yu
Reply-To: kyhui@netvigator.com
X-Mailer: Mozilla 4.01 [en] (Win95; I)
MIME-Version: 1.0
To: gamedesign@mail.digiweb.com
Subject: Re: Game AI
X-Priority: 3 (Normal)
References: <199707111920.OAA20395@smtp.gte.net>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Resent-Message-ID: <"aU5Or.A.coG.HG0xz"@mail>
Resent-From: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/319
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 1612
Status: RO
The discussion just reminded me some of the "learning" ai attempts.
Remeber MK3 for PC? I do think that it adopts a learning ai based on a
scoring system for different moves. When a move is effective/defective,
the opponent will perform the move more/less accordingly. (somebody
mentioned it in the discussion)
Sounds good for a duel game, yet the result was disappointing, coz the
so-called "learning" is _so_ obvious to players.(can't describe in
words, but you will know if you've tried it) The computer players turn
out to be more _unnatural_ than non-learning opponents, let alone human
players, and playing with them was really IMHO no fun. It might beat the
player, but player won't find it FUN. A human player won't learn that
way.
So it seems that making a learning ai with this "good result->do more,
bad results->do less" sounds easier than it should be. To make it look
natural, it think you need to define vast amount of alternatives first
and perform complex adoptations so that players cannot recognize them
all and find it _so_ artificial and impractical. e.g. in a C&C type
game, when a player attack more often by sea, the computer adjust more
fund to sea defence. A player can cheat the ai simply by attack 10 times
(hoax) by sea and then crush the computer with land unit with ease
_every_ time he plays. That sounds really bad and no fun! A human player
will suspect that this is only part of a tactics and won't fall into
such trap - and this disobey the "good results->do more" ai system.
Just some comments....
Psycloid kyhui@netvigator.com
Afods - freelance game developer team
From condor@neotechonline.com Sat Jul 12 13:25:47 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id NAA17238; Sat, 12 Jul 1997 13:25:46 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id NAA23794; Sat, 12 Jul 1997 13:25:40 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id NAA12186
for woodcock@real3d.com; Sat, 12 Jul 1997 13:24:47 -0400 (EDT)
Resent-Date: Sat, 12 Jul 1997 13:24:47 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-Id: <199707121723.NAA11696@mail.digiweb.com>
X-Mailer: Microsoft Outlook Express 4.71.0544.0
From: "John Vanderbeck (NeoTECH)"
To:
Subject: Re: Game AI
Date: Sat, 12 Jul 1997 11:29:36 -0500
X-Priority: 3
X-MSMail-Priority: Normal
MIME-Version: 1.0
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Mimeole: Produced By Microsoft MimeOLE Engine V4.71.0544.0
Resent-Message-ID: <"g9-enB.A.a3C.O27xz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/322
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 4662
Status: RO
My comments are inline
Thanks,
John Vanderbeck
Lead Programmer - NeoTECH Online
condor@neotechonline.com
http://www.neotechonline.com
GAME DESIGN mailing list:
Email gamedesign-request@digiweb.com , SUBJECT subscribe
----
From: Ryan T Drake
To: gamedesign@mail.digiweb.com
Date: Friday, July 11, 1997 3:24 PM
Subject: Re: Game AI
>On Fri, 11 Jul 1997, Steve Schonberger wrote:
>
>> At the Computer Game Developers' Convention, I attended a game AI
>> round-table (one of Steven Woodcock's sessions). An interesting topic
that
>> came up was whether it is a good idea for game AI to learn while playing
a
>> single human player. It seemed to me that there were more people
>> advocating non-learning AI models, but a few people did think learning
was
>> a good thing.
>
>I would say that adaptive AI would make a really good option in most
>cases, but it also would depend on the game. Something else that would
>need consideration: Does the game remember what it learned when you quit
>the game and come back in?
>
>My feeling is, you can tell what is good AI if the computer's actions
>look like the way a human would play. For instance: If I am playing
>Quake, and i see a monster trying to follow me around the corner and
>hitting a wall, i would not consider that good AI. On the other hand, if
>i had a monster after me that hits every shot and follows me perfectly, i
>would also not consider that good AI. Reason being, I know a human
cannot
>possibly be dumb enough to try to follow me through walls, and I also
know
>a human cannot possibly be good enough to hit each shot and follow
>perfectly. In essence, to have good artificial intellegence, you also
>need artificial stupidity.
[John Vanderbeck]
I completely agree with this statement. It is important to realize that
creating an artificial intelligence doesn't necesarily mean creating a
super-smart computerized brain with the knowledge to conquer the world.
Maybe in the scope of gaming, Artificial Intelligence should be better
known as Artificial Humanity. Our goal is to create a realistic HUMAN
opponent that is controlled by the computer but otherwise not noticable as
the computer. Our goal is _not_ to create a COMPUTER player. IMHO.
>
>One idea of implementing this kind of AI is to start by giving the
>computer only as much information as a human would get. For example,
>sticking with the Quake idea. Only give the computer information about
>its surroundings, and not give it access to where everything is on the
>map. If a normal player would get sound cues give the computer ai sound
>cues.
[John Vanderbeck]
This is very interesting, and comething I have always wondered about as I
played games. Sometimes it seems very obvious that the computer is making
decisions based on information it should _not_ know. I think this has a
tendancy to irrate the player.
>
>Another important thing to give any AI is reaction time. No human being
>has perfect 0ms reaction time. In a ``deathmatch'' for instance, a human
>being could take anywhere between 100-500ms to react to events on the
>screen. Add this factor in when calculating how the computer reacts to
>things. Reasonable AI can vary given a user-selected ``skill level'' but
>it should always be beatable.
[John Vanderbeck]
I don't have alot of experience in AI or AH as I stated above, but wouldn't
this point be rather mute if your AH routine was complex enough to
reasonably simulate humanity? I would think that just running through a
complex routine would create the required delay. I don't see how you could
run the AH routine in 0ms :)
>
>Now to your original point... Yes, I think ideally AI should be adaptive,
>so long as its adaptiveness tops off at a certain point. Anyone who
>considers themself very good at a certain game can relate to what I mean.
>Eventually the learning curve just stops and you are at a point that
>you can't really get much better...Usually this is when you can beat most
>other players and there is no one better to challenge you. This should
>also happen with AI. There should be a point where the AI stops getting
>better. As soon as it looks like the AI is doing a certain amount better
>than the human player, it should shut off and not learn anything new.
>Then as the human player starts getting better you can turn the learning
>on again.
[John Vanderbeck[
Essentially correct. I beleive the AH should always be just a few steps
ahead of the player, to give them a growing challenge. Just like
weight-lifting.
>
>
>
>= Ryan Drake =
>= drake@cse.psu.edu =
>= http://www.cse.psu.edu/~drake =
>
From condor@neotechonline.com Sat Jul 12 13:25:48 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id NAA17242; Sat, 12 Jul 1997 13:25:47 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id NAA23798; Sat, 12 Jul 1997 13:25:46 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id NAA12197
for woodcock@real3d.com; Sat, 12 Jul 1997 13:24:52 -0400 (EDT)
Resent-Date: Sat, 12 Jul 1997 13:24:52 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-Id: <199707121723.NAA11689@mail.digiweb.com>
X-Mailer: Microsoft Outlook Express 4.71.0544.0
From: "John Vanderbeck (NeoTECH)"
To:
Subject: Re: Game AI
Date: Sat, 12 Jul 1997 11:31:07 -0500
X-Priority: 3
X-MSMail-Priority: Normal
MIME-Version: 1.0
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Mimeole: Produced By Microsoft MimeOLE Engine V4.71.0544.0
Resent-Message-ID: <"OcusQC.A.C3C.J27xz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/321
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 5174
Status: RO
These "Genetic Algorithims" sound very interesting. Where can I find out
more about them?
Thanks,
John Vanderbeck
Lead Programmer - NeoTECH Online
condor@neotechonline.com
http://www.neotechonline.com
GAME DESIGN mailing list:
Email gamedesign-request@digiweb.com , SUBJECT subscribe
----
From: woodcock@real3d.com
To: gamedesign@mail.digiweb.com
Date: Friday, July 11, 1997 6:06 PM
Subject: Re: Game AI
>> (Comments inline. Damn MS-mail screwed up reply format...)
>
> I hate when that happens. That's why I only use Unix Elm (tm).
>Command line is where it's at! ;)
>
>> > On Fri, 11 Jul 1997, Steve Schonberger wrote:
>> >
>> > Does the game remember what it learned when you quit
>> > the game and come back in?
>> [Paul Nash]
>> Most definitely it should remember that. The problem of
>> training on player B and wiping out player A is not a problem at all,
>> either. Don't a lot of games have saved game information? Is it so
>> hard for the AI to save it's brain to disk? Not in a properly designed
>> AI, I say. For instance, genetic algortihms should allow the saving of
>> the genes to prescribe a persitent state, I would think. So save that
>> for each player or each saved game.
>
> In fact GAs were one of the methods of building a learning AI
>discussed in the sessions, mostly with regards to the Creatures game
>and its A-Life techniques.
>
> One side aspect of this that somebody brought up (it might have
>been me, but I'm not sure) is that by having the AI "brain" loaded
>rather than coded you can release add-on expansion packs for the game
>containing additional AIs. Better yet (separate topic), if the AI
>code is accessible to the player (ala Quake-C), then you can sponsor
>contests amongst players to see who can develop the most devious
>AIs. The best 10 get released on a CD, or posted to a web site.
>
>> >
>> > (Ryan says games should throttle back to match the players' level)
>> >
>> [Paul Nash]
>> Sorry Ryan, but I 100% disagree. You've just shot down your own
>> argument here, because you say that AI should level off to match the
>> player, but you say that the player levels off because there's noone
>> left to challenge them. BUT, the AI IS the challenger. If the AI is
>> always challenging a person, then they won't necessarily level off,
>> right? I don't consider this to be like weightlifting, because it's
the
>> human brain, which is an entirely different kind of muscle. :)
>
> I don't know about that, Paul. I can see value in adjusting the AI
>so that it's always a bit tougher than the player is. I remember the
>first few times I played C&C I was sweating bullets over the AI, but
>once my expertise got high enough he ceased to become a threat in
>all but the most unbalanced scenarios.
>
> The trick is capturing parameters which you can use to accurately
>judge how experienced a player is.
>
>> That said, the AI should definitely *track* the player, and not
>> necessarily always be better than the human. If the AI sense the human
>> is thoroughly getting his butt kicked regularly, the AI is too strong
>> and it should back off a little to let the human "catch up."
>
> Er....isn't that what Ryan said?
>
>> If you
>> could design an AI that can detect specific gameplay deficiencies in
the
>> human and somehow adapt to them, that would be very cool.
>
> Genetic algorithsm have a possibility here, provided the game itself
>is "big enough" to permit them time to evolve (they can be notoriously
>slow). Consider a space strategy game ala MOO, in which individual
>ship designs are modified by the AI over time based on those ships
>which do well against the player. If you the player tend to build
>lots of fighters and carriers, then gradually over time the AI will
>adapt to that by building ships based on surviving ship types
>(those that have more anti-fighter defenses).
>
>> Of course, I am not suggesting that any of this is really easy,
>> but all of it has potential.
>
> That's why they pay us the big bucks! (Well, it's rumored some folks
>get big bucks...I wouldn't know..... ;).
>
>
>
>Steve
>
From mark_a@cix.compulink.co.uk Sat Jul 12 23:05:02 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id XAA24382; Sat, 12 Jul 1997 23:05:00 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id XAA24486; Sat, 12 Jul 1997 23:04:59 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id XAA20001
for woodcock@real3d.com; Sat, 12 Jul 1997 23:03:53 -0400 (EDT)
Resent-Date: Sat, 12 Jul 1997 23:03:53 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Date: Sun, 13 Jul 97 04:02 BST-1
From: mark_a@cix.compulink.co.uk (Mark Atkinson)
Subject: Re: Game AI
To: gamedesign@mail.digiweb.com
Cc: mark_a@cix.compulink.co.uk
Reply-To: mark_a@cix.compulink.co.uk
Message-Id:
Resent-Message-ID: <"JYsV1.A.f0E.OVEyz"@mail>
Resent-From: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/324
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 1006
Status: RO
In-Reply-To: <199707121723.NAA11689@mail.digiweb.com>
> These "Genetic Algorithims" sound very interesting. Where can I find
> out
> more about them?
A web search should turn up plenty of info and sample code.
Alternatively, the canonical introductory text is "Genetic Algortihms in
Search, Optimization & Machine Learning", David E Goldberg,
Addison-Wesley 1989, ISBN 0-201-15767-5. Note he uses 'Roulette Wheel'
selection, which is IMHO a bad thing; most modern implementations use
ranking of some sort. There's also comp.ai.genetic.
I'd echo the sentiments about trying to create a fun-to-beat opponent,
rather than a Computer Brain(TM). This is the primary reason to avoid
rule-based systems IMHO.
-=Mark=-
Mark Atkinson Voice: +44 171 828 6990
Technical Director Fax: +44 171 828 6997
Computer Artworks Ltd. Mail: mark@artworks.co.uk
London, UK. Web: http://www.artworks.co.uk
From rick@polylang.com Mon Jul 14 05:12:48 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id FAA15126; Mon, 14 Jul 1997 05:12:48 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id FAA27041; Mon, 14 Jul 1997 05:12:11 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id FAA29059
for woodcock@real3d.com; Mon, 14 Jul 1997 05:11:15 -0400 (EDT)
Resent-Date: Mon, 14 Jul 1997 05:11:15 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-Id: <3.0.32.19970724095734.00683d88@MAILHOST>
X-Sender: rick@MAILHOST
X-Mailer: Windows Eudora Pro Version 3.0 (32)
Date: Thu, 24 Jul 1997 10:05:05 +0100
To: gamedesign@mail.digiweb.com
From: rick cronan
Subject: Re: Game AI
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Resent-Message-ID: <"0z0vPC.A.eAH.bzeyz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/326
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 2084
Status: RO
At 17:48 11/07/97 -0400, drake@cse.psu.edu wrote:
>
>There are certain things that make a Quake player (or any other game
>player) ``good.'' I will break it down into 3 categories for now: skill,
>tactics, and strategy.
I think there's a fourth and extremely relevant category (which was kind of
touched on last week or the week before on this list) which is that of
knowledge. In my experience, the Quake death-match winners are not
necessarily the most skilled, or those with the best tactics and
strategies, but those who know the levels best, those who know where to get
the 200 armour, the Quad-damage, etc.
Those kind of players (in single-player mode) are the one your learning AI
would be targetted at, because for the game to challenge them, it would
need to combat their knowledge that around corner X there are three
monsters of type Y, two health packs and some rockets.
For someone playing a level for the first time, the monsters don't
necessarily have to act in such a way as to defeat your style. You'll get
fragged because you don't know what's around the next corner, not because
the monsters 'know' your tactics.
Which also brings me on to my other point. The learn / not learn dilemma
should also be looked at in terms of what the game is, especially if the
player can discern its presence. From an internal logic point of view, a
C&C type game can have a learning AI because it represents a long,
drawn-out war in which an enemy could learn your tactics, and come up with
counters. However, in your Quake type game it is less plausible that the
enemies would have time to learn from your actions.
The only exception to this IMHO is what has already been mentioned, and
that is where learning AI is used to promote entertainment, rather than
simply to help the computer player win.
| rick cronan | email: rick@polylang.com |
| production manager | phone: +44 (0) 114 267 0017 |
| cool beans productions ltd | fax: +44 (0) 114 268 7487 |
| url: http://www.polylang.com/polylang2/Coolbeans/home.htm |
From Swoodcoc@concentric.net Mon Jul 14 10:57:51 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id KAA24266; Mon, 14 Jul 1997 10:57:50 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id KAA00186; Mon, 14 Jul 1997 10:57:14 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id KAA17019
for woodcock@real3d.com; Mon, 14 Jul 1997 10:56:15 -0400 (EDT)
Resent-Date: Mon, 14 Jul 1997 10:56:15 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
From: Swoodcoc@concentric.net
Message-Id: <199707141455.KAA12971@galileo.cris.com>
Subject: Re: Game AI
To: gamedesign@mail.digiweb.com
Date: Mon, 14 Jul 1997 10:55:06 -0400 (EDT)
In-Reply-To: from "Ryan T Drake" at Jul 11, 97 05:48:53 pm
X-Mailer: ELM [version 2.4 PL25]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Resent-Message-ID: <"I2TKmD.A.pFE.E3jyz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/327
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 5454
Status: RO
Apologies to all ahead of time for the length of this post....Ryan had
some good points that I'm loath to trim out.
> On Fri, 11 Jul 1997 woodcock@real3d.com wrote:
> > I'm curious how you might spot that the human player has "leveled out"?
> > Taking Quake as an example, what criteria might you use? Kills vs.
> > shots? Average time playing a level? Some combination of the two might
> > suffice....that's an interesting problem.
>
> There are certain things that make a Quake player (or any other game
> player) ``good.'' I will break it down into 3 categories for now: skill,
> tactics, and strategy. Although they may sonund the same they have very
> different meanings when you look at them from an AI perspective...
>
> SKILL
>
> I consider ``skill'' to be mastery of a game's physical interaction. For
> instance reaction time would be a skill. The ability to perform complex
> maneuvers with the joystick or keyboard would also be a skill. Think
> about Command and Conquer--the ability to keep two different sets of tanks
> driving in two different directions (with keyboard shortcuts or by
> flipping back and forth between the two groups) would also be a skill.
> This is the easy part of programming AI. We start with the assertion that
> a computer has 100% skill. Normally there is no reaction time and the
> computer has as many fingers as it needs to control itself and its
> character. By adding code to modify the computer's reaction time, or
> limiting the number of things it can keep track of, you are lowering the
> computer's SKILL to a human's level.
Okay, this is simple enough. Most games today, I'd wager, manipulate
these values to achieve "tougher" (note I didn't say better) gameplay.
I would pick a nit and say that boiling skill down to nothing but reaction
times is a bit of a simplification, but I understand what you're driving
at here.
> TACTICS
>
> Tactics describe a player's specific reactions to the game environment and
> to his or her opponents. In Command and Conquer, if my opponent starts
> driving 20 tanks in the direction of my base, I will react by erecting a
> few turrets and sending a force of my own out. Tactics vary considerably
> from player to player. They can be thought of as a person's style of
> gameplay. In Quake if I make a habit of going for health boxes whenever I
> drop below 30%, that would be a tactic. Tactics are also rather trivial
> to program, and a rudimentary set of tactics can be programmed with a
> bunch of if (player does this) then (react with this action)
This is a bit shakier as tactics vary so widely from game to game.
One can easily come up with tactics for combined arms in a game like
C&C, for example, but I'm not so sure about a game like Quake. What
are the tactics there....hide when you're being shot at (though in many
3D POV games that *would* be a great improvement)? The very rapidly
changing environment that make up 3D POV games can make this partiuclar
element difficult to pin down.
Tactics are also something the AI could learn from the player, which
is something I'd like to see it try to do. If the AI sees that 2 flamethrowers
and 2 hovertanks seems to work well as an assalt force for the player, then
maybe it ought to consider adding that same formation to its own book of
tricks. (Of course, that leads to other problems...recognizing formations
(neural networks?), building formations, etc.)
> STRATEGY
>
> Strategy is a little harder to pin down. I would say this is the overall
> flow of your game. Another way of putting it is strategy is "A player's
> guess at what set of tactics will work to defeat the opponent" How am I
> going to win the game? Some are conservative and usually work fairly
> well, some are risky, but can really hammer an unsuspecting opponent.
>
> (Deep Blue analogy deleted)
>
> After your AI decides on a certain strategy, then you have to be able to
> implement it. This would be done by deploying a number of tactics that
> make up this strategy, but which ones? and when?
Truly the toughest part (IMO). Third Reich has been roundly criticized
as having either no grand strategy or simply playing along textbook strategy
lines. The first leads to an essentially defensive position while the
second can get predictable quick.
> A truly successful AI would be able to evaluate its opponent's tactics,
> come up with an overall strategy, and translate that strategy into tactics
> of its own. A game that can do all this would be incredible.
'Tis true.
Steve
+=============================================================================+
| _ |
| Steven Woodcock _____C .._. |
| Hired Gun, Gameware & AI ____/ \___/ |
| Wyrd Wyrks Consulting <____/\_---\_\ "Ferretman" |
| Phone: 719-392-4746 |
| E-mail: swoodcoc@concentric.net |
| Web: http://www.concentric.net/~swoodcoc/ai.html (Dedicated to Game AI) |
| Disclaimer: Yeah, I work for Lockheed-Martin Real3D....you think |
| anybody there ever listens to *my* opinion? Get *serious*. |
+=============================================================================+
From Swoodcoc@concentric.net Mon Jul 14 11:02:27 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id LAA24554; Mon, 14 Jul 1997 11:02:26 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id LAA00208; Mon, 14 Jul 1997 11:02:23 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id LAA18250
for woodcock@real3d.com; Mon, 14 Jul 1997 11:01:25 -0400 (EDT)
Resent-Date: Mon, 14 Jul 1997 11:01:25 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
From: Swoodcoc@concentric.net
Message-Id: <199707141500.LAA13269@galileo.cris.com>
Subject: Re: Game AI
To: gamedesign@mail.digiweb.com
Date: Mon, 14 Jul 1997 11:00:05 -0400 (EDT)
In-Reply-To: <199707112235.RAA11449@smtp.gte.net> from "Steve Schonberger" at Jul 11, 97 03:31:33 pm
X-Mailer: ELM [version 2.4 PL25]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Resent-Message-ID: <"gYTL7D.A.uWE.W7jyz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/328
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 2632
Status: RO
> This is a point Steven Woodcock brought up. One answer is to have a "who
> are you?" page as part of the startup, so that the computer doesn't
> instantly kill player two because player one has worked up to a high skill
> level. That's annoying, but fixes the problem. An "I'm the only one who
> plays this game on this computer" option could be used to avoid asking the
> problem unnecessarily.
I'm not sure we ever had a good solution offered either, beyond the
two you mention above. I'm not entirely convinced that an AI that
has been "trained" to be my opponent would, necessarily, make a good
opponent for somebody else....play styles influence that quite a bit.
On the other hand, when we're talking about AI modifications along the
lines of decreasing AI reaction times that maakes perfect sense....if I've
played 100 games of Doom and you've played 4, I *ought* to know my way
around that keyboard better.
> So, new topic (please change the subject line if you run with it): What's
> the best way to manage keeping track of user settings (preferences,
> difficulty, etc.) on a computer shared by more than one user?
I like the simple data file tagged by user name.
> I think this was another topic in the AI round-table. Do we make the AI
> enemies play by exactly the same rules and information a player uses? (In
> chess, of course, other games, maybe.) Or do we make up for weaknesses in
> the enemies' AI by giving them "hints from God"?
>
> In short, does computer-controlled player cheat or play fair?
I've seen a trend to try to make the AIs play exactly the same way,
with the same rules, as the player. I know that Enemy Nations does this,
and most of the upcoming RTS games are promising the same.
Steve
+=============================================================================+
| _ |
| Steven Woodcock _____C .._. |
| Hired Gun, Gameware & AI ____/ \___/ |
| Wyrd Wyrks Consulting <____/\_---\_\ "Ferretman" |
| Phone: 719-392-4746 |
| E-mail: swoodcoc@concentric.net |
| Web: http://www.concentric.net/~swoodcoc/ai.html (Dedicated to Game AI) |
| Disclaimer: Yeah, I work for Lockheed-Martin Real3D....you think |
| anybody there ever listens to *my* opinion? Get *serious*. |
+=============================================================================+
From t-pauln@microsoft.com Mon Jul 14 14:46:25 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id OAA03879; Mon, 14 Jul 1997 14:46:24 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id OAA01587; Mon, 14 Jul 1997 14:45:48 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id OAA19034
for woodcock@real3d.com; Mon, 14 Jul 1997 14:43:39 -0400 (EDT)
Resent-Date: Mon, 14 Jul 1997 14:43:39 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-ID: <4152F7B641AFCF11A49800805F680B3F3F094D@RED-36-MSG.dns.microsoft.com>
From: Paul Nash
To: "'gamedesign@mail.digiweb.com'"
Subject: RE: Game AI
Date: Mon, 14 Jul 1997 11:42:42 -0700
X-Priority: 3
X-Mailer: Internet Mail Service (5.0.1458.49)
Resent-Message-ID: <"ddGQN.A.ZjE.7Lnyz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/329
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 4858
Status: RO
More inline...
-Paul R. Nash, Multimedia Developer At Large
Microsoft Multimedia Dev. Intern http://www.uiuc.edu/ph/www/pr-nash/
(As usual, I'm not speaking for Microsoft -- these ideas and thoughts
are mine alone.)
> -----Original Message-----
> From: woodcock@real3d.com [SMTP:woodcock@real3d.com]
> Sent: Friday, July 11, 1997 4:05 PM
> To: gamedesign@mail.digiweb.com
> Subject: Re: Game AI
>
> > > On Fri, 11 Jul 1997, Steve Schonberger wrote:
> > >
> > hard for the AI to save it's brain to disk? Not in a properly
> designed
> > AI, I say. For instance, genetic algortihms should allow the saving
> of
> > the genes to prescribe a persitent state, I would think. So save
> that
> > for each player or each saved game.
>
> In fact GAs were one of the methods of building a learning AI
> discussed in the sessions, mostly with regards to the Creatures game
> and its A-Life techniques.
>
> One side aspect of this that somebody brought up (it might have
> been me, but I'm not sure) is that by having the AI "brain" loaded
> rather than coded you can release add-on expansion packs for the game
> containing additional AIs. Better yet (separate topic), if the AI
> code is accessible to the player (ala Quake-C), then you can sponsor
> contests amongst players to see who can develop the most devious
> AIs. The best 10 get released on a CD, or posted to a web site.
>
[Paul Nash]
Those are some very cool ideas indeed -- we have a contest here
(UIUC) sponsored by ACM every year where they get teams of people
together and give them shell code for a mech type game. The teams then
have like 24 hours of lab time to plugin the best AI they can come up
with, and then the mechs compete to the death in a tournament. It's a
cool idea, though I think it'd be better if the AI's were developed over
longer periods (though cheating is then a big factor).
> > >
> > > (Ryan says games should throttle back to match the players' level)
> > >
> > right? I don't consider this to be like weightlifting, because it's
> the
> > human brain, which is an entirely different kind of muscle. :)
>
> I don't know about that, Paul. I can see value in adjusting the AI
> so that it's always a bit tougher than the player is. I remember the
> first few times I played C&C I was sweating bullets over the AI, but
> once my expertise got high enough he ceased to become a threat in
> all but the most unbalanced scenarios.
>
> The trick is capturing parameters which you can use to accurately
> judge how experienced a player is.
>
[Paul Nash]
Agreed, the parameters are important. However, what I was
disagreeing with specifically was that Ryan seemed to be suggesting that
a player will always level out at some maximum potential, and I'm not
sure I believe that. However, either way an adjusting AI should be able
to compensate for that. (Perhaps change its tactics to force the player
to do something different -- maybe the AI has multiple game strategies
that it can go between???)
> > That said, the AI should definitely *track* the player, and not
>
> Er....isn't that what Ryan said?
>
[Paul Nash]
Yeah, more or less. :) Like I said, that's not necessarily what
I was objecting to, rather the concept of designing for a finite player
capabilities limit.
> > If you
> > could design an AI that can detect specific gameplay deficiencies in
> the
> > human and somehow adapt to them, that would be very cool.
>
> Genetic algorithsm have a possibility here, provided the game itself
> is "big enough" to permit them time to evolve (they can be notoriously
> slow). Consider a space strategy game ala MOO, in which individual
> ship designs are modified by the AI over time based on those ships
> which do well against the player. If you the player tend to build
> lots of fighters and carriers, then gradually over time the AI will
> adapt to that by building ships based on surviving ship types
> (those that have more anti-fighter defenses).
>
[Paul Nash]
I'd be interested to know if any of those ideas are embodied in
AOE type games insofar as "building" people. What if people got smarter
as their civilization advanced, or learned from previous battles. I
guess Close Combat has some complex soldier AI, but I'd like to see it a
little more subtle -- sort of a developing collective consciousness or
something instead of "Joe Sixpack is scared and he has three rounds
left." That tends to promote micro-management because you know exactly
what everyone is doing and feeling and thus feel obligated to correct it
all.
> > Of course, I am not suggesting that any of this is really easy,
> > but all of it has potential.
>
> That's why they pay us the big bucks! (Well, it's rumored some
> folks
> get big bucks...I wouldn't know..... ;).
>
[Paul Nash]
Hehe. Something like that. :-)
From DPottinger@Ensemble-Studios.com Mon Jul 14 21:12:06 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id VAA20377; Mon, 14 Jul 1997 21:12:05 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id VAA02834; Mon, 14 Jul 1997 21:11:29 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id VAA05482
for woodcock@real3d.com; Mon, 14 Jul 1997 21:10:33 -0400 (EDT)
Resent-Date: Mon, 14 Jul 1997 21:10:33 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-ID: <802B50C269DECF11B6A200A0242979EF33CFBA@consulting.ensemble.net>
From: David Pottinger
To: gamedesign@mail.digiweb.com
Subject: RE: Game AI
Date: Mon, 14 Jul 1997 20:12:04 -0500
X-Priority: 3
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.0.1457.3)
Content-Type: text/plain
Resent-Message-ID: <"oZHN1D.A.uOB.e2syz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/330
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 2508
Status: RO
> -----Original Message-----
> From: Nick Shaffner [SMTP:nshaf@intur.net]
> Sent: Friday, July 11, 1997 4:35 PM
> To: gamedesign@mail.digiweb.com
> Subject: Re: Game AI
>
> Hey David, remember me? :)
> [] Yup! We're still looking for good people like yourself:)
>
> > You do have to consider the game (as Ryan Drake already mentioned).
> No
> > AI should ever make the game unfun for people to play. Does that
> mean
> > that you shouldn't do learning AI then? No way! There is a lot of
> room
> > for learning AIs to be applied in ways that still make games fun.
>
> Agreed, in addition, one could design the AI with the primary
> purpose
> of making the game fun, rather than victory. This certainly be a more
> difficult task, but (for example) by assessing the users levels of
> interaction with the game, the AI could forcibly create incidents to
> break up 'slow' periods - or perhaps not attack the users when he's
> down
> or running low on resources, etc...
> [] 100% agreement. I'd guess (maybe hope is a better word?:) that
> it's not too long (within the next two years) before we see game AIs
> coming out that can regularly begin to whoop up on everyone w/o
> cheating. At that point, I think the next phase will be to make the
> AIs more full-featured (as opposed to just making them win) like
> you're talking here...
>
> A thought to ponder: How would you determine what a slow period is?
> Is it just the simple calculation of how much interaction the human
> has had with the AI? What about using "cheating" to determine where
> the human is? Is that okay?
>
> > the time to code against. Learning is a great way to help alleviate
> > that problem and thus create a better playing experience.
>
> Agreed, so long as it is used properly - it could possibly be
> used to
> help plug unforseen gaps in game mechanics (perhaps like the sandbag
> thing in C&C) ... It can also help make the game more user extensible,
> for example in Mission to Nexus Prime (plug, plug) - the user can
> actually design/create completely new types of units, and learning is
> essential in order to get the AI to the point where it can use them
> effectively...
> [] Yup, since the trend is to make more and more pieces of games
> open-ended, I think it will place more of an emphasis on the game
> being able to handle unforeseen things.
>
> Nick Shaffner
> http://www.digifx.net/
>
>
>
> dave
>
> Dave C. Pottinger
> Engine Lead and AI Guy
> Ensemble Studios, Inc.
>
>
From DPottinger@Ensemble-Studios.com Mon Jul 14 21:20:23 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id VAA20554; Mon, 14 Jul 1997 21:20:22 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id VAA02865; Mon, 14 Jul 1997 21:20:21 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id VAA07758
for woodcock@real3d.com; Mon, 14 Jul 1997 21:19:25 -0400 (EDT)
Resent-Date: Mon, 14 Jul 1997 21:19:25 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-ID: <802B50C269DECF11B6A200A0242979EF33CFBB@consulting.ensemble.net>
From: David Pottinger
To: "'gamedesign@mail.digiweb.com'"
Subject: RE: Game AI
Date: Mon, 14 Jul 1997 20:21:39 -0500
X-Priority: 3
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.0.1457.3)
Content-Type: text/plain
Resent-Message-ID: <"PwsMLD.A.s0B.c_syz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/331
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 2119
Status: RO
> -----Original Message-----
> From: woodcock@real3d.com [SMTP:woodcock@real3d.com]
> Sent: Friday, July 11, 1997 5:49 PM
> To: gamedesign@mail.digiweb.com
> Subject: Re: Game AI
>
> [] [snip]
>
> More on subject, I'm surprised that you actually found strategies to
> be similar enough from game to game to make saving such information
> useful. If I replay a given scenario (and I'll admit it has to
> be pretty compelling for me to do so) I usually try something
> different
> than I did the time before.
> [] The scenario replay learning feature was actually created out of a
> desire to have people who didn't win the first time get a different
> play experience when they replayed. The goal here was to remove the
> need to just optimize your strategy well enough so that you can
> eventually beat the scenario with the same thing you tried to do the
> last five times. If the AI does something markedly different (yet
> still intelligent, etc.) each time you play, then you get a more
> enjoyable experience, I think. It does help out replaying scenarios
> that you've already beaten, too (that just wasn't the genesis of the
> idea).
>
> Your solution of saving general playing tendencies (which I presume
> are things like types of units the player likes to build, battle
> formations they prefer, etc.) for randomly generated games seems very
> clever. The AI in C&C, for example (just to pick on the game we
> picked on in the roundtable discussions) seems to be especially tuned
> for the "canned" scenarios and flails somewhat when presented with
> a new user-designed map.
> [] It's primarily which types of units they like to build along with
> a few other things. AOE is very much a rock-paper-scissors game
> (infantry slaughter archers, but cavalary rocks infantry, etc.), so
> concentrating on the contextual unit prefs of players is what provided
> the most useful info and conveniently takes up very little memory:).
> We did try a lot of other things, though:).
>
> [] [snip]
>
> Steve
>
> dave
>
> Dave C. Pottinger
> Engine Lead and AI Guy
> Ensemble Studios, Inc.
>
>
From stevesch@csealumni.UNL.edu Tue Jul 15 00:29:24 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id AAA24281; Tue, 15 Jul 1997 00:29:23 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id AAA03298; Tue, 15 Jul 1997 00:28:46 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id AAA25823
for woodcock@real3d.com; Tue, 15 Jul 1997 00:27:46 -0400 (EDT)
Resent-Date: Tue, 15 Jul 1997 00:27:46 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-Id: <199707150427.XAA21914@smtp.gte.net>
From: "Steve Schonberger"
To:
Subject: Re: Game AI
Date: Mon, 14 Jul 1997 21:23:23 -0700
X-MSMail-Priority: Normal
X-Priority: 3
X-Mailer: Microsoft Internet Mail 4.70.1161
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Resent-Message-ID: <"GwJ3gB.A.pMG.rvvyz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/332
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 5891
Status: RO
> From: Paul Nash
> Date: Monday, July 14, 1997 11:42 AM
[...]
> > One side aspect of this that somebody brought up (it might have
> > been me, but I'm not sure) is that by having the AI "brain" loaded
> > rather than coded you can release add-on expansion packs for the game
> > containing additional AIs. Better yet (separate topic), if the AI
> > code is accessible to the player (ala Quake-C), then you can sponsor
> > contests amongst players to see who can develop the most devious
> > AIs. The best 10 get released on a CD, or posted to a web site.
> >
> [Paul Nash]
> Those are some very cool ideas indeed -- we have a contest here
> (UIUC) sponsored by ACM every year where they get teams of people
> together and give them shell code for a mech type game. The teams then
> have like 24 hours of lab time to plugin the best AI they can come up
> with, and then the mechs compete to the death in a tournament. It's a
> cool idea, though I think it'd be better if the AI's were developed over
> longer periods (though cheating is then a big factor).
That reminds me of the old Apple 2 game "Robot Wars". Each player selected
rules for the robots to fight by, then sent them into the battle arena,
where the computer controlled them all, according to the player-defined
rules. The game also came with a bunch of sets of pre-written rules, so
that solo players had someone to play against. Cool stuff.
In modern usage, it would cool for a multi-user game to treat "bots" as an
approved part of the game, rather than as a cheat. Come up with a good
"bot", and maybe the publisher will add it to the game as a monster and add
the designer to the credits. Of course, it takes a different kind of game
design, if human players are to compete against human player with "bot"
helpers.
[...]
> > The trick is capturing parameters which you can use to accurately
> > judge how experienced a player is.
> >
> [Paul Nash]
> Agreed, the parameters are important. However, what I was
> disagreeing with specifically was that Ryan seemed to be suggesting that
> a player will always level out at some maximum potential, and I'm not
> sure I believe that. However, either way an adjusting AI should be able
> to compensate for that. (Perhaps change its tactics to force the player
> to do something different -- maybe the AI has multiple game strategies
> that it can go between???)
> > > That said, the AI should definitely *track* the player, and not
> >
> > Er....isn't that what Ryan said?
> >
> [Paul Nash]
> Yeah, more or less. :) Like I said, that's not necessarily what
> I was objecting to, rather the concept of designing for a finite player
> capabilities limit.
Aside from adventure and puzzle games, you shouln't be able to "solve" a
game, in the sense of coming to a point of not being able to improve any
further. Sure, there comes a point where your reaction time doesn't get
any better, and your knowledge of the levels doesn't meaningfully improve,
but a game should have depth where even an expert can learn new tricks.
That should be the case in any game that doesn't have a real beginning and
end, or at least the space to keep advancing should be large enough that
hardcore players are still amused by the time the sequal is done.
> > > If you could design an AI that can detect specific
> > > gameplay deficiencies in the human and somehow adapt to
> > > them, that would be very cool.
I'm not convinced that would be so cool, at least not from the "fun" point
of view. It's obviously very cool from the technology point of view!
[...]
> [Paul Nash]
> I'd be interested to know if any of those ideas are embodied in
> AOE type games insofar as "building" people. What if people got smarter
> as their civilization advanced, or learned from previous battles. I
> guess Close Combat has some complex soldier AI, but I'd like to see it a
> little more subtle -- sort of a developing collective consciousness or
> something instead of "Joe Sixpack is scared and he has three rounds
> left." That tends to promote micro-management because you know exactly
> what everyone is doing and feeling and thus feel obligated to correct it
> all.
I still think that the pathological cases of learning forbid using learning
in a game, aside from tuning a few behavior parameters within the framework
of a bunch of pre-written rules that are known to produce a challenging
opponent. Full learning is too likely to produce the phenomenom of leaving
the land attack route completely unguarded if a player attacks ten times
from the sea (to use someone else's example). I think reasonable future
technology limits us to presenting the appearance of learning, rather than
trying to use real learning.
Does anyone remember the "Trillion Credit Squadron" game for the
_Traveller_ game system? The idea of the game was to build a fleet of
space warships, and fight them against other players' fleets. The players
were human, but because of how the rules worked, it turned out that optimal
play for that game consisted of a strategy that produced very boring games,
in part because there was a very specific strategy that always defeated any
other strategy, and in part because that strategy itself wasn't very much
fun. Finding that optimal strategy was kind of fun, in a math-puzzle sort
of way, but once it was found there was no point to playing the game again.
If learning produces boring play, don't learn!
Getting around the sandbag C&C strategy would be something that learning
might be able to do, but pre-written rules could do it too, if play-testing
found out how that strategy broke the game. A nice rule-based solution
would be for the computer to fall for that strategy for 3 games (or
whatever), then start using some pre-written counter-strategy.
Steve Schonberger
From jjudd@matcom.com.au Tue Jul 15 01:35:43 1997
Return-Path:
Received: from mailrelay.real3d.com by real3d.com (SMI-8.6/SMI-SVR4)
id BAA25577; Tue, 15 Jul 1997 01:35:42 -0400
Received: from mail.digiweb.com by mailrelay.real3d.com (SMI-8.6/SMI-SVR4)
id BAA03389; Tue, 15 Jul 1997 01:35:06 -0400
Received: (from condor@localhost)
by mail.digiweb.com (8.8.5/8.8.5) id BAA09171
for woodcock@real3d.com; Tue, 15 Jul 1997 01:33:10 -0400 (EDT)
Resent-Date: Tue, 15 Jul 1997 01:33:10 -0400 (EDT)
X-Authentication-Warning: mail.digiweb.com: condor set sender to gamedesign-request@digiweb.com using -f
Message-ID:
From: "Judd, John"
To: "'gamedesign@mail.digiweb.com'"
Subject: RE: Game AI
Date: Tue, 15 Jul 1997 15:00:26 +0930
X-Mailer: Microsoft Exchange Server Internet Mail Connector Version 4.0.995.52
Encoding: 102 TEXT
Resent-Message-ID: <"o63-uB.A.yKC.hswyz"@mail>
Resent-From: gamedesign@mail.digiweb.com
Reply-To: gamedesign@mail.digiweb.com
X-Mailing-List: archive/latest/333
X-Loop: gamedesign@digiweb.com
Precedence: list
Resent-Sender: gamedesign-request@mail.digiweb.com
Content-Length: 4484
Status: RO
>----------
>From: Steve Schonberger[SMTP:stevesch@csealumni.UNL.edu]
>Sent: Tuesday, 15 July 1997 13:53
>To: gamedesign@mail.digiweb.com
>Subject: Re: Game AI
>
>> From: Paul Nash
>> Date: Monday, July 14, 1997 11:42 AM
>[...]
...snip
>
>Aside from adventure and puzzle games, you shouln't be able to "solve" a
>game, in the sense of coming to a point of not being able to improve any
>further. Sure, there comes a point where your reaction time doesn't get
>any better, and your knowledge of the levels doesn't meaningfully improve,
>but a game should have depth where even an expert can learn new tricks.
>That should be the case in any game that doesn't have a real beginning and
>end, or at least the space to keep advancing should be large enough that
>hardcore players are still amused by the time the sequal is done.
In complete agreement
>> > > If you could design an AI that can detect specific
>> > > gameplay deficiencies in the human and somehow adapt to
>> > > them, that would be very cool.
>
>I'm not convinced that would be so cool, at least not from the "fun" point
>of view. It's obviously very cool from the technology point of view!
It may not be fun at a lower level of play, but depending on the level
of play that the player has chosen (this should be a hard option ) it
would force the human player to correct his/her deficiencies based on a
increased challenge.
>[...]
>> [Paul Nash]
>> I'd be interested to know if any of those ideas are embodied in
>> AOE type games insofar as "building" people. What if people got smarter
>> as their civilization advanced, or learned from previous battles. I
>> guess Close Combat has some complex soldier AI, but I'd like to see it a
>> little more subtle -- sort of a developing collective consciousness or
>> something instead of "Joe Sixpack is scared and he has three rounds
>> left." That tends to promote micro-management because you know exactly
>> what everyone is doing and feeling and thus feel obligated to correct it
>> all.
>
>I still think that the pathological cases of learning forbid using learning
>in a game, aside from tuning a few behavior parameters within the framework
>of a bunch of pre-written rules that are known to produce a challenging
>opponent. Full learning is too likely to produce the phenomenom of leaving
>the land attack route completely unguarded if a player attacks ten times
>from the sea (to use someone else's example). I think reasonable future
>technology limits us to pres