Online learning in commercial computer games allows computer-controlled opponents to adapt to the way the game is being played.
As such it provides a mechanism to deal with weaknesses in the game AI, and to respond to changes in human player tactics.
We argue that online learning of game AI should meet four computational and four functional requirements. The computational
requirements are speed, effectiveness, robustness and efficiency. The functional requirements are clarity, variety, consistency
and scalability. This paper investigates a novel online learning technique for game AI called ‘dynamic scripting’, that uses
an adaptive rulebase for the generation of game AI on the fly. The performance of dynamic scripting is evaluated in experiments
in which adaptive agents are pitted against a collection of manually-designed tactics in a simulated computer roleplaying
game. Experimental results indicate that dynamic scripting succeeds in endowing computer-controlled opponents with adaptive
performance. To further improve the dynamic-scripting technique, an enhancement is investigated that allows scaling of the
difficulty level of the game AI to the human player’s skill level. With the enhancement, dynamic scripting meets all computational
and functional requirements. The applicability of dynamic scripting in state-of-the-art commercial games is demonstrated by
implementing the technique in the game
Neverwinter Nights. We conclude that dynamic scripting can be successfully applied to the online adaptation of game AI in commercial computer
games.
Keywords Computer game - Reinforcement learning - Dynamic scripting
Editors: Michael Bowling, Johannes Fürnkranz, Thore Graepel, and Ron Musick