Motivated Reinforcement Learning

by ;
Format: Hardcover
Pub. Date: 2009-06-01
Publisher(s): Springer-Verlag New York Inc
List Price: $159.99

Rent Textbook

Select for Price
There was a problem. Please try again later.

Rent Digital

Rent Digital Options
Online:30 Days access
Downloadable:30 Days
$35.64
Online:60 Days access
Downloadable:60 Days
$47.52
Online:90 Days access
Downloadable:90 Days
$59.40
Online:120 Days access
Downloadable:120 Days
$71.28
Online:180 Days access
Downloadable:180 Days
$77.22
Online:1825 Days access
Downloadable:Lifetime Access
$118.80
*To support the delivery of the digital material to you, a digital delivery fee of $3.99 will be charged on each digital item.
$77.22*

New Textbook

We're Sorry
Sold Out

Used Textbook

We're Sorry
Sold Out

How Marketplace Works:

  • This item is offered by an independent seller and not shipped from our warehouse
  • Item details like edition and cover design may differ from our description; see seller's comments before ordering.
  • Sellers much confirm and ship within two business days; otherwise, the order will be cancelled and refunded.
  • Marketplace purchases cannot be returned to eCampus.com. Contact the seller directly for inquiries; if no response within two days, contact customer service.
  • Additional shipping costs apply to Marketplace purchases. Review shipping costs at checkout.

Summary

Motivated learning is an emerging research field in artificial intelligence and cognitive modelling. Computational models of motivation extend reinforcement learning to adaptive, multitask learning in complex, dynamic environments ' the goal being to understand how machines can develop new skills and achieve goals that were not predefined by human engineers. In particular, this book describes how motivated reinforcement learning agents can be used in computer games for the design of non-player characters that can adapt their behaviour in response to unexpected changes in their environment.This book covers the design, application and evaluation of computational models of motivation in reinforcement learning. The authors start with overviews of motivation and reinforcement learning, then describe models for motivated reinforcement learning. The performance of these models is demonstrated by applications in simulated game scenarios and a live, open-ended virtual world. Researchers in artificial intelligence, machine learning and artificial life will benefit from this book, as will practitioners working on complex, dynamic systems ' in particular multiuser, online games.

Table of Contents

Non-Player Characters and Reinforcement Learning
Non-Player Characters in Multiuser Gamesp. 3
Types of Multiuser Gamesp. 4
Massively Multiplayer Online Role-Playing Gamesp. 4
Multiuser Simulation Gamesp. 5
Open-Ended Virtual Worldsp. 5
Character Roles in Multiuser Gamesp. 8
Existing Artificial Intelligence Techniques for Non-Player Characters in Multiuser Gamesp. 9
Reflexive Agentsp. 9
Learning Agentsp. 12
Evolutionary Agentsp. 14
Smart Terrainp. 14
Summaryp. 15
Referencesp. 15
Motivation in Natural and Artificial Agentsp. 17
Defining Motivationp. 17
Biological Theories of Motivationp. 20
Drive Theoryp. 20
Motivational State Theoryp. 22
Arousalp. 23
Cognitive Theories of Motivationp. 26
Curiosityp. 26
Operant Theoryp. 28
Incentivep. 29
Achievement Motivationp. 30
Attribution Theoryp. 31
Intrinsic Motivationp. 33
Social Theories of Motivationp. 35
Conformityp. 35
Cultural Effectp. 36
Evolutionp. 36
Combined Motivation Theoriesp. 37
Maslow's Hierarchy of Needsp. 38
Existence Relatedness Growth Theoryp. 38
Summaryp. 39
Referencesp. 40
Towards Motivated Reinforcement Learningp. 45
Defining Reinforcement Learningp. 45
Dynamic Programmingp. 47
Monte Carlo Methodsp. 48
Temporal Difference Learningp. 49
Reinforcement Learning in Complex Environmentsp. 52
Partially Observable Environmentsp. 52
Function Approximationp. 53
Hierarchical Reinforcement Learningp. 54
Motivated Reinforcement Learningp. 57
Using a Motivation Signal in Addition to a Reward Signalp. 58
Using a Motivation Signal Instead of a Reward Signalp. 64
Summaryp. 67
Referencesp. 68
Comparing the Behaviour of Learning Agentsp. 71
Player Satisfactionp. 71
Psychological Flowp. 72
Structural Flowp. 73
Formalising Non-Player Character Behaviourp. 73
Models of Optimality for Reinforcement Learningp. 74
Characteristics of Motivated Reinforcement Learningp. 78
Comparing Motivated Reinforcement Learning Agentsp. 81
Statistical Model for Identifying Learned Tasksp. 83
Behavioural Varietyp. 83
Behavioural Complexityp. 85
Summaryp. 86
Referencesp. 87
Developing Curious Characters Using Motivated Reinforcement Learning
Curiosity, Motivation and Attention Focusp. 91
Agents in Complex, Dynamic Environmentsp. 91
Statesp. 93
Actionsp. 94
Reward and Motivationp. 94
Motivation and Attention Focusp. 95
Observationsp. 96
Eventsp. 98
Tasks and Task Selectionp. 100
Experience-Based Reward as Cognitive Motivationp. 102
Arbitration Functionsp. 108
A General Experience-Based Motivation Functionp. 109
Curiosity as Motivation for Support Charactersp. 111
Curiosity as Interesting Eventsp. 111
Curiosity as Interesting and Competencep. 116
Summaryp. 119
Referencesp. 119
Motivated Reinforcement Learning Agentsp. 121
A General Motivated Reinforcement Learning Modelp. 121
Algorithms for Motivated Reinforcement Learningp. 123
Motivated Flat Reinforcement Learningp. 123
Motivated Multioption Reinforcement Learningp. 126
Motivated Hierarchical Reinforcement Learningp. 131
Summaryp. 133
Referencesp. 134
Curious Characters in Games
Curious Characters for Multiuser Gamesp. 137
Motivated Reinforcement Learning for Support Characters in Massively Multiplayer Online Role-Playing Gamesp. 138
Character Behaviour in Small-Scale, Isolated Games Locationsp. 141
Case Studies of Individual Charactersp. 142
General Trends in Character Behaviourp. 145
Summaryp. 148
Referencesp. 149
Curious Characters for Games in Complex, Dynamic Environmentsp. 151
Designing Characters That Can Multitaskp. 152
Case Studies of Individual Charactersp. 155
General Trends in Character Behaviourp. 156
Designing Characters for Complex Tasksp. 159
Case Studies of Individual Charactersp. 159
General Trends in Character Behaviourp. 161
Games That Change While Characters Are Learningp. 163
Case Studies of Individual Charactersp. 164
General Trends in Character Behaviourp. 167
Summaryp. 169
Referencesp. 170
Curious Characters for Games in Second Lifep. 171
Motivated Reinforcement Learning in Open-Ended Simulation Gamesp. 171
Game Designp. 172
Character Designp. 172
Evaluating Character Behaviour in Response to Game Play Sequencesp. 176
Discussionp. 187
Summaryp. 188
Referencesp. 189
Future
Towards the Futurep. 193
Using Motivated Reinforcement Learning in Non-Player Charactersp. 193
Other Gaming Applications for Motivated Reinforcement Learningp. 194
Dynamic Difficulty Adjustmentp. 194
Procedural Content Generationp. 195
Beyond Curiosityp. 195
Biological Models of Motivationp. 195
Cognitive Models of Motivationp. 196
Social Models of Motivationp. 196
Combined Models of Motivationp. 196
New Models of Motivated Learningp. 197
Motivated Supervised Learningp. 197
Motivated Unsupervised Learningp. 198
Evaluating the Behaviour of Motivated Learning Agentsp. 198
Concluding Remarksp. 198
Referencesp. 199
Indexp. 201
Table of Contents provided by Ingram. All Rights Reserved.

An electronic version of this book is available through VitalSource.

This book is viewable on PC, Mac, iPhone, iPad, iPod Touch, and most smartphones.

By purchasing, you will be able to view this book online, as well as download it, for the chosen number of days.

Digital License

You are licensing a digital product for a set duration. Durations are set forth in the product description, with "Lifetime" typically meaning five (5) years of online access and permanent download to a supported device. All licenses are non-transferable.

More details can be found here.

A downloadable version of this book is available through the eCampus Reader or compatible Adobe readers.

Applications are available on iOS, Android, PC, Mac, and Windows Mobile platforms.

Please view the compatibility matrix prior to purchase.