Optimizing Agent Parameters

An implementation of the N-Tuple Bandit Evolutionary Algorithm (NTBEA) is included in the framework in the package players.optimisers.

In order for an AI player to be compatible with the NTBEA tuning process, it must:

• extend from ParameterizedPlayer.java instead of Player.java and implement reset(seed, playerID) as an extra method.
• bundle its parameters in a class extending from ParameterSet.java and implement the methods required for extracting, setting and translating parameters.

In the framework, currently RHEA and MCTS implementations only are compatible.

How to run

Step 1

Navigate to players.optimisers.ntbea.RunNTBEA.java class.

Step 2

Create an instance of a ParameterSet class corresponding to the player to be optimized, e.g.:

RHEAParams parameterSet = new RHEAParams();

As NTBEA does not currently support parameter dependencies, only non-dependant parameters are going to be optimised (line 36 in RunNTBEA.java can be commented out to include all parameters).

Step 3

Create an instance of the player to be optimised, received its ParameterSet, e.g.

ParameterizedPlayer player = new RHEAPlayer(0, 0, parameterSet);

Step 4

Adjust any NTBEA parameters desired in this class, such as kExplore, epsilon, nEvals (iterations of the algorithm) or tuples to use (lines 51-54 in RunNTBEA.java).

Step 5

Class players.optimisers.evodef.EvaluatePommerman.java evaluates a given parameter set for the agent in Pommerman. For each evaluation, it runs 4 games against SimplePlayer agents, one game in each starting position, keeping the same board. The resulting fitness of a parameter set is the average of all 4 games played. Changes that can be done in the evaluation function include changing opponent players or repeating the evaluation several times.

Step 6

Run class RunNTBEA.java to optimise parameters.