Skip to content

Commit 5f164e7

Browse files
authored
Update IntelligentPoolDetails.md
1 parent 73191a3 commit 5f164e7

File tree

1 file changed

+15
-6
lines changed

1 file changed

+15
-6
lines changed

Documents/IntelligentPoolDetails.md

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,15 +18,19 @@ During the development of the materials for the Computational Intelligence in Ga
1818
alt="SimpleCase"
1919
width="600" border="10" />
2020
</p>
21-
21+
<p align="center">
22+
<em>Simple case scene</em>
23+
</p>
2224
In the end, at least until now, PPO is not working at all. We ended up make a even simpler case than the simple case in the beginning, with only one red ball and 4 pockets, and the game restarts after every shot.
2325

2426
<p align="center" id="simpler-case-image">
2527
<img src="Images/IntelligentPool/SimplerCase.png"
2628
alt="SimplerCase"
2729
width="600" border="10" />
2830
</p>
29-
31+
<p align="center">
32+
<em>Even simpler case scene</em>
33+
</p>
3034
Here I will go through the development process, describe each example scenes, tell how to play with them, and explain why I think the billiard game does not work directly with pure PPO or supervised learning.
3135

3236
## What we have tried
@@ -120,7 +124,10 @@ This case, the same method is used as in case 3, but with much simpler scenario.
120124
<p align="center">
121125
<img src="Images/IntelligentPool/HeatMapSimpler.png"
122126
alt="HeatMapSimpler"
123-
width="600" border="10" />
127+
width="400" border="10" />
128+
</p>
129+
<p align="center">
130+
<em>Heapmap</em>
124131
</p>
125132

126133
According to the heatmap, now the "better solutions" are not that scatterd as in case 3 anymore. That means it is easier for MAES to find the optimal solutions and the average solutions learned by neural network make more sense.
@@ -135,15 +142,17 @@ After collecting 20000 samples and training the neural network as in case 2 for
135142
</p>
136143

137144
<p align="center">
138-
<em>With neural network only</em>
145+
<em>Simpler case played by neural network only</em>
139146
</p>
140147

141-
If we use the output from neural network as the initial guess of the optimizer, the iteration count is about reduced from 10 to 5 in our case. Nice!
148+
If we use the output from neural network as the initial guess of the optimizer, the average iteration count to find a satisfied solution is about reduced from 10 to 5 in our case. Nice!
142149
<p align="center">
143150
<img src="Images/IntelligentPool/ReducedMAESIteration.png"
144151
alt="ReducedMAESIteration"
145152
width="600" border="10" />
146-
153+
</p>
154+
<p align="center">
155+
<em>MAES iteration comparison when changingfrom MAES only to MAES with neural network- blue line: average iteration count. purple line: average score. </em>
147156
</p>
148157

149158
### Case 6 - 1 red ball, 4 pockets, one shot, MAES and Supervised Learning using GAN

0 commit comments

Comments
 (0)