Policy or Value ? Loss Function and Playing Strength in AlphaZero
Por um escritor misterioso
Last updated 18 junho 2024
Results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Recently, AlphaZero has achieved outstanding performance in playing Go, Chess, and Shogi. Players in AlphaZero consist of a combination of Monte Carlo Tree Search and a Deep Q-network, that is trained using self-play. The unified Deep Q-network has a policy-head and a value-head. In AlphaZero, during training, the optimization minimizes the sum of the policy loss and the value loss. However, it is not clear if and under which circumstances other formulations of the objective function are better. Therefore, in this paper, we perform experiments with combinations of these two optimization targets. Self-play is a computationally intensive method. By using small games, we are able to perform multiple test cases. We use a light-weight open source reimplementation of AlphaZero on two different games. We investigate optimizing the two targets independently, and also try different combinations (sum and product). Our results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Moreover, we find that care must be taken in computing the playing strength. Tournament Elo ratings differ from training Elo ratings—training Elo ratings, though cheap to compute and frequently reported, can be misleading and may lead to bias. It is currently not clear how these results transfer to more complex games and if there is a phase transition between our setting and the AlphaZero application to Go where the sum is seemingly the better choice.
Simple Alpha Zero
The Evolution of AlphaGo to MuZero, by Connor Shorten
🔵 AlphaZero Plays Connect 4
Value targets in off-policy AlphaZero: a new greedy backup
AlphaGo/AlphaGoZero/AlphaZero/MuZero: Mastering games using progressively fewer priors
reference request - How do neural networks play chess? - Artificial Intelligence Stack Exchange
RankNet for evaluation functions of the game of Go - IOS Press
Frontiers AlphaZe∗∗: AlphaZero-like baselines for imperfect information games are surprisingly strong
AlphaDDA: strategies for adjusting the playing strength of a fully trained AlphaZero system to a suitable human training partner [PeerJ]
Lecture 13: Reinforcement learning
AlphaDDA: strategies for adjusting the playing strength of a fully trained AlphaZero system to a suitable human training partner [PeerJ]
Monte-Carlo Graph Search for AlphaZero – arXiv Vanity
Frontiers AlphaZe∗∗: AlphaZero-like baselines for imperfect information games are surprisingly strong
Representation Matters: The Game of Chess Poses a Challenge to Vision Transformers – arXiv Vanity
AlphaZero Explained · On AI
Recomendado para você
-
AlphaZero really is that good18 junho 2024
-
Deepmind's AlphaZero Plays Chess18 junho 2024
-
Google's self-learning AI AlphaZero masters chess in 4 hours18 junho 2024
-
AlphaZero - Chessprogramming wiki18 junho 2024
-
DeepMind: the existence proof for RL at scale, by Nathan Lambert18 junho 2024
-
PDF) Reproducing Neural Network Research Findings via Reverse Engineering: Replication of AlphaGo Zero by Crowdsourced Leela Zero18 junho 2024
-
Alpha Kappa Alpha Sorority, Incorporated - Rho Xi Omega Chapter18 junho 2024
-
PDF] Reproducibility via Crowdsourced Reverse Engineering: A18 junho 2024
-
PDF] Reproducibility via Crowdsourced Reverse Engineering: A Neural Network Case Study With DeepMind's Alpha Zero18 junho 2024
-
How AlphaZero Learns Chess?. DeepMind and Google Brain researchers…, by Gayan Samuditha, Expo-MAS18 junho 2024
você pode gostar
-
Tinder introduces Blind Date: Users to get paired, make conversations before viewing profile, Technology News18 junho 2024
-
Receita de bolo formigueiro fofinho fácil de fazer18 junho 2024
-
Brain test nivel 88 deja entrar al gato por favor se esta congelando18 junho 2024
-
9.5.-N.A.S.T.Y Chords & Tabs - W.A.S.P18 junho 2024
-
Raelza, Handyman Saitou in Another World Wiki18 junho 2024
-
Gollum remake (by GhostinHell) video and gameplay soon : r/lordoftherings18 junho 2024
-
Adventure Circus Baby, Wiki18 junho 2024
-
The Faraway Paladin18 junho 2024
-
Gold Diggers of 1933'.. dir. Mervyn LeRoy..18 junho 2024
-
3D file Deinocheirus - Dinosaur 🦖・3D print design to download・Cults18 junho 2024