Neural network optimtzation of algorithm DeepStack for playing in Leduc Hold’em

Artificial intelligence is used to automatically control cars, underwater vehicles, aircraft, rockets, robots, text and speech recognition, medical diagnostics and so on. Also in recent years, artificial intelligence is often used in games - in computer games to find ways in two-dimensional or three-dimensional space, simulating the behavior of units, and in board games and card games. Since in most card games players can not accurately develop a winning strategy because the opponent always operates on certain information is unknown to the player, with the evolution of artificial intelligence the question arose whether such a system is capable of playing at the same level as a person and whether it can win a person. The article considers the implementation of a neural network and the selection of its structure, which is used in the DeepStack algorithm. Its detailed description and principle of operation are given. This algorithm is used to make decisions during poker. Poker is represented as a game with incomplete information. The calculation of the strategy is based on two parameters - the counterfactual values of the opponent and the range of the player. The player's range is the probability distribution of the player's possible hands, taking into account the achievement of some public state. Counterfactual values - the probability of the opponent receiving such a hand, which in any case will be higher than the player's hand. These parameters have been described in detail using the formulas from the game theory. The proposed neural network is used to calculate the strategy, namely counterfactual values of the opponent. As a neural network, a feedforward network was chosen. The input values for the network are the bet size as a part of the player's total bank and the encoded ranges of players that depend on the open cards. The network output is a vector of counterfactual values for each player and hand, which are then interpreted as a bet size. As the data for training, a set of solved poker situations was used, which included various bet values and hand combinations. The proposed neural network is used to calculate the strategy, namely counterfactual values of the opponent. The dataset consisted of 1 000 000 solved poker situations. Several network structures are considered: a network with 2 hidden layers, an activation function of Tanh, a network with 2 hidden layers, an activation function of ReLU, a network with 2 hidden layers, an activation function of SoftPlus, a network with 4 hidden layers, a Tanh activation function, a network with 4 hidden layers, a ReLU activation function, Network with 4 hidden layers SoftPlus activation function. From these structures, the optimal one was chosen. The criterion of choice is the exploitability of the strategy, which shows the number of chips that a player loses by playing a certain strategy. Of the considered structures, the optimal network has 4 hidden layers with the SoftPlus activation function. Ref. 13, fig. 9.