Return to Answer

This is a good question. your understanding in general is correct. Indeed, data can be used to construct a proper evaluation of a move/board position and recommended moves based on its history (at least alphago does).

Regarding your first point, it is possible that scenarios that never occurred in historical data could occur, but that's not a problem if your valuation procedure is strong. Typically this is modeled by some continuous function that takes your input board and gives some scalar value. Doing so implicitly assumes you learn the patterns among data pretty well so that ideally you can extrapolate sufficiently in new scenarios. If you had used a lookup table for every board state, you would be screwed in both of the respects you mentioned above.

Regarding your second point, problems related to storing the tree itself and efficiently querying has other solutions that essentially involve a clever pruning method (minimax, and its variants) and/or an efficient cache, like zobrist hashing (see chessprogramming wiki for more techniques).

Now to the core of the question, how do you evaluate and setup a loss function? There are several ways to do this though: roughly you can think about this as falling into the following buckets

• (i) a reinforcement learning approach, which Alphago uses (https://www.chess.com/article/view/whats-inside-alphazeros-brain) . This is interesting because RL is quite a different paradigm of learning than supervised learning. I'd recommend reading through some of sutton/barto (http://www.incompleteideas.net/book/the-book-2nd.html) to see how questions of policy learning and move recommendations can be derived in general; this is quite a readable book. Then read about alphazero and alphago https://www.science.org/doi/10.1126/science.aar6404.

• (ii) supervised learning. When I initially tried this out, I trained a neural net to output a value at least as good as the moves outputted by a grandmaster (see Should I use neural networks or genetic algorithms to solve Gomoku? for more details). Another way can be based on whether the player won or lost at the end, assuming that in general their moves were better when they won than when they lost (a little strong of an assumption to me personally). See https://www.cs.tau.ac.il/~wolf/papers/deepchess.pdf - They randomly sample moves from a game white won and lost, and learns to choose between the moves with a neural network.

To be explicit, once you setup the appropriate loss or policy optimization problem, you can perform gradient descent as usual. what you are optimizing would of course differ based on if you choose the RL route or the supervised learning route.

Since this is in the title of your question, I will additionally say that NNs help in chess because of this complex problem of how to valuate a position based on some unique progression and state of the game, and the combinatorial explosion (which is somewhat rectified by a move recommender in alphago, and/or intuitively to me, better evaluating positions may allow for more robust move selection given less plies). Without a NN, some more static evaluator or less complex function approximator may be too simplistic for robustly learning `good moves' in chess.

It is interesting to see that some methods do not require you to actually encode which moves are legal and which are not.

This is a good question. your understanding in general is correct. Indeed, data can be used to construct a proper evaluation of a move/board position and recommended moves based on its history (at least alphago does).

Regarding your first point, it is possible that scenarios that never occurred in historical data could occur, but that's not a problem if your valuation procedure is strong. Typically this is modeled by some continuous function that takes your input board and gives some scalar value. Doing so implicitly assumes you learn the patterns among data pretty well so that ideally you can extrapolate sufficiently in new scenarios. If you had used a lookup table for every board state, you would be screwed in both of the respects you mentioned above.

Regarding your second point, problems related to storing the tree itself and efficiently querying has other solutions that essentially involve a clever pruning method (minimax, and its variants) and/or an efficient cache, like zobrist hashing (see chessprogramming wiki for more techniques).

Now to the core of the question, how do you evaluate and setup a loss function? There are several ways to do this though: roughly you can think about this as falling into the following buckets

• (i) a reinforcement learning approach, which Alphago uses (https://www.chess.com/article/view/whats-inside-alphazeros-brain) . This is interesting because RL is quite a different paradigm of learning than supervised learning. I'd recommend reading through some of sutton/barto (http://www.incompleteideas.net/book/the-book-2nd.html) to see how questions of policy learning and move recommendations can be derived in general; this is quite a readable book. Then read about alphazero and alphago https://www.science.org/doi/10.1126/science.aar6404.

• (ii) supervised learning. When I initially tried this out, I trained a neural net to output a value at least as good as the moves outputted by a grandmaster (see Should I use neural networks or genetic algorithms to solve Gomoku? for more details). Another way can be based on whether the player won or lost at the end, assuming that in general their moves were better when they won than when they lost (a little strong of an assumption to me personally). See https://www.cs.tau.ac.il/~wolf/papers/deepchess.pdf - They randomly sample moves from a game white won and lost, and learns to choose between the moves with a neural network.

To be explicit, once you setup the appropriate loss or policy optimization problem, you can perform gradient descent as usual. what you are optimizing would of course differ based on if you choose the RL route or the supervised learning route.

Since this is in the first part of your question, I will additionally say that NNs help in chess because of this complex problem of how to valuate a position based on some unique progression and state of the game, and the combinatorial explosion (which is somewhat rectified by a move recommender in alphago, and/or intuitively to me, better evaluating positions may allow for more robust move selection given less plies). Without a NN, some more static evaluator or less complex function approximator may be too simplistic for robustly learning `good moves' in chess.

It is interesting to see that some methods do not require you to actually encode which moves are legal and which are not.

This is a good question. your understanding in general is correct. Indeed, data can be used to construct a proper evaluation of a move/board position and recommended moves based on its history (at least alphago does). Problems related to storing the tree itself and efficiently querying has other solutions that essentially involve a clever pruning method (minimax, and its variants) and/or an efficient cache, like zobrist hashing (see chessprogramming wiki for more techniques).

Regarding your first point, it is possible that scenarios that never occurred in historical data could occur, but that's not a problem if your valuation procedure is strong. Typically this is modeled by some continuous function that takes your input board and gives some scalar value. Doing so implicitly assumes you learn the patterns among data pretty well so that ideally you can extrapolate sufficiently in new scenarios. If you had used a lookup table for every board state, you would be screwed in both of the respects you mentioned above.

Now to the core of the question, how do you evaluate and setup a loss function? There are several ways to do this though: roughly you can think about this as falling into the following buckets

• (i) a reinforcement learning approach, which Alphago uses (https://www.chess.com/article/view/whats-inside-alphazeros-brain) . This is interesting because RL is quite a different paradigm of learning than supervised learning. I'd recommend reading through some of sutton/barto (http://www.incompleteideas.net/book/the-book-2nd.html) to see how questions of policy learning and move recommendations can be derived in general; this is quite a readable book. Then read about alphazero and alphago https://www.science.org/doi/10.1126/science.aar6404.

• (ii) supervised learning. When I initially tried this out, I trained a neural net to output a value at least as good as the moves outputted by a grandmaster (see Should I use neural networks or genetic algorithms to solve Gomoku? for more details). Another way can be based on whether the player won or lost at the end, assuming that in general their moves were better when they won than when they lost (a little strong of an assumption to me personally). See https://www.cs.tau.ac.il/~wolf/papers/deepchess.pdf - They randomly sample moves from a game white won and lost, and learns to choose between the moves with a neural network.

To be explicit, once you setup the appropriate loss or policy optimization problem, you can perform gradient descent as usual. what you are optimizing would of course differ based on if you choose the RL route or the supervised learning route.

It is interesting to see that some methods do not require you to actually encode which moves are legal and which are not.

This is a good question. your understanding in general is correct. Indeed, data can be used to construct a proper evaluation of a move/board position and recommended moves based on its history (at least alphago does).

Regarding your first point, it is possible that scenarios that never occurred in historical data could occur, but that's not a problem if your valuation procedure is strong. Typically this is modeled by some continuous function that takes your input board and gives some scalar value. Doing so implicitly assumes you learn the patterns among data pretty well so that ideally you can extrapolate sufficiently in new scenarios. If you had used a lookup table for every board state, you would be screwed in both of the respects you mentioned above.

Regarding your second point, problems related to storing the tree itself and efficiently querying has other solutions that essentially involve a clever pruning method (minimax, and its variants) and/or an efficient cache, like zobrist hashing (see chessprogramming wiki for more techniques).

Now to the core of the question, how do you evaluate and setup a loss function? There are several ways to do this though: roughly you can think about this as falling into the following buckets

• (i) a reinforcement learning approach, which Alphago uses (https://www.chess.com/article/view/whats-inside-alphazeros-brain) . This is interesting because RL is quite a different paradigm of learning than supervised learning. I'd recommend reading through some of sutton/barto (http://www.incompleteideas.net/book/the-book-2nd.html) to see how questions of policy learning and move recommendations can be derived in general; this is quite a readable book. Then read about alphazero and alphago https://www.science.org/doi/10.1126/science.aar6404.

• (ii) supervised learning. When I initially tried this out, I trained a neural net to output a value at least as good as the moves outputted by a grandmaster (see Should I use neural networks or genetic algorithms to solve Gomoku? for more details). Another way can be based on whether the player won or lost at the end, assuming that in general their moves were better when they won than when they lost (a little strong of an assumption to me personally). See https://www.cs.tau.ac.il/~wolf/papers/deepchess.pdf - They randomly sample moves from a game white won and lost, and learns to choose between the moves with a neural network.

To be explicit, once you setup the appropriate loss or policy optimization problem, you can perform gradient descent as usual. what you are optimizing would of course differ based on if you choose the RL route or the supervised learning route.

Since this is in the title of your question, I will additionally say that NNs help in chess because of this complex problem of how to valuate a position based on some unique progression and state of the game, and the combinatorial explosion (which is somewhat rectified by a move recommender in alphago, and/or intuitively to me, better evaluating positions may allow for more robust move selection given less plies). Without a NN, some more static evaluator or less complex function approximator may be too simplistic for robustly learning `good moves' in chess.

It is interesting to see that some methods do not require you to actually encode which moves are legal and which are not.

This is a good question. your understanding in general is correct. Indeed, data can be used to construct a proper evaluation of a move/board position and recommended moves based on its history (at least alphago does). Problems related to storing the tree itself and efficiently querying has other solutions that essentially evolve an efficient cache, like zobrist hashing (see chessprogramming wiki for more techniques).

Regarding your first point, it is possible that scenarios that never occurred in historical data could occur, but that's not a problem if your valuation procedure is strong. Typically this is modeled by some continuous function that takes your input board and gives some scalar value. Doing so implicitly assumes you learn the patterns among data pretty well so that ideally you can extrapolate sufficiently in new scenarios. If you had used a lookup table for every board state, you would be screwed in both of the respects you mentioned above.

Now to the core of the question, how do you evaluate and setup a loss function? There are several ways to do this though: roughly you can think about this as falling into the following buckets

• (i) a reinforcement learning approach, which Alphago uses (https://www.chess.com/article/view/whats-inside-alphazeros-brain) . This is interesting because RL is quite a different paradigm of learning than supervised learning. I'd recommend reading through some of sutton/barto to see how questions of policy learning and move recommendations can be derived in general; this is quite a readable book. Then read about alphazero and alphago https://www.science.org/doi/10.1126/science.aar6404.

• (ii) supervised learning. When I initially tried this out, I trained a neural net to output a value at least as good as the moves outputted by a grandmaster (see Should I use neural networks or genetic algorithms to solve Gomoku? for more details). Another way can be based on whether the player won or lost at the end, assuming that in general their moves were better when they won than when they lost (a little strong of an assumption to me personally). See https://www.cs.tau.ac.il/~wolf/papers/deepchess.pdf - They randomly sample moves from a game white won and lost, and learns to choose between the moves with a neural network.

To be explicit, once you setup the appropriate loss or policy optimization problem, you can perform gradient descent as usual. what you are optimizing would of course differ based on if you choose the RL route or the supervised learning route.

It is interesting to see that some methods do not require you to actually encode which moves are legal and which are not.

This is a good question. your understanding in general is correct. Indeed, data can be used to construct a proper evaluation of a move/board position and recommended moves based on its history (at least alphago does). Problems related to storing the tree itself and efficiently querying has other solutions that essentially involve a clever pruning method (minimax, and its variants) and/or an efficient cache, like zobrist hashing (see chessprogramming wiki for more techniques).

Regarding your first point, it is possible that scenarios that never occurred in historical data could occur, but that's not a problem if your valuation procedure is strong. Typically this is modeled by some continuous function that takes your input board and gives some scalar value. Doing so implicitly assumes you learn the patterns among data pretty well so that ideally you can extrapolate sufficiently in new scenarios. If you had used a lookup table for every board state, you would be screwed in both of the respects you mentioned above.

Now to the core of the question, how do you evaluate and setup a loss function? There are several ways to do this though: roughly you can think about this as falling into the following buckets

• (i) a reinforcement learning approach, which Alphago uses (https://www.chess.com/article/view/whats-inside-alphazeros-brain) . This is interesting because RL is quite a different paradigm of learning than supervised learning. I'd recommend reading through some of sutton/barto (http://www.incompleteideas.net/book/the-book-2nd.html) to see how questions of policy learning and move recommendations can be derived in general; this is quite a readable book. Then read about alphazero and alphago https://www.science.org/doi/10.1126/science.aar6404.

• (ii) supervised learning. When I initially tried this out, I trained a neural net to output a value at least as good as the moves outputted by a grandmaster (see Should I use neural networks or genetic algorithms to solve Gomoku? for more details). Another way can be based on whether the player won or lost at the end, assuming that in general their moves were better when they won than when they lost (a little strong of an assumption to me personally). See https://www.cs.tau.ac.il/~wolf/papers/deepchess.pdf - They randomly sample moves from a game white won and lost, and learns to choose between the moves with a neural network.

To be explicit, once you setup the appropriate loss or policy optimization problem, you can perform gradient descent as usual. what you are optimizing would of course differ based on if you choose the RL route or the supervised learning route.

It is interesting to see that some methods do not require you to actually encode which moves are legal and which are not.