0

Related, but outdated: Tips on how to beat a computer?

How to beat a super Go computer: https://paperswithcode.com/paper/adversarial-policies-beat-professional-level

This of course suggests that the same specific weakness holds for (the newest era - it won't work against, say, Stockfish) chess computers. Do you know of any scientific work on the area? Or at least of a severe blunder by a NN/DL-based computer?

Hauke Reddmann
  • 16,242
  • 3
  • 27
  • 70
  • The same specific weakness is unlikely to hold for a chess program, as chess and Go are different games and therefore failure modes of bots will be different.

    It is easy to find positions that are misevaluated and misplayed by top computer programs, both neural and traditional ones, even at high depth. I don't think anyone has gone to the trouble of using a significant amount of computation in an attempt to find out if an adversarial policy can systematically generate such positions in real games.

     – Polytropos Apr 03 '23 at 21:47

1 Answers1

2

People have tried, to no success. My justification for saying this comes from two angles:

  1. Leela Chess Zero is trained by getting each new neural network to beat the previous one. In other words, each new network is an "adversary" of the previous network.
  2. At one point, people also tried to create "Antifish" which was a neural network built to beat Stockfish (which was at the time using handcrafted eval). Antifish didn't beat Stockfish convincingly, and was noticeably weaker than Leela. The project has since been discontinued, as you can see from the fact that the last update to that repository was 5 years ago.

That doesn't mean NN engines don't blunder catastrophically, see e.g. the answer I wrote here.

Allure
  • 26,534
  • 1
  • 68
  • 146