Reasonable engines to test against (Not StockFish)

Question

I've built a chess engine in Python which I've been testing against StockFish through the use of the Python-Chess package.

Testing against StockFish seems/is problematic, and as a result I'm looking for suggestions of engines to test against.

There are two major issues with testing against StockFish (the first much more significant than the next)

StockFish is partially used in generating the training data for the engine's evaluation function.
StockFish is extremely good, and even given extremely little time will perform well. This makes it difficult to see the changes in the engine's strategy which resulted from a change in the engine's code

The engines I'm hoping to play against need to have no serious bugs or problems, and ideally would play at levels similar or a bit below a human master, without spending too much time running them. UCI engines are prefered so that my engine can easily communicate with them through the Python-Chess API (note that this doesn't mean my engine supports UCI).

If it helps anyone to have some context, my engine is Batch First.

Laschet Jain · Accepted Answer · 2023-09-13T16:37:14.623

8

Since your engine supports UCI, I would suggest checking out lichess.org. You can make a BOT account there and test against several other BOTs and people as well! Also, you can get your BOT rated in different time formats!!

I must warn you that most of the BOTs on lichess are some version of StockFish or other strong engines but you can find ample other BOTs.

Among strong BOTs (apart from Stockfish ones), I would suggest checking out:

LeelaChess: A neural net based BOT.
ASMFish

There are many more BOTs on lichess, you can view them here on lichess BOT team.

You can also find many not so strong self-implemented engines that you might like testing against, for example this.

edited Sep 13 '23 at 16:37

answered Jul 06 '18 at 21:21

Laschet Jain

598
4
12

That seems particularly enticing! Competing against a top-tier engine restricted to very little time is more appealing for obvious reasons, but I have concerns that the elo/give_time curve might not behave as I expect, and cause weird/undesirable results.
When restricted to very little time, would top-tier engines play similarly to the less strong engines?
– Sam Ragusa Jul 07 '18 at 02:30
But both the BOTs will restricted to little time, right? So I guess the optimal BOT would search more depth than the weaker BOT doesn't matter the time. Regarding performance given low time, I would suggest checking out this video, where a human defeat a strong engine in hyperbullet match https://www.streamerclips.com/twitch/penguingm1/RefinedAverageLaptopRedCoat. – Laschet Jain Jul 07 '18 at 07:03
Damn, I didn't knew you could setup a bot account there! – Fernando Aug 16 '18 at 18:35
@Fernando This feature was added around 2 months ago. – Laschet Jain Aug 16 '18 at 18:36
@LaschetJain I'm checking it right now, thank you! – Fernando Aug 16 '18 at 18:37

score 3 · Answer 2 · answered Jul 06 '18 at 16:26

3

http://computer-chess.org/doku.php?id=computer_chess:wiki:lists:chess_engine_list

Arasan
GreKo
Crafty
Rodent
Toga etc

answered Jul 06 '18 at 16:26

SmallChess

22,476
2
45
82

Umlin · Answer 3 · 2018-07-06T17:51:31.257

2

To complement the list given by @SmallChess, you can find at https://www.computerchess.org.uk/ccrl/4040/ an ELO ranking of many chess engines.

In the 2300-2600 range ("similar or a bit below a human master"), and amongst open-source engines, you have :

K2 0.87 (2580 ELO)
Jazz 840 64-bit 4CPU (2552 ELO)
Maverick 1.5 64-bit (2522 ELO)
Winter 0.1.1 64-bit (2492 ELO)
Lozza 1.18 64-bit (2452 ELO)
Topple 0.1.1 64-bit (2360 ELO)

I haven't tested these engines myself, but they seem to fit your criterias.

Update : If you're looking more for engines in the 2000-2300 range, then these engines might suit you better :

Absolute Zero 2.4.7.2 64-bit (ELO 2294)
CT800 1.30 64-bit (ELO 2253)
Zevra 1.8.4 r650 64-bit (ELO 2236)
ZCT 0.3.2451 (ELO 2210)
Tunguska 1.0 64-bit (ELO 2133)
Cinnamon 2.0 64-bit (ELO 2054)

edited Jul 06 '18 at 17:51

answered Jul 06 '18 at 17:12

Umlin

518
2
13

Similar or a bit below human master would be 2000-2200 USCF or up to 2200-2300 FIDE. – Herb Jul 06 '18 at 17:22
1

@HerbWolfe I interpreted "master" as GM level (2500-2600), and "a bit below" as FM to IM level (2300-2500). The original question is a little unclear on what "master" level mean exactly (I would have commented about it if I could). I'll update my answer anyway, but it would be best if the author could confirm what he meant exactly. – Umlin Jul 06 '18 at 17:43
1

I left the term 'master' purposely vague, because my familiarity with ELOs and the level of play they represents is lacking. I used just below a master level because my main point of reference is a friend who's rank is right below master (though I'm unsure what his actual ELO is).
Either way, your summarized list of engine's is appreciated!
– Sam Ragusa Jul 07 '18 at 02:37

score 1 · Answer 4 · answered Aug 16 '18 at 18:36

1

Try https://github.com/fernandotenorio/Tunguska And let me know if your Python engine beat mine! =D

answered Aug 16 '18 at 18:36

Fernando

425
6
12

score 1 · Answer 5 · answered Mar 14 '23 at 11:24

StockFish is extremely good, and even given extremely little time will perform well. This makes it difficult to see the changes in the engine's strategy which resulted from a change in the engine's code

You can use go depth x or go nodes y to get Stockfish to almost arbitrary strength levels.

score 0 · Answer 6 · answered Sep 14 '23 at 08:55

StockFish is extremely good, and even given extremely little time will perform well. This makes it difficult to see the changes in the engine's strategy which resulted from a change in the engine's code

Aside from the mentioned UCI depth option, Stockfish also supports different playing strengths. See Is there a list of approximate Elo ratings for each Stockfish level? for discussion on Elo.

score 0 · Answer 7 · answered Sep 15 '23 at 14:48

0

Here are a two suggestions:

answered Sep 15 '23 at 14:48

score -1 · Answer 8 · answered Jul 08 '18 at 00:32

-1

Here is a list of computer chess engine rankings. Elo's are provided.

http://www.computerchess.org.uk/ccrl/4040/

answered Jul 08 '18 at 00:32

hangejj

729
4
11

4

you post an URL that was already posted like 15 hours ago, what's the point? – lenik Jul 08 '18 at 07:44
Guess I overlooked the original post. People can do with the post as they like(ie reuse, remove because it is duplicated etc.) – hangejj Jul 08 '18 at 15:03

Reasonable engines to test against (Not StockFish)

8 Answers8