Export KPvK endings positions (up to black/white symmetry) from ChessBase 15 using python-chess

Question

I have the Mega Database 2020 from ChessBase and ChessBase 15. I am using ChessBase 15 to search the Mega Database 2020, my search criterion being all games ended through a KPvK (king and pawn vs. king) up to white/black symmetry.

For this I open the Mega Database in ChessBase 15 and I do "Filter List --> Material". The best I can achieve so far is :

(If I click on "No pieces" as I should, it resets the "Total" four cases, and I don't understand why, so that I am not sure that my filter does exactly want I and gives me all games I really want. I checked on https://en.chessbase.com/support-kb/content/details/1053/Material_searches_with_ChessBase but it did not help very much, but that is another question ...)

This gives a list of games, and if I click on the line corresponding to a game, the position appears for instance as follows :

We see that the board is automatically set up to the position of game defined by the ply where the KPvK (up to colour symmetry) begins.

Is it possible to export all of these positions in a database, retaining the ply move plus players, dates and others usual information?

This could allow to create a KPvK training database. Of course, it is somewhat trivial regarding the nature of the ending, but the principle would be useful for other types of endings.

Dont have chessbase but I guess you click on pieces [rather than no pieces] and then select just kings for W/B. — edwina oliver, Feb 17 '20 at 19:20
@Olorin Sounds like something that you can easily solve by yourself with a bit of scripting. I recommend using python-chess for such purposes, and you'll also find various solved examples here on chess SE using that library. For instance here's one on parsing PGNs in order to find games with quadrupled pawn structure. By analogy, first figure out the shape of FENs that match your requirement then look for that pattern. — Ellie, Feb 18 '20 at 09:21
@Phonon Yes, this is actually the conclusion I drew, see my comment to PhishMaster's answer. I already wrote a python script and I am actually just debugging it. Will post it as an answer when it will be done. — Olórin, Feb 18 '20 at 09:53
@Phonon Thank's for the link, it will be useful later for sure. In my case it is quite easy, I loop on games in the pgn, then on moves of the games by pushing them on the board, to each such board I get the board FEN (without castling info) and in this FEN I just search get the occurences of the pieces/pawns and check it they match the occurences of the pieces/pawns I want in my ending and then I print that in a new pgn, without games algebraic notations but with a FEN header, the FEN being the FEN of the position I want if it exists in the game — Olórin, Feb 18 '20 at 09:56
@Olorin Great to see you opted for this solution, well done! Might want to update the question post/title accordingly for future readers. — Ellie, Feb 18 '20 at 11:00

Olórin · Answer 1 · 2020-02-18T18:56:09.260

I used the following python script using python-chess library :

import chess
import chess.pgn

def KPvK_up_to_symmetry_export_to_FEN_func(pgn_file_path, nb_games, output_pgn_file_name):
    output_pgn = open(output_pgn_file_name,"w+")
    games = 0
    pgn_file = open(pgn_file_path, encoding="utf-8-sig")
    while nb_games == 0 or games < nb_games:
        game = chess.pgn.read_game(pgn_file)
        board = game.board()
        has_wanted_position = False
        wanted_fen = ""
        for move in game.mainline_moves():
            board.push(move)
            fen = board.board_fen()

            nb_p = fen.count("p")
            nb_P = fen.count("P")

            nb_n = fen.count("n")
            nb_N = fen.count("N")

            nb_b = fen.count("b")
            nb_B = fen.count("B")

            nb_r = fen.count("r")
            nb_R = fen.count("R")

            nb_q = fen.count("q")
            nb_Q = fen.count("Q")

            nb_k = fen.count("k")
            nb_K = fen.count("K")

            no_pieces = ((nb_n == 0) and (nb_N == 0) and (nb_b == 0) and (nb_B == 0) and (nb_r == 0) and (nb_R == 0) and (nb_q == 0) and (nb_Q == 0))

            cond = ((nb_k == 1) and (nb_K == 1) and (nb_P == 1) and (nb_p == 0)) #KPvK
            cond_symmetrical = ((nb_k == 1) and (nb_K == 1) and (nb_P == 0) and (nb_p == 1)) #KvKP

            #cond = (nb_B == 2 and nb_b < 2)
            #cond_symmetrical = (nb_b == 2 and nb_B < 2)

            if ((cond or cond_symmetrical) and no_pieces):
                has_wanted_position = True
                wanted_fen = board.fen()
                break
            else:
                continue
        #We are out of the loop on game moves, with a FEN or with nothing (if the game hasn't the ending we want)
        if (has_wanted_position):
            for header_key, header_value in game.headers.items():
                output_pgn.write("[" + header_key + " \"" + header_value + "\"]")
                output_pgn.write("\n")
            output_pgn.write("[FEN \"" + wanted_fen + "\"]")
            output_pgn.write("\n")
            output_pgn.write("\n")
        games += 1
    output_pgn.close()
    return None

nb_games_total = 1523849
KPvK_up_to_symmetry_export_to_FEN_func(r"C:\CHESS\PYTHON\megadatabase2020.pgn", nb_games_total, "megadatabase2020_KPvP.pgn")

I did this for the pgn file I got from chessbase by exporting to text PGN my whole megadatabase 2020. Executing this script will produce a pgn file containing positions, that I'll be able to use back in chessbase.

score 0 · Answer 2 · answered Feb 17 '20 at 20:56

0

To be honest, I am not 100% sure what you are asking for by "up to white/black symmetry."

I think this solution will help you find what you are looking for. Theme keys are not enabled by default in ChessBase. There are amazingly detailed Theme, Tactics, Strategy, and Endgame keys built into Mega 2020 (and prior versions).

Take a look at the difference in the tabs with the theme keys enables. Once you have them enabled, you can take whatever games you want, and put them in the clipboard, and export the clipboard to a new database.

Next under File>Options>Misc, check the "theme keys box".

answered Feb 17 '20 at 20:56

PhishMaster

32,477
4
105
177

By "KPvK up to colour symmetry" I mean "KPvK or KvKP". Let me see what I get when I enable the themes – Olórin Feb 17 '20 at 21:18
Ok, when I check the "Use Theme Key box, and when I open the database and go to the endings tab, I have an empty windows with four buttons : "select key", "install empty key", "intall big key", "install small key". – Olórin Feb 17 '20 at 21:23
@Olorin, did you pay for the database, and install it from the DVD? Or did you just get the database files? If you did not install it from the DVD, I could see them not being there. – PhishMaster Feb 17 '20 at 21:26
1

I paid the web version of it and installed it after download. I just clicked the "install big key" button and after some time it installed me everything and I have the same as you. Let me see in details now. – Olórin Feb 17 '20 at 21:36
Ok, I am in the pawn endings, in the "1..8 P : K" section. The name of the section makes me thing that it means "K + 1 up to 8 P against K" but so far I see only "KPvK up to symmetry" games in it. Now I will see if I can export the desired positions, because that is the point of my question. – Olórin Feb 17 '20 at 21:42
To be precise : using you "ending tab" filtering method or the one I describe in my question, I end with a list of games, and if I click on any game in the list, the board will open the game at the first position where there only two kings and a pawn. The gives a position P(G) per game G. I would like to export the list of P(G)'s for all game G in the list, in a PGN file. – Olórin Feb 17 '20 at 21:45
What I am trying to do : in a given chessbase database (let's say the metadatabase 2020), find the list L of all games having let's say KPvK (up to symmetry) ending (apparently this already not possible), for each such game G in the list L find the first position P(G) of the game where there is only two kings and a pawn. When G varies in L, the P(G)'s give a list M of positions (one position per game in L). I want to export the list M in a pgn file. I hope I make more sense now ? – Olórin Feb 17 '20 at 22:17
First, the part about symmetry, I finally got it, and you mean "up to equal material". It is not going to be possible for it to truncate the the games, and only show the position from the K+P ending. That would be a manual task. – PhishMaster Feb 17 '20 at 22:28
Ah ah now I don't understand you. So I repeat : by "KPvK" I mean "white king + one pawn vs black king" and by "KPvK up to colour symmetry" I mean "white king + one pawn vs black king or black king + one pawn vs white king". As for the truncation, I think that I will select the whole games list obtained after filtering, right click on "output" and choose text file, PGN format. Then I will have to parse this file and for each game in it convert the game in a list of FEN positions (one per game move), and then parse this FEN list get the first FEN matching the material (KPvK or KvKP) I want. – Olórin Feb 18 '20 at 07:58
And I will have to code to do it. :) – Olórin Feb 18 '20 at 07:58
Actually it seems that I won't have to code much for it, as there is a python chess library https://python-chess.readthedocs.io/en/latest/index.html allowing nice manipulation of PGN's. Will post the final code in the answer I will give to my own question. – Olórin Feb 18 '20 at 08:52
Final code posted as an answer – Olórin Feb 18 '20 at 10:54

Export KPvK endings positions (up to black/white symmetry) from ChessBase 15 using python-chess

2 Answers2