Automatic opening book creation....

[Roger Brown]

Geschrieben von: / Posted by: Roger Brown at 07 February 2004 18:30:45:

Hello all,
A question if I might?
Let us say that I have a desire to create an opening book based on the games of one of my all-time favourite players, Alexander Alekhine.
I would assemble a pgn file with say 1000 of his best games then what?
I want the chess engine to play as Alekhine all the time - not as the opponent 50% of the time and Alekhine 50% of the time. Puzzled?
Let us say that in my 1000 games Alekhine played with the black pieces 500 times, then it stands to reason the the opponents played 500 times with white. The chess engine would then be playing those moves for white for some of the games - not my goal.
The only way that I see around it would be to create a white and a black opening book. That way all games that Alekhine played as white could be placed in one file and all of his games as black in another. The engine would then be playing as Alekhine all the time.
There are a number of engines that use this approach such as the Baron and Averno but there are several dozens more which do not use this binary (separate black and white books)setup for opening books.
How do I remove the ill-effects of the opponent from my Alekhine, Capablanca etc books? Perhaps it is shockingly obvious in which case I ask that you be gentle in your replies....
Oh yes, I am well acquainted with the idea of using the high class games of GM's to get the best in opening theory and practice. I want to take the idea a step further to include only the moves of my favourite player.
Crafty or Yace would be good points of departure as they both have extensive documentation particularly on opening book creation.
Later.

[Matt McKnight]

Geschrieben von: / Posted by: Matt McKnight at 07 February 2004 20:16:59:
Als Antwort auf: / In reply to: Automatic opening book creation.... geschrieben von: / posted by: Roger Brown at 07 February 2004 18:30:45:

What about using only games which Alekhine won?

[Roger Brown]

Geschrieben von: / Posted by: Roger Brown at 07 February 2004 20:30:42:
Als Antwort auf: / In reply to: Automatic opening book creation.... geschrieben von: / posted by: Roger Brown at 07 February 2004 18:30:45:

Thanks for all the answers.
Of course, the King of Kings has the option I require to make it an automatic process for that engine. Oh but that more engines could do the same!
Tony, Michael, do you know of a way to automate the process of setting flags and/or extracting the moves of the winner only? I suspect that it would not be a genuine pgn file then as the standard would incorporate two players...
I just cannot see myself sitting down with 1000 Alekhine games and setting flags (?, !, !!) throughout. There are workarounds such as using hand tuned books but these are limited to the professional guis - Chesspartner comes to mind.
Any software ideas welcome....
Thanks again for contributing gentlemen.
Later.
Ps. A question. Does anyone know a way to make Winboard or Arena force an engine to play one side for a series of matches? I am thinking that if I could create a white book and a black book (oh the humanity!) then I could have engine play white for ten games in an engine-engine match then switch sides - after loading the black book - and play another ten games as black.
Any ideas?

Later, again.

[Roger Brown]

Geschrieben von: / Posted by: Roger Brown at 07 February 2004 20:40:42:
Als Antwort auf: / In reply to: Re: Automatic opening book creation.... geschrieben von: / posted by: Matt McKnight at 07 February 2004 20:16:59:

What about using only games which Alekhine won?

Hello Matt,
Let me use a simple example to make my point clear.
Suppose my pgn database consists of 10 Alekhine games - 5 as white, 5 as black and all of them victories for Alekhine.
The typical engine chews up the pgn and produces a book file. Now, if the engine is playing white, it may select the opening moves from the five games that Alekhine won - but it may also select the opening moves from the five games of the loser who played white when Alekhine played with the black pieces.
That would result in an engine that played like Alekhine only some of the time which would not be good enough for me.
Do you see my point now?
There have been all sorts of suggestions about using flags etc. but that is so much labour for a hobby!
Thanks for your post.
Later.

[Dieter Bürßner]

Geschrieben von: / Posted by: Dieter Bürßner at 07 February 2004 21:07:31:
Als Antwort auf: / In reply to: Automatic opening book creation.... geschrieben von: / posted by: Roger Brown at 07 February 2004 18:30:45:

Let us say that I have a desire to create an opening book based on the games of one of my all-time favourite players, Alexander Alekhine.

A feature like this was suggested to my by Matt Josephson. He was also nice enought, to write some documentation about it. You can find it in the file gmstyle.txt inside yace.zip. The procedure may sound a bit complicated - don't get frightened - it should be rather easy. All the things could be automated by a small batch file.
I suggest, you use a not annotated book as "main book" (dbbook.bin) and create a "learn book" of Alekhine's games as Matt describes. While Yace still has choices inside the Alekhine book, it will only use the info inside there. When falling out of the Alekhine book, it will chose moves from the main book.
With book_mode 2 (in yace.ini), Yace will play the moves with the same probabilities as Alekhine did on average. (Normally it would play the !! moves all with equal probability).
Regards,
Dieter

[Matt McKnight]

Geschrieben von: / Posted by: Matt McKnight at 07 February 2004 21:23:05:
Als Antwort auf: / In reply to: Re: Automatic opening book creation.... geschrieben von: / posted by: Roger Brown at 07 February 2004 20:40:42:

What about using only games which Alekhine won?

Hello Matt,
Let me use a simple example to make my point clear.
Suppose my pgn database consists of 10 Alekhine games - 5 as white, 5 as black and all of them victories for Alekhine.
The typical engine chews up the pgn and produces a book file. Now, if the engine is playing white, it may select the opening moves from the five games that Alekhine won - but it may also select the opening moves from the five games of the loser who played white when Alekhine played with the black pieces.
That would result in an engine that played like Alekhine only some of the time which would not be good enough for me.
Do you see my point now?
There have been all sorts of suggestions about using flags etc. but that is so much labour for a hobby!
Thanks for your post.
Later.

Well, I'm not exactly sure how other engines create the books, but I don't store a move unless it was found in games that won > 50% of the time. Using your example, there are no draws (I don't like to have many drawn games in book) so I'm not considering those. So when the engine is playing white, it would never play a move from one of the other five games because those moves never won! This is why I suggested have a book with only won games, where Alekhine won, and no drawn games, although I think you would probably be okay with them. Of course you might only be able to use Dorky for this, and that doesn't help you considering Dorky can't analyse it's way out of a wet paper bag.
I think that someone should be able to do this with their favorite player's pgn database. IMHO That would be the first step to having 'personalities'
Matt

[Dieter Bürßner]

Geschrieben von: / Posted by: Dieter Bürßner at 07 February 2004 22:05:38:
Als Antwort auf: / In reply to: Automatic opening book creation.... geschrieben von: / posted by: Roger Brown at 07 February 2004 18:30:45:

For Yace, save the following as alekhine.bat and run it. It assumes, that you have 2 files alw.pgn and alb.pgn with the games of Alekhine as white and black (use your database program to possibly filter out losses - if you want to). All files are assumed to be in the yace directory.
echo carlos_pgn alw.pgn alwa.pgn n 3 0 > tmp.inp
echo carlos_pgn alb.pgn alba.pgn n 0 3 >> tmp.inp
echo book_f alekhine.bin >> tmp.inp
echo book_c alwa.pgn 50000 60 1 1 1 >> tmp.inp
echo book_a alba.pgn 60 1 1 >> tmp.inp
yace nul < tmp.inp
After you ran the batch (which is untested ...), add
book_f alekhine.bin
; not really needed, see my other post
book_m 2
to yace.ini. Or perhaps lbook_f instead of book_f. If you use this as only book, moves from the opponent of Alekhine will not be selected, just as you wanted.
Regards,
Dieter

[Roger Brown]

Geschrieben von: / Posted by: Roger Brown at 07 February 2004 22:26:52:
Als Antwort auf: / In reply to: Re: Automatic opening book creation.... geschrieben von: / posted by: Dieter Bürßner at 07 February 2004 22:05:38:

For Yace, save the following as alekhine.bat and run it. It assumes, that you have 2 files alw.pgn and alb.pgn with the games of Alekhine as white and black (use your database program to possibly filter out losses - if you want to). All files are assumed to be in the yace directory.
echo carlos_pgn alw.pgn alwa.pgn n 3 0 > tmp.inp
echo carlos_pgn alb.pgn alba.pgn n 0 3 >> tmp.inp
echo book_f alekhine.bin >> tmp.inp
echo book_c alwa.pgn 50000 60 1 1 1 >> tmp.inp
echo book_a alba.pgn 60 1 1 >> tmp.inp
yace nul < tmp.inp
After you ran the batch (which is untested ...), add
book_f alekhine.bin
; not really needed, see my other post
book_m 2
to yace.ini. Or perhaps lbook_f instead of book_f. If you use this as only book, moves from the opponent of Alekhine will not be selected, just as you wanted.
Regards,
Dieter

Dieter, you are amazing!
I will try this as soon as I am my home computer.
Hmmmm, then again, why am I surprised at anything you do?
Thanks.
Later.

[Roger Brown]

Geschrieben von: / Posted by: Roger Brown at 07 February 2004 22:34:14:
Als Antwort auf: / In reply to: Re: Automatic opening book creation.... geschrieben von: / posted by: Matt McKnight at 07 February 2004 21:23:05:

Well, I'm not exactly sure how other engines create the books, but I don't store a move unless it was found in games that won > 50% of the time. Using your example, there are no draws (I don't like to have many drawn games in book) so I'm not considering those. So when the engine is playing white, it would never play a move from one of the other five games because those moves never won! This is why I suggested have a book with only won games, where Alekhine won, and no drawn games, although I think you would probably be okay with them.
I think that someone should be able to do this with their favorite player's pgn database. IMHO That would be the first step to having 'personalities'
Matt

So Dorky, like King of Kings, has a feature in the book creation function that allows for partial automation of the creation of my player's opening book. Good.

Of course you might only be able to use Dorky for this, and that doesn't help you considering Dorky can't analyse it's way out of a wet paper bag.

I think you do Dorky a great dis-service here.





My point of departure exactly! I want an automated way to have a pgn base file - which most engines can process - that contains only the moves of Alexander Alekhine or any other player I desire.
Thanks much for the information and please, watch what you say in front of Dorky.
Baron did not promote and I have not said one harsh word.....


Later.

[Thomas McBurney]

Geschrieben von: / Posted by: Thomas McBurney at 08 February 2004 05:44:14:
Als Antwort auf: / In reply to: Automatic opening book creation.... geschrieben von: / posted by: Roger Brown at 07 February 2004 18:30:45:

I have created an opening book tool for Kanguruh that would allow you to achieve this. Here is a screen shot of the book editor...

This tool allows you to choose a PGN file to import moves for the opening book. As you can see from the check boxes in the bottom left corner, you can choose to add winning games and/or drawing games. All you need to do is create a PGN file that only has winning games for Alekhine.

The edit tab allows you to customise the opening book to your liking by allowing you to delete lines you do not want and customise the probability of certain lines being choosen over others. The two columns on the right of the screen shows the moves found in the opening book for that position and what score each move has been allocated. You can change the scores by simply clicking on it with the mouse and typing in a new score. The higher the score the more Kanguruh will likely pick that move. Note: the scores are measured in centipawns.
Finally, Kanguruh has two parameters in the kanguruh.ini file. They are as follows -
BookVariation=10
ExitBook=-35
BookVariation allows you to choose how random Kanguruh chooses its moves. If you set it to 0, then Kanguruh will always choose the move with the best score. If you set it to 200, then Kanguruh will more likely play gambits or just plain bad openings. Of course you can set it to any number you like, the higher the number, the more random it will choose its moves.
ExitBook tells Kanguruh when to stop using the opening book and start thinking. If the score stored in the opening book file is lower than this setting, then Kanguruh will exit out of book and start thinking.

Cheers,
Tom.

[Roger Brown]

Geschrieben von: / Posted by: Roger Brown at 08 February 2004 12:10:53:
Als Antwort auf: / In reply to: Automatic opening book creation.... geschrieben von: / posted by: Roger Brown at 07 February 2004 18:30:45: