It's a plague of Winboard engines

Discussions about Winboard/Xboard. News about engines or programs to use with these GUIs (e.g. tournament managers or adapters) belong in this sub forum.

Moderator: Andres Valverde

It's a plague of Winboard engines

Postby Kirill Kryukov » 20 Aug 2009, 17:20

I tried to run some matches with GNU Chess 5.07, and noticed that it has the same bug with number of other Winboard engines: It does not count the opening moves from external book as part of time control. :( This is really getting annoying. So many engines are hit by this bug. The result - they have to move instantly from move 49 to 80 in my tourney, also from move 89 to 120 if they survive that long, etc. (See here for details and discussion). Needless to say that this hurts the performance badly. So instead I have to run such engines without any book at all, which also hurts the performance and does not allow to accurately estimate their strength. The source of the problem - the horrible Winboard protocol which leaves important details unspecified.

So the list of engines bugged with this same problem (known to me) (Last edited 2011-02-20):

Open source:
GNU Chess 5.07 => Fixed by Michel in version 5.07.2b (source, Jim Ablett's compile).
Resp 0.19
Exchess 5.02 beta

Closed source:
Bodo 0.2b
Betsy 6.51
Grizzly 1.40.1b
La Dame Blanche 2.0c
Laurifer 1.0
Matant 5.04
Mustang 4.97
Requiem 0.53
Storm 0.6

If at least the open source engines could be fixed - that would solve half of the problem. The closed source engines would still remain impossible to test properly.

Is there anything that can be done with this issue on the interface level?

Or is there anyone who could try and patch the affected open source engine? It's a protocol issue after all, not some deep logic.

In the meantime I'll test GNU Chess both with and without book (both handicapped in different ways), and the one which turns out stronger will be listed in my tourney.

Best,
Kirill
Last edited by Kirill Kryukov on 05 Aug 2012, 05:36, edited 8 times in total.
User avatar
Kirill Kryukov
 
Posts: 127
Joined: 21 Sep 2005, 09:56

Re: It's a plague of Winboard engines

Postby H.G.Muller » 20 Aug 2009, 18:20

Indeed, many engines are plagued by this. The only solution I found for it was to play incremental controls.

The source of the problem is not so much WB protocol, as well the protocol specs that were very unclear. (Especially the phrase "clock that always belongs to the engine, even if the engine changes color this clock remains with the engine" seems to cause confusion. Many people seem to interpret this as "time remains with the engine", which then leads to the logical conclusion that an engine times only its own moves, which leads to this bug. (I fell for that too, when I first adapted my engine to WB protocol.) What was meant is that the time command is always referring to the clock that the engine is playing. But the clocks remain with their color, whatever the engine does.

The current version of the protocol specs addresses this point unambiguously:
The clocks in fact always remain with the color. Which clock reading is relayed with "time", and which by "otim", is determined by which side the engine plays. Note that the way the clocks operate and receive extra time (in accordance with the selected time control) is not affected in any way by which moves are made by the engine, which by the opponent, and which were forced.


So I think I did solve the root cause, but that of course does no good for existing engines. Filing bug reports withe the authors is the only solution.
User avatar
H.G.Muller
 
Posts: 3078
Joined: 16 Nov 2005, 12:02
Location: Diemen, NL

Re: It's a plague of Winboard engines

Postby Kirill Kryukov » 21 Aug 2009, 07:18

Hi H.G.,

It's great that you improved the protocol specs! Hopefully new engines will not have this problem. (Still I stand that a stateful protocol is inherently full of pitfalls).

I mailed a bug report to bug-gnu-chess@gnu.org, but I'm not holding my breath. In my experience the longer an engine was not updated - the smaller is the chance that it will ever get updated.

Hoping for the best. :-)

Regards,
Kirill
User avatar
Kirill Kryukov
 
Posts: 127
Joined: 21 Sep 2005, 09:56

Re: It's a plague of Winboard engines

Postby Olivier Deville » 21 Aug 2009, 08:08

Hi Kirill

I have the email address of Chua Kong Sian. I can forward him the bug report if you want.

Olivier
User avatar
Olivier Deville
 
Posts: 1175
Joined: 26 Sep 2004, 19:54
Location: Aurec, France

Re: It's a plague of Winboard engines

Postby Kirill Kryukov » 21 Aug 2009, 08:15

Olivier Deville wrote:Hi Kirill

I have the email address of Chua Kong Sian. I can forward him the bug report if you want.

Olivier

Hi Oliver,

That would be nice! I emailed you a copy of the bug report. (You can also attach your comments to it if you like)

Thanks,
Kirill
User avatar
Kirill Kryukov
 
Posts: 127
Joined: 21 Sep 2005, 09:56

Re: It's a plague of Winboard engines

Postby H.G.Muller » 21 Aug 2009, 09:22

Kirill Kryukov wrote:(Still I stand that a stateful protocol is inherently full of pitfalls).

That might be, but I don't think that in this case staeless vs stateful has anything to do with it in this case. The problem is that on forcing in a move the state is not consistently updated: the move is accepted and the position altered, but the move counter is not incremented. Exactly the same could happen in an UCI engine: there you force in moves for every move the engine has to make. If an engine would not increment the move counter then, you would get similar problems...
User avatar
H.G.Muller
 
Posts: 3078
Joined: 16 Nov 2005, 12:02
Location: Diemen, NL

Re: It's a plague of Winboard engines

Postby plattyaj » 21 Aug 2009, 13:30

So, looking at the code for Schola I see I would fall prey to this gotcha too. If I understand correctly, what I need to do is keep move counts for each color, update them regardless of whether the engine or the controller "play" the move and that way if I get a series of moves while in force, the move counter is correct when I receive the "go" command?

Andy.
User avatar
plattyaj
 
Posts: 9
Joined: 13 Aug 2008, 02:30

Re: It's a plague of Winboard engines

Postby H.G.Muller » 21 Aug 2009, 15:52

That sounds like it would do it. You could also just count plies from the beginning of the game, and if you play M minutes for N moves, you should expect to get new time when the ply counter hits 2*N (and 4*N, etc.). It is not clear what you should do if you start from a FEN with a move number larger than 1. Fortunately, no known GUI seems to send such FENs. But to prepare for the future, you should take the move number contained in the FEN seriously. I.e. if it says 39, and you play 40/5, expect to get 5 more minutes after the next move.
User avatar
H.G.Muller
 
Posts: 3078
Joined: 16 Nov 2005, 12:02
Location: Diemen, NL

Re: It's a plague of Winboard engines

Postby Michel » 21 Aug 2009, 17:41

I tried to run some matches with GNU Chess 5.07, and noticed that it has the same bug with number of other Winboard engines: It does not count the opening moves from external book as part of time control. :(


Are you sure about this? I had a quick look at the source code of GNU Chess and the ply counter (called GameCnt) seems to be updated correctly during force mode.
Michel
 
Posts: 513
Joined: 01 Oct 2008, 12:15

Re: It's a plague of Winboard engines

Postby Kirill Kryukov » 21 Aug 2009, 18:05

Michel wrote:
I tried to run some matches with GNU Chess 5.07, and noticed that it has the same bug with number of other Winboard engines: It does not count the opening moves from external book as part of time control. :(


Are you sure about this? I had a quick look at the source code of GNU Chess and the ply counter (called GameCnt) seems to be updated correctly during force mode.

Yes I'm sure. This is how it behaves in my tournament, in every game. For example you can download the already finished GNU Chess 5.07 games from KCEC. (Only one 32-game match with Madeleine 0.2 so far). The time usage pattern is exactly same with other engines suffering from this bug, and it's repeating in every game.

Details: I am running GNU Chess 5.07 64-bit JA, through WB2UCU adapter, under ChessProgram8 GUI by Chessbase. The command I use is "gnuchess_x64.exe -s 2097152" to adjust the hash.

May be I should try it with different GUIs to make sure it's not the GUI or adapter problem. Although countless other Winboard engines play without this problem. Only these few have this issue.
User avatar
Kirill Kryukov
 
Posts: 127
Joined: 21 Sep 2005, 09:56

Re: It's a plague of Winboard engines

Postby Teemu Pudas » 21 Aug 2009, 19:58

H.G.Muller wrote:
Kirill Kryukov wrote:(Still I stand that a stateful protocol is inherently full of pitfalls).

That might be, but I don't think that in this case staeless vs stateful has anything to do with it in this case. The problem is that on forcing in a move the state is not consistently updated: the move is accepted and the position altered, but the move counter is not incremented. Exactly the same could happen in an UCI engine: there you force in moves for every move the engine has to make. If an engine would not increment the move counter then, you would get similar problems...

Not true. Why would a UCI engine even have a move counter when the GUI sends the number of remaining moves with the go command?
Teemu Pudas
 
Posts: 124
Joined: 16 Apr 2007, 14:03

Re: It's a plague of Winboard engines

Postby H.G.Muller » 21 Aug 2009, 20:21

Oh, does it? Well, then you would get an error when the engine would decrement that remaining-move-count for every move it receives.

It is always possible to make an erroneous implementation (by design, not mistake) if you don't properly understand what the counter is supposed to represent.
User avatar
H.G.Muller
 
Posts: 3078
Joined: 16 Nov 2005, 12:02
Location: Diemen, NL

Re: It's a plague of Winboard engines

Postby Teemu Pudas » 21 Aug 2009, 21:44

H.G.Muller wrote:Oh, does it? Well, then you would get an error when the engine would decrement that remaining-move-count for every move it receives.

The engine won't receive any moves during a search. Even when pondering, it's either stop->position(moves)->go or just "ponderhit".
Teemu Pudas
 
Posts: 124
Joined: 16 Apr 2007, 14:03

Re: It's a plague of Winboard engines

Postby Zach Wegner » 21 Aug 2009, 22:09

Kirill Kryukov wrote:It's great that you improved the protocol specs! Hopefully new engines will not have this problem. (Still I stand that a stateful protocol is inherently full of pitfalls).


UCI is not fully stateless, and even a stateless protocol could theoretically have the same problem. UCI just "holds the engine's hand" throughout the game. The bug is primarily an engine problem, and less so an xboard protocol problem since it naively assumes that engines will be able to handle this without bugs. Xboard could easily add new commands that give this information for every move (as it does with time/otim), but that wouldn't help these old engines. Plus, if you were to go to the effort of adding support for this command, you might as well just fix the bug.
User avatar
Zach Wegner
 
Posts: 182
Joined: 26 Sep 2004, 22:02
Location: Austin, Texas, USA

Re: It's a plague of Winboard engines

Postby Kirill Kryukov » 22 Aug 2009, 03:45

Back to GNU Chess 5.07. Here are details for one typical game showing this bug:


GNU Chess was doing well till move 41. Then it used up all its time on moves 41-48. After that it had about 5 seconds for remaining 32 moves, so has to move instantly. It gradually lost its advantage and eventually lost the game too.

How to read the log file. Lines with "S>" - What engine sent to the WB2UCI adapter. Lines with "S<" - what adapter sent to engine.

Since I test in blitz, normally I disable all logging. This is just one demonstration game, not a part of my tournament.
User avatar
Kirill Kryukov
 
Posts: 127
Joined: 21 Sep 2005, 09:56

Re: It's a plague of Winboard engines

Postby H.G.Muller » 22 Aug 2009, 07:36

Well, it is clear that GNU Chess counts erroneously. It ives its moves in the non-compliant "NR... MOVE" format, and it starts by saying: "1... e1g1". So it thinks the castling is the wrong move.

It seems that the "white" command resets its move count. I played it from the command line forcing in a few moves, and after you type "go" it produces a move with the correct number. But if you type "white" first (when white is to move, so that it should be a no-op), then the next move is number 1 again.

Proposed cure: Use InBetween to filter all "black" and "white" commands out of the GUI -> engine communcation.
User avatar
H.G.Muller
 
Posts: 3078
Joined: 16 Nov 2005, 12:02
Location: Diemen, NL

Re: It's a plague of Winboard engines

Postby Kirill Kryukov » 24 Aug 2009, 04:32

Interesting. Did you try it? Could you post a complete configuration? A workaround like this would be great.
User avatar
Kirill Kryukov
 
Posts: 127
Joined: 21 Sep 2005, 09:56

Re: It's a plague of Winboard engines

Postby H.G.Muller » 24 Aug 2009, 14:51

I did not try it, and I cannot guarantee it works for other engines. I only used InBetween once, for swapping the commands white and black when playing GNU Shogi under WinBoard. The InBetween.ini file I used for that was:

Code: Select all
[InBetween]
; You can set program option here if its not possible in the server program
;
; The priority flag is if you want the server app. to run with lower priority.
; This could be when you want use an engine to analyze games/positions and
; you also want to use the computer to other work. If this is not set it will
; try to see what the client wants and set the server to this priority.
; Don't use the high option is you don't know what you do.
;Priority := low, normal or high
;
;CommandLine := f:\cygwin\home\chess\GNShogi\gnushogi.exe
;
; The debug switch is if you want to see the command flow in a window when the
; server and client talks. 1 means viewpoint is on the client interface, 2 means
; viewpoint is on the server interface, and finaly 3 means that the viewpont is
; after the translation (input to client and input to server).
;Debug := 0, 1, 2 or 3
Debug := 0
;
; A try to pass any control signal.
;Ctrl := False or True
;
; The logfile is if you want that InBetween log all communications.
;Log := logfile.log

[Client2Server]
; Set the translation of client command to server here.
; Format:
;    clientword := serverword
black := zzzzz
white := black
zzzzz := white
[Server2Client]
; Set the translation of server response to the client here.
; Format:
;    serverword := clientword


So my guess is that

Code: Select all
white :=
black :=


in the Client2Server section is all you need. (The swapping actually was tricky, as there is apparently no guarantee in which order InBetween performs the substitutions.)
User avatar
H.G.Muller
 
Posts: 3078
Joined: 16 Nov 2005, 12:02
Location: Diemen, NL

Re: It's a plague of Winboard engines

Postby Kirill Kryukov » 25 Aug 2009, 04:30

Thanks H.G. Yes, I tried it, and also I tried
Code: Select all
white := <delete>
black := <delete>

in client to server part. Both did not have any effect on time usage pattern.
User avatar
Kirill Kryukov
 
Posts: 127
Joined: 21 Sep 2005, 09:56

Re: It's a plague of Winboard engines

Postby F. Bluemers » 25 Aug 2009, 05:57

Jim Ablett made several GNUchess compilations.
He might be able to fix it.
F. Bluemers
 
Posts: 174
Joined: 04 Sep 2008, 16:56
Location: Netherlands

Next

Return to Winboard and related Topics

Who is online

Users browsing this forum: Yahoo [Bot] and 2 guests

cron