Polyglots, List and Bookthinker

[Volker Pittlik]

Geschrieben von:/Posted by: Volker Pittlik at 02 April 2004 15:45:17:

I tested the new version of Polyglot with List.
Without Bookthinker it seems to work fine, but if I try to use Bookthinker additionally it exits.


POLYGLOT WAITING
< ENGINE 
< ENGINE List 512
< ENGINE 
< ENGINE id name List 512
< ENGINE id author Fritz Reul
< ENGINE option name Hash type spin default 64 min 1 max 1024
< ENGINE option name NalimovCache type spin min 8 max 8 default 8
< ENGINE option name NalimovPath type string default c:\tb
< ENGINE option name DrawScore type spin default 15 min -50 max 50
< ENGINE option name PlayingSpeed type spin default 27 min 25 max 30
< ENGINE uciok
< ENGINE readyok
> XBOARD feature done=1
< XBOARD EOF
It happenend after Bookthinker has ended to move:
<tt>
<pcode>
Interrupting first
13219 >first : time 12954
otim 12914
13219 >first : b8d7
Fatal Error: Error: first chess program (Bookthinker-List polyglot) exited unexpectedly
GameEnds(0, (null), 2)
Interrupting first
13359 >first : quit
13359 >second: quit


With the version from yesterday that doesn't happened.
Volker

[Fabien Letouzey]

Geschrieben von:/Posted by: Fabien Letouzey at 02 April 2004 15:55:35:
Als Antwort auf:/In reply to: Polyglots, List and Bookthinker geschrieben von:/posted by: Volker Pittlik at 02 April 2004 15:45:17:

I tested the new version of Polyglot with List.
Without Bookthinker it seems to work fine, but if I try to use Bookthinker additionally it exits.

The PolyGlot log file is not helpful, it does not correspond to the problem.
So I can't see what happened.
There must be some "assertion" mentionned in the correct log file.
Fabien.

[Fabien Letouzey]

Geschrieben von:/Posted by: Fabien Letouzey at 02 April 2004 15:57:58:
Als Antwort auf:/In reply to: Polyglots, List and Bookthinker geschrieben von:/posted by: Volker Pittlik at 02 April 2004 15:45:17:

I tested the new version of Polyglot with List.
Without Bookthinker it seems to work fine, but if I try to use Bookthinker additionally it exits.

BookThinker's log file might be better then WinBoard's one in this case.
It's getting complex with so many programs between the interface and the engine ...
Fabien.

[Volker Pittlik]

Geschrieben von:/Posted by: Volker Pittlik at 02 April 2004 16:01:45:
Als Antwort auf:/In reply to: Re: Polyglots, List and Bookthinker geschrieben von:/posted by: Fabien Letouzey at 02 April 2004 15:57:58:

...

BookThinker's log file might be better then WinBoard's one in this case.

Unfortunately there isn't one.
Volker

[Volker Pittlik]

Geschrieben von:/Posted by: Volker Pittlik at 02 April 2004 16:02:52:
Als Antwort auf:/In reply to: Re: Polyglots, List and Bookthinker geschrieben von:/posted by: Fabien Letouzey at 02 April 2004 15:55:35:

...

There must be some "assertion" mentionned in the correct log file.

Unfortunately there isn't.
Volker

[Fabien Letouzey]

Geschrieben von:/Posted by: Fabien Letouzey at 02 April 2004 16:13:26:
Als Antwort auf:/In reply to: Polyglots, List and Bookthinker geschrieben von:/posted by: Volker Pittlik at 02 April 2004 15:45:17:

I tested the new version of Polyglot with List.
Without Bookthinker it seems to work fine, but if I try to use Bookthinker additionally it exits.

This in the PolyGlot log file:


< XBOARD EOF
</tt>

Means that BookThinker either crashed or closed the communication pipe.
There are only two differences between yesterday's PolyGlot and the one 3 days ago: one of them is a patch for List (not activated by default), and the other one alters log output when EOF is received.
So in short the behaviour is not supposed to be different as before.
Fabien.

[Matthias Gemuh]

Geschrieben von:/Posted by: Matthias Gemuh at 02 April 2004 17:40:35:
Als Antwort auf:/In reply to: Re: Polyglots, List and Bookthinker geschrieben von:/posted by: Fabien Letouzey at 02 April 2004 16:13:26:

I tested the new version of Polyglot with List.
Without Bookthinker it seems to work fine, but if I try to use Bookthinker additionally it exits.

This in the PolyGlot log file:


Means that BookThinker either crashed or closed the communication pipe.
There are only two differences between yesterday's PolyGlot and the one 3 days ago: one of them is a patch for List (not activated by default), and the other one alters log output when EOF is received.
So in short the behaviour is not supposed to be different as before.
Fabien.

>< XBOARD EOF
></tt>
>


I don't think an adapter must provide special support for every engine.
The handful engines that have special peculiarities can be ignored.
/Matthias.


BigLion + Taktix

[Fabien Letouzey]

Geschrieben von:/Posted by: Fabien Letouzey at 02 April 2004 17:52:31:
Als Antwort auf:/In reply to: Too much List compatibility ? geschrieben von:/posted by: Matthias Gemuh at 02 April 2004 17:40:35:

I don't think an adapter must provide special support for every engine.
The handful engines that have special peculiarities can be ignored.

I know. If List were not that strong I wouldn't spend even 5s thinking about it. Also because of the problems the author had in the past, I have like 1% hope for a potential bugfix.
Note that it does not seem to help anyway, List needs to be used with the "reuse" WinBoard option deactivated, and seems to work fine then. I'm going to remove the work around.
Fabien.

[Günther Simon]

Geschrieben von:/Posted by: Günther Simon at 02 April 2004 23:15:41:
Als Antwort auf:/In reply to: Re: Polyglots, List and Bookthinker geschrieben von:/posted by: Fabien Letouzey at 02 April 2004 15:55:35:

I tested the new version of Polyglot with List.
Without Bookthinker it seems to work fine, but if I try to use Bookthinker additionally it exits.

The PolyGlot log file is not helpful, it does not correspond to the problem.
So I can't see what happened.
There must be some "assertion" mentionned in the correct log file.
Fabien.

Same here tonight. With BookThinker Polyglot exits
immediately after book end. No log will be written too.
Best regards,
Günther

[Lance Perkins]

Geschrieben von:/Posted by: Lance Perkins at 02 April 2004 23:20:35:
Als Antwort auf:/In reply to: Re: Polyglots, List and Bookthinker geschrieben von:/posted by: Fabien Letouzey at 02 April 2004 16:13:26:

BookThinker never closes io handles. Its the robust thing to do. When the application exits, the OS process manager will close them anyway.
If BookThinker has crashed, DrWatson would have put out a message box for it. It does not have a top-level exception handler, so the OS and the default debugger (Dr Watson) would catch it.
BookThinker logs to the debug console. Just run dbmon, debugview, or any other viewer of the the debug console.
Have you actually tried the scenario that made you arrive at your conclusions?
Here are a few suggestions for your code:
1. Do not use stdio functions for dealing with pipes in Win32 (as in your 'my_file_read_line' function). Instead use native Win32 API's (GetStdHandle, ReadFile/ReadFileEx, etc.). Doing feof (stdin) with pipes is not realiable. You could get an eof even for other errors.
2. Do not assume that the application that you spawned has terminated if you get pipe IO errors with it (as in your 'engine_get' function). Instead, test the process handle to see if the child process has actually terminated.
3. Use threads to handle IO or use overlapped IO. Your use of stdio in your single thread design causes you application to block on IO. You will miss async IO from either the IU or the child engine (this could explain the timer problems).
See the code of WinBoard for a good example.

I tested the new version of Polyglot with List.
Without Bookthinker it seems to work fine, but if I try to use Bookthinker additionally it exits.

This in the PolyGlot log file:


Means that BookThinker either crashed or closed the communication pipe.
There are only two differences between yesterday's PolyGlot and the one 3 days ago: one of them is a patch for List (not activated by default), and the other one alters log output when EOF is received.
So in short the behaviour is not supposed to be different as before.
Fabien.

>< XBOARD EOF
></tt>
>

[Fabien Letouzey]

Geschrieben von:/Posted by: Fabien Letouzey at 05 April 2004 12:09:12:
Als Antwort auf:/In reply to: Re: Polyglots, List and Bookthinker geschrieben von:/posted by: Lance Perkins at 02 April 2004 23:20:35:


So you understand the situation better, it is useful to let you know
that I am not a Windows user, let alone a Windows programmer.
PolyGlot is not a Win32 application, but a POSIX application compiled
with GCC. Of course, you have noticed that.
In the long term, the correct solution is probably that a Windows
programmer takes over and "ports" PolyGlot to the Win32 API, as this
was done with XBoard. This would take a lot of time though and what
I'm offering instead is a sort of emulated solution.
Also note that I have completely re-written the I/O routines last
week-end, and the currently known problems might already be gone.

BookThinker never closes io handles. Its the robust thing to do. When
If BookThinker has crashed, DrWatson would have put out a message box
BookThinker logs to the debug console. Just run dbmon, debugview, or
Have you actually tried the scenario that made you arrive at your
Here are a few suggestions for your code:
1. Do not use stdio functions for dealing with pipes in Win32 (as in
2. Do not assume that the application that you spawned has terminated
3. Use threads to handle IO or use overlapped IO. Your use of stdio
See the code of WinBoard for a good example.

the application exits, the OS process manager will close them anyway.
From PolyGlot's point of view, this makes no difference; the pipe was
closed, it doesn't matter by whom. Also in the POSIX world, closing
pipes is the natural way of saying "quit". There is nothing to be
read anymore, so the job is done.
for it. It does not have a top-level exception handler, so the OS and
the default debugger (Dr Watson) would catch it.
any other viewer of the the debug console.
conclusions?
I don't have Windows; that's why I can only rely on problems beeing
reported here. Hopefully, Windows users can now describe what happens
with more details.
your 'my_file_read_line' function). Instead use native Win32 API's
(GetStdHandle, ReadFile/ReadFileEx, etc.). Doing feof (stdin) with
pipes is not realiable. You could get an eof even for other errors.
1a) using the stdio API was for portability
1b) "not reliable"? Please explain.
Does the ANSI C implementation not fully work on Windows?
In POSIX there is "file abstraction"; it does not matter whether
the file is a regular one, a pipe or a socket. End-of-file is
meaningful for them all, and is treated in the same way.
Update. I don't use fgets()/feof() anymore in PolyGlot, but I would
be glad to learn the correct way to distinguish between actual errors
and end-of-file. Clearly the definition of fgets() is problematic.
Any experience in that area, let me know.
if you get pipe IO errors with it (as in your 'engine_get'
function). Instead, test the process handle to see if the child
process has actually terminated.
That's an idea, but IMO it can't be the *source* of a problem. I
don't know any engine which closes "stdout", then keeps on doing
stuff. So yes there is potential improvement, but in my view it would
be a work around for strange behaviour. I will do it at some point,
though.
in your single thread design causes you application to block on
IO. You will miss async IO from either the IU or the child engine
(this could explain the timer problems).
Non-blocking I/O is another possibility. What I do now is probably
what you call "overlapped" I/O, PolyGlot does not block waiting for
end-of-lines. Again, IMO this could not be the *source* of a problem,
as both xboard and UCI protocols require that all I/O must be line
terminated. Any counter example welcome, of course.
As for threads. If performance is an issue (e.g. hundreds of
simultaneously-connected engines), they are the right approach. For
PolyGlot they would be overkill. The select() approach (multi-file
polling) is the natural idiom in POSIX. The main problem with threads
is that more and more race conditions appear when the main code grows
in size. I chose select() because it's free of race conditions.
Also, I would like to hear about a way to do multi-file polling in
Win32 (without threads).
WinBoard is a *port* of xboard. If you want an efficient
implementation of PolyGlot, somebody will have to port it as well.
Fabien.

[Fabien Letouzey]

Geschrieben von:/Posted by: Fabien Letouzey at 05 April 2004 12:50:18:
Als Antwort auf:/In reply to: Re: Polyglots, List and Bookthinker geschrieben von:/posted by: Lance Perkins at 02 April 2004 23:20:35: