Surprising performance {to me} by GLC 3 verses Comet B60

[Jose Carlos]

Geschrieben von: / Posted by: Jose Carlos at 02 April 2003 09:21:58:
Als Antwort auf: / In reply to: Surprising performance {to me} by GLC 3 verses Comet B60 geschrieben von: / posted by: Dann Corbit at 02 April 2003 08:06:06:

Now, as theory would have it, programs with access to tablebase files should do very well in the endgame. At least as well as programs without EGTB, one would think. Watching some games by GLC 3.0 against Comet B60, I saw Comet often get the upper hand, only to be trounced in the late endgame. I find this result quite surprising.
Using this binary:
Directory of E:\programme\winboard\Comet
06/03/03 08:38p 290,816 Comet-B60.exe
E:\programme\winboard\Comet>comet-b60

Comet Version B.60
Copyright (C) 1998-2003, Ulrich Tuerke, 28.02.2003

initializing table base
found 66 uncompressed and 154 compressed new TBs
found 5-piece TBs in e:\programme\winboard\nalimov
using 14019 KB for Compression
using 6144 KB for TB cache

Hash Table Sizes:
Transposition Table: 24575K
EGTB Probe Cache: 4096K
Evaluation Cache: 4096K
Chess
Against this binary:
Directory of E:\programme\winboard\glc
17/03/03 12:49p 565,328 glc300.exe
E:\programme\winboard\glc>glc300.exe
Green Light Chess
Version 3.00
Copyright (c) 2000, 2001, 2002, 2003 by Tim Foden; All Rights Reserved.
Information: Failed to open file: glc300.eval
-- Default evaluation values will be used.
Console is a TTY
Hash table size is 6.0MB
Opening book "primary.hbk" contains 31911 positions

We see that Comet finds the EGTB files just fine, and that GLC does not use them (indeed, it has no provision for me to tell it where they are, even).
And yet, here is the outcome of some slow time control games. They become even more surprising when you play through them:

I haven't read the games. I firmly believe EGTB's _must_ help, as they are perfect knowledge and thus you can completely prune the subtree below, which can't be bad ever. But as they're so slow to access, the outcome can be productive or counter-productive, depending on the implementation.
In my new program (which developes very slowly due to lack of time) I'm trying to put much knowledge in the endgame eval instead of using tablebases, to see how it works. At this moment, Anubis (the new program) kills Averno in the endgame almost always. My next test will be to port Anubis eval into Averno (quite difficult as Averno is 0x88 and Anubis is bitboard) and see how it works.
I'll post my results when I think the data is reliable.
José C.

[Aaron]

Geschrieben von: / Posted by: Aaron at 02 April 2003 12:59:54:
Als Antwort auf: / In reply to: Re: Surprising performance {to me} by GLC 3 verses Comet B60 geschrieben von: / posted by: Jose Carlos at 02 April 2003 09:21:58:

Now, as theory would have it, programs with access to tablebase files should do very well in the endgame. At least as well as programs without EGTB, one would think. Watching some games by GLC 3.0 against Comet B60, I saw Comet often get the upper hand, only to be trounced in the late endgame. I find this result quite surprising.
We see that Comet finds the EGTB files just fine, and that GLC does not use them (indeed, it has no provision for me to tell it where they are, even).
And yet, here is the outcome of some slow time control games. They become even more surprising when you play through them:

I haven't read the games. I firmly believe EGTB's _must_ help, as they are perfect knowledge and thus you can completely prune the subtree below, which can't be bad ever. But as they're so slow to access, the outcome can be productive or counter-productive, depending on the implementation.
In my new program (which developes very slowly due to lack of time) I'm trying to put much knowledge in the endgame eval instead of using tablebases, to see how it works. At this moment, Anubis (the new program) kills Averno in the endgame almost always. My next test will be to port Anubis eval into Averno (quite difficult as Averno is 0x88 and Anubis is bitboard) and see how it works.
I'll post my results when I think the data is reliable.
José C.

From your PGN, it looks like GLC won 6-0. This is IMHO even more suprising.
GLC has improved a lot recently, but 6-0 (statistical flukes not withstanding) against Comet at slow time controls is quite astonishing.
Aaron

[Audy Arandela]

Geschrieben von: / Posted by: Audy Arandela at 02 April 2003 16:01:29:
Als Antwort auf: / In reply to: Re: Surprising performance {to me} by GLC 3 verses Comet B60 geschrieben von: / posted by: Aaron at 02 April 2003 12:59:54:

Now, as theory would have it, programs with access to tablebase files should do very well in the endgame. At least as well as programs without EGTB, one would think. Watching some games by GLC 3.0 against Comet B60, I saw Comet often get the upper hand, only to be trounced in the late endgame. I find this result quite surprising.
We see that Comet finds the EGTB files just fine, and that GLC does not use them (indeed, it has no provision for me to tell it where they are, even).
And yet, here is the outcome of some slow time control games. They become even more surprising when you play through them:

I haven't read the games. I firmly believe EGTB's _must_ help, as they are perfect knowledge and thus you can completely prune the subtree below, which can't be bad ever. But as they're so slow to access, the outcome can be productive or counter-productive, depending on the implementation.
In my new program (which developes very slowly due to lack of time) I'm trying to put much knowledge in the endgame eval instead of using tablebases, to see how it works. At this moment, Anubis (the new program) kills Averno in the endgame almost always. My next test will be to port Anubis eval into Averno (quite difficult as Averno is 0x88 and Anubis is bitboard) and see how it works.
I'll post my results when I think the data is reliable.
José C.

From your PGN, it looks like GLC won 6-0. This is IMHO even more suprising.
GLC has improved a lot recently, but 6-0 (statistical flukes not withstanding) against Comet at slow time controls is quite astonishing.
Aaron

Yes, I have tested GLC v3.00 and it is very strong...but is there a way where it can access EGTB, so it can get better:)
Audy

[Tim Foden]

Geschrieben von: / Posted by: Tim Foden at 02 April 2003 16:09:48:
Als Antwort auf: / In reply to: Re: Surprising performance {to me} by GLC 3 verses Comet B60 geschrieben von: / posted by: Audy Arandela at 02 April 2003 16:01:29:

Now, as theory would have it, programs with access to tablebase files should do very well in the endgame. At least as well as programs without EGTB, one would think. Watching some games by GLC 3.0 against Comet B60, I saw Comet often get the upper hand, only to be trounced in the late endgame. I find this result quite surprising.
We see that Comet finds the EGTB files just fine, and that GLC does not use them (indeed, it has no provision for me to tell it where they are, even).
And yet, here is the outcome of some slow time control games. They become even more surprising when you play through them:

I haven't read the games. I firmly believe EGTB's _must_ help, as they are perfect knowledge and thus you can completely prune the subtree below, which can't be bad ever. But as they're so slow to access, the outcome can be productive or counter-productive, depending on the implementation.
In my new program (which developes very slowly due to lack of time) I'm trying to put much knowledge in the endgame eval instead of using tablebases, to see how it works. At this moment, Anubis (the new program) kills Averno in the endgame almost always. My next test will be to port Anubis eval into Averno (quite difficult as Averno is 0x88 and Anubis is bitboard) and see how it works.
I'll post my results when I think the data is reliable.
José C.

From your PGN, it looks like GLC won 6-0. This is IMHO even more suprising.
GLC has improved a lot recently, but 6-0 (statistical flukes not withstanding) against Comet at slow time controls is quite astonishing.
Aaron

Yes, I have tested GLC v3.00 and it is very strong...but is there a way where it can access EGTB, so it can get better:)

I'm afraid not. EGTB's (of a sort... they are unlikely to be Namilovs... more likely some kind of BitTables) are on my TODO list... but there are a lot of other things on it too!
Cheers, Tim.



Green Light Chess Site

[Tim Foden]

Geschrieben von: / Posted by: Tim Foden at 02 April 2003 16:21:16:
Als Antwort auf: / In reply to: Surprising performance {to me} by GLC 3 verses Comet B60 geschrieben von: / posted by: Dann Corbit at 02 April 2003 08:06:06:

Now, as theory would have it, programs with access to tablebase files should do very well in the endgame. At least as well as programs without EGTB, one would think. Watching some games by GLC 3.0 against Comet B60, I saw Comet often get the upper hand, only to be trounced in the late endgame. I find this result quite surprising.

Here are the final positions from the games (I've attempted to display diags):
3r1rk1/8/6p1/p2n4/PpR1QP2/1P5P/3p3K/q2N4 b - - 0 42

4r2k/2Q2R2/1p4pP/p2q4/3p4/P3p3/1P5P/5R1K w - - 4 61

8/8/8/7p/5P1P/8/2rkp3/1K4R1 b - - 3 58

7k/6p1/2Bq1p1p/4p3/4P1P1/4BQ2/1r5P/2R3K1 w - - 0 90

8/1k6/p3p3/1p1r4/2p1n2p/R1P1BKpP/8/8 b - - 1 59

8/8/5k2/pb6/2p3P1/P1P1N3/1K2B3/8 w - - 1 76

Most of these positions still have quite a lot of pieces on the board, so I wouldn't expect endgame tablebases to have much effect. It looks to me as if they were all won/lost quite a while before the EGTB would really start to come into play.
Cheers, Tim.


Green Light Chess Site

[U.Tuerke]

Geschrieben von: / Posted by: U.Tuerke at 02 April 2003 18:34:28:
Als Antwort auf: / In reply to: Re: Surprising performance {to me} by GLC 3 verses Comet B60 geschrieben von: / posted by: Tim Foden at 02 April 2003 16:21:16:

Now, as theory would have it, programs with access to tablebase files should do very well in the endgame. At least as well as programs without EGTB, one would think. Watching some games by GLC 3.0 against Comet B60, I saw Comet often get the upper hand, only to be trounced in the late endgame. I find this result quite surprising.

Here are the final positions from the games (I've attempted to display diags):
3r1rk1/8/6p1/p2n4/PpR1QP2/1P5P/3p3K/q2N4 b - - 0 42

4r2k/2Q2R2/1p4pP/p2q4/3p4/P3p3/1P5P/5R1K w - - 4 61

8/8/8/7p/5P1P/8/2rkp3/1K4R1 b - - 3 58

7k/6p1/2Bq1p1p/4p3/4P1P1/4BQ2/1r5P/2R3K1 w - - 0 90

8/1k6/p3p3/1p1r4/2p1n2p/R1P1BKpP/8/8 b - - 1 59

8/8/5k2/pb6/2p3P1/P1P1N3/1K2B3/8 w - - 1 76

Most of these positions still have quite a lot of pieces on the board, so I wouldn't expect endgame tablebases to have much effect. It looks to me as if they were all won/lost quite a while before the EGTB would really start to come into play.
Cheers, Tim.

Agreed, for most of these positions, egtbs are completely irrelevant. For the remaining few, i consider it very unlikely that they would have had any effect.
Cordial congratulations, Tim.
Very impressive playing level !
Uli