One suggestion for computer chess bench and one utility

[Igor Gorelikov]

Geschrieben von:/Posted by: Igor Gorelikov at 14 April 2004 14:35:18:

One suggestion for computer chess bench and one utility
=======================================================
1. Suggestion
-------------
Nodes per second are usufull but what if they cannot help an engine to get more
depth in the chess game?
Better is to compare values of level/depth an engine can get during actual
chess games.
I assume that the best bench for chess engines is a tournament.
My suggestion is to select one well-known engine for reference to compare
its level/depth on different hardware, at different time controls and hash
settings.
My candidate is Yace:
- It is a strong and well-known chess engine.
- It stores level values into its own PGN file.
I suggest a new bench unit: Yace Mean Level (YML).
YML is arithmetical average of the first Yace's 10 values of depth (level)
in each game after it is out of book at the given time control and hash
settings. I select the most hard and therefore stable part of the chess
game. If you take an entire game you've got too big variance.
Where it can be used
=====================
- To compare the tournament results of an engine on different hardware and
under different conditions.
- To select the best time control for engine test
- To select the best hash settings for a engine at the given time cotrol.
To obtain YML, you have to either run matches/tournament with Yace
participants, or look into your yacegam.pgn file if you yet run Yace.
2. Utility
----------
I have created a utility to calculate Yace Mean Level (YML) on the base of
Yace game file (yacegam.pgn). Off course, you may feed it also with any PGN
file which contains depth values (Arena, Fritz GUI can produce such PGN
files) and obtain mean value for any chess engine (though you need to
run matches of the engine vs itself because the values are stored for both
sides).

The utility can be downloaded from here (2K):
http://www.digichess.gr/infiniteloop/tools/YML1.zip
It is a batch file that uses SED & AWK to do the job.
Further details can be found in the readme.txt file.
SED & AWK for Windows are available from here:
SED for Windows (40K)
http://sed.sourceforge.net/grabbag/ssed/sed-3.59.zip
AWK for Windows (140K)
http://cm.bell-labs.com/cm/cs/who/bwk/awk95.exe

Best regards,
Igor Gorelikov