Hi
,we would appreciate it very much if more testers would to participate in our winboard engine grand test. There is still time until Sunday for those who want to have some influence on voting concerning participants and conditions like time control, hash, EGTB etc.. Of course others can participate later, but will then have to take those engines already chosen by the others. I do not post in other fora, but testers who are known there can of course try to post there in oder to receive more help.
Up to now we have the following who showed interest:
for group A (strongest free engines):
Volker Pittlik
Olivier Deville
Igor Gorelikov
Slobodan R. Stojanovic
Christian Koch
Roger Brown
Heinz van Kempen
for group B (middle class and weaker engines)
Volker Anuss
Heinz van Kempen
especially here I hope for more assistance
I am dreaming for some time already to have a tournament like this, because it is not possible (not even with three faster computers) to play a tournament with more time for games and at the same time have enough games for all the fast updating top engines, that way that it is of some statistical significance. To be really statistically valuable more than 1000 games per engine must be played, what would require dozens of testers for only 30 engine versions.
For me it was always annoying being told for instance that quantity for me would be more important than quality, when running many hundreds of Blitz games for a lot of engines just to have some good statistics. You cannot tell this to a correspondence player
. What I do not like when someone claims for example, that even five times faster hardware will give no difference in performance or outcome and then telling at the same time, that 40/40 game on a Pentium 600 is giving significantly other results than shorter games on four or five times faster hardware, because this is highly contradictory. Such a game on P600 would have been with "more time" in the year 1995 or so, but not in 2004. So I think we already agreed that with the help of Axon Benchmark or a similar tool in this tournament we should all be able to simulate more or less the same hardware conditions (2 Ghz for instance).One interesting proposal came from Igor. Starting with only ten engines and then adding one by one all the other strong ones by gauntlets, that give them the same amount of games.
For me a big tournament means not only many games, but also a lot of participants. The top amateurs (very strong engines) in my opinion include at least 50 engines (if not more). So here is a short calculation example that should not have influence on our democratic votes on Sunday:
Start with 16 engines, round robin, all 4 games against all others=60 games for all for the beginning. After that each tester could add one strong engine of his choice after the other. Igor for example picks Fruit (just examples, I still do not know who will be in the first 16, as I did not receive votes from all up to now) and Fruit plays 4x16 games=64 games. Christian picks WildCat, playing also against Fruit and all the others, what means 4x17 games = 68 games. After including another 8 engines (and than totally having in 26 engines all would have played 100 games and this could be continued for all who are still interested. So gradually both aims are fulfilled, having in more and more engines and having more and more games for each one in the end.
The bad thing is when starting such a tournament there are already obsolete versions in after only a few weeks, because some authors are updating frequently. But I would opt then to add first all strong engines and later on replace older versions by updates. This all depends on how long people are willing to proceed with a tournamet like that, until they want again do things exclusively on their own and it depends on other factors, for example new testers that could be interested later on.
How much time this will all need? Given an average of 60 moves per game a 40/40 game will last two hours. A game 30+3 (Igor´s proposal) would only last one hour making possible twice as much games. The question is, if there would be a high difference in quality between those two time controls. I made some tests a few months ago, when playing with tournament time control, I watched the move that would have been chosen in the main line after only five seconds.
Eight of ten moves on average were identical after five seconds and three minutes for most engines. Nonetheless the remaining 20% different moves can well be decisive ones, they can as well be alternatives, that only differ by a centipawn in evaluation.
A last request to all testers. Please write to me about thoughts, proposals, etc., so that I can include those in the voting list. For example those tournament modes Slobodan wanted to propose. Is their something other than round robin possible in such an event? Or does this refer to number of games, that should be in my opinion the same amount for all?
Best Regards
Heinz