just a try :-)

[Heinz van Kempen]

Geschrieben von:/Posted by: Heinz van Kempen at 15 July 2004 10:41:16:

Hi ,
although I know that there a already a lot of interesting tournaments I just wanted to ask if there are other testers willing to combine their efforts and CPU power to run a really big tournament over a few weeks or longer for the best amateur engines with more time (40 moves in 40 minutes minimum).
My proposal would be that at least 20 engines (better some more) should be included, every GUI should be allowed and maybe we can have a mix of own books, Nunn, Noomen or other positions. All results combined should give the overall crosstable. Identical should be the time given for each game, slower computers add a bit of time and faster ones give a bit less, maybe that way that all are simulating a 2000 Mhz CPU. Of course I am also open for alternative proposals. To decide the participants for instance every tester could give a list with preferred engines via email to the others and those will be included that have the most votes. Another fair method would be to run a qualify with more engines to choose those that will play the main event.
I can´t do this alone when wanting to have a statistically sufficient number of games, but I can add at least two of my computers when we start in two weeks.
There must not be a chief organizer, but when anyone wants to do that job, it will be welcome. All testers should interchange results and games, so that interim reports and download links can be given here or at the websites.
In my opinion that would be an opportunity to say thanks to all who are still giving us strong engines for free.
Comments here and emails to hvankempen@web.de are highly appreciated.
I have few hopes that there will be a lot of interest, but it was worth a try.
Best Regards
Heinz

[Volker Pittlik]

Geschrieben von:/Posted by: Volker Pittlik at 15 July 2004 12:10:42:
Als Antwort auf:/In reply to: just a try geschrieben von:/posted by: Heinz van Kempen at 15 July 2004 10:41:16:

Hi ,
although I know that there a already a lot of interesting tournaments I just wanted to ask if there are other testers willing to combine their efforts and CPU power to run a really big tournament over a few weeks or longer for the best amateur engines with more time (40 moves in 40 minutes minimum).
My proposal would be that at least 20 engines (better some more) should be included, every GUI should be allowed and maybe we can have a mix of own books, Nunn, Noomen or other positions.
...slower computers add a bit of time and faster ones give a bit less, maybe that way that all are simulating a 2000 Mhz CPU.
Of course I am also open for alternative proposals. To decide the participants for instance every tester could give a list with preferred engines via email to the others and those will be included that have the most votes.
There must not be a chief organizer, but when anyone wants to do that job, it will be welcome.
I have few hopes that there will be a lot of interest, but it was worth a try.

A very good idea IMHO!

I don't care about the GUI as long as no engine is handicapped.

Ack.

I would prefer engines actively under developement but this is not an exclusive condition.
....
I nominate ... Heinz van Kempen .
There are some conditions to consider. RR or Swiss tournament, availability of CPU time (I can run tournament games in the night only for example) and so on.
...
I would like to participate but will be away for some days next week.
Volker

[Olivier Deville]

Geschrieben von:/Posted by: Olivier Deville at 15 July 2004 12:50:20:
Als Antwort auf:/In reply to: just a try geschrieben von:/posted by: Heinz van Kempen at 15 July 2004 10:41:16:

Hi ,
although I know that there a already a lot of interesting tournaments I just wanted to ask if there are other testers willing to combine their efforts and CPU power to run a really big tournament over a few weeks or longer for the best amateur engines with more time (40 moves in 40 minutes minimum).
My proposal would be that at least 20 engines (better some more) should be included, every GUI should be allowed and maybe we can have a mix of own books, Nunn, Noomen or other positions. All results combined should give the overall crosstable. Identical should be the time given for each game, slower computers add a bit of time and faster ones give a bit less, maybe that way that all are simulating a 2000 Mhz CPU. Of course I am also open for alternative proposals. To decide the participants for instance every tester could give a list with preferred engines via email to the others and those will be included that have the most votes. Another fair method would be to run a qualify with more engines to choose those that will play the main event.
I can´t do this alone when wanting to have a statistically sufficient number of games, but I can add at least two of my computers when we start in two weeks.
There must not be a chief organizer, but when anyone wants to do that job, it will be welcome. All testers should interchange results and games, so that interim reports and download links can be given here or at the websites.
In my opinion that would be an opportunity to say thanks to all who are still giving us strong engines for free.
Comments here and emails to hvankempen@web.de are highly appreciated.
I have few hopes that there will be a lot of interest, but it was worth a try.
Best Regards
Heinz

Hello Heinz !
A very interesting project
My Athlon 64 is playing the games of ChessWar tournament night and day, but I have free CPU on my 2 other computers : a PIV 3G and an Athlon 1700+.
I agree with Volker concerning the tournament manager , but we will give our opinion, of course !
Olivier



ChessWar

[Heinz van Kempen]

Geschrieben von:/Posted by: Heinz van Kempen at 15 July 2004 14:17:21:
Als Antwort auf:/In reply to: Re: just a try geschrieben von:/posted by: Olivier Deville at 15 July 2004 12:50:20:

Hello Heinz !
A very interesting project
My Athlon 64 is playing the games of ChessWar tournament night and day, but I have free CPU on my 2 other computers : a PIV 3G and an Athlon 1700+.
I agree with Volker concerning the tournament manager , but we will give our opinion, of course !
Olivier

Hello Olivier,
this is good news. Anyway I would not like to be the tournament manager. I like more the idea that we interchange all games and results and periodically (once or twice a week) combine the results on our pages and of course post them here giving the winning percentages in the overall table in case that there is a different amount of games for some. Together with one or two nice games each time in pgn. We will not be able to finish all at the same time.
For further proceeding (choice of participants or qualify, etc.) we should interchange mails that go to all testers interested and participating.
Thanks a lot for your interest
Heinz

[Heinz van Kempen]

Geschrieben von:/Posted by: Heinz van Kempen at 15 July 2004 14:26:57:
Als Antwort auf:/In reply to: Re: just a try geschrieben von:/posted by: Volker Pittlik at 15 July 2004 12:10:42:

Hello Volker

A very good idea IMHO!>>
I don't care about the GUI as long as no engine is handicapped.
I would prefer engines actively under developement but this is not an exclusive condition.
....

There must not be a chief organizer, but when anyone wants to do that job, it will be welcome.

I nominate ... Heinz van Kempen >I have few hopes that there will be a lot of interest, but it was worth a try.
I would like to participate but will be away for some days next week.
Volker

I was already thinking about that possibility for a long time. It is the only way to combine more time in games with a decent number of games, giving the event although statistical relevance.
I am always carefully checking ini.files, hash used etc.
Yes, I would support this. Engines should be under development. This seems more interesting than those that were not updated for years already, maybe with the exception of a free Ruffian version, to have at least the strongest in.
The tournament will need some preparation, so I propose that we only start in two weeks or so. This additionally gives the oppotunity to more testers to join.
Thanks for your interest
Heinz

[Igor Gorelikov]

Geschrieben von:/Posted by: Igor Gorelikov at 15 July 2004 15:06:24:
Als Antwort auf:/In reply to: just a try geschrieben von:/posted by: Heinz van Kempen at 15 July 2004 10:41:16:


IMHO, mix-up of different computers and conditions is bad.
If you want to arrange an overall event you need
- the very same hardware,
- the very same software,
- the very same conditions.
For the different computers, I see only one reasonable solution,
that is to run the same event
- under the very same conditions,
- with the same participants.
And then compare individual results between them and against overall.
Igor

[Uri Blass]

Geschrieben von:/Posted by: Uri Blass at 15 July 2004 15:21:10:
Als Antwort auf:/In reply to: just a try geschrieben von:/posted by: Heinz van Kempen at 15 July 2004 10:41:16:

Hi ,
although I know that there a already a lot of interesting tournaments I just wanted to ask if there are other testers willing to combine their efforts and CPU power to run a really big tournament over a few weeks or longer for the best amateur engines with more time (40 moves in 40 minutes minimum).
My proposal would be that at least 20 engines (better some more) should be included, every GUI should be allowed and maybe we can have a mix of own books, Nunn, Noomen or other positions. All results combined should give the overall crosstable. Identical should be the time given for each game, slower computers add a bit of time and faster ones give a bit less, maybe that way that all are simulating a 2000 Mhz CPU.

The problem is that different program do not have the same speed change from different processors.
It is possible that some engine is optimized for the PIV and is only 1.5 times faster when it is using A3000 when another program is optimized for A3000 and is 2 times faster when it is using A3000.
Uri

[Heinz van Kempen]

Geschrieben von:/Posted by: Heinz van Kempen at 15 July 2004 15:22:07:
Als Antwort auf:/In reply to: another try geschrieben von:/posted by: Igor Gorelikov at 15 July 2004 15:06:24:

IMHO, mix-up of different computers and conditions is bad.
If you want to arrange an overall event you need
- the very same hardware,
- the very same software,
- the very same conditions.
For the different computers, I see only one reasonable solution,
that is to run the same event
- under the very same conditions,
- with the same participants.
And then compare individual results between them and against overall.
Igor

Hello Igor,
to be very strict you are correct with all those conditions. Anyway I think that results will be comparable within the usual statistical error margins.
And wanting to be too accurate is too pedantic for such a project and will make it finally impossible. No one of us will have for example the same hardware, so only time adaptations can be done.
Of course it will be done that way that results and exact conditions from any tester will be kept seperately with a crosstable. But I think an total score crosstable would also be interesting to see who are the best with "overall conditions". The most important thing here should be the almost the same time control.
There are a lot of testers using different hardware or making some adaptations and combine the results either in crosstables or from different tournaments with different hardware in rating lists.
Best Regards
Heinz

[Volker Annuss]

Geschrieben von:/Posted by: Volker Annuss at 15 July 2004 15:29:11:
Als Antwort auf:/In reply to: just a try geschrieben von:/posted by: Heinz van Kempen at 15 July 2004 10:41:16:

Hi Heinz,
that sounds interesting. My PIV 2.66GHz is idle some days (and nights) per week. Sometimes is is very busy for two or three days for my own tests.
I would like to test some weaker engines, that do not get so much public interest, but its your project, so you decide who will play.
I don't like engines that require Installation of many megabytes of aditional software that might affect my system stability, especially the .net runtime system. There is no problem with cygwin.dll.
I will definitely not test my own engine in a public tournament because that may be regarded as unfair.
Volker

[Heinz van Kempen]

Geschrieben von:/Posted by: Heinz van Kempen at 15 July 2004 15:30:13:
Als Antwort auf:/In reply to: Re: just a try geschrieben von:/posted by: Uri Blass at 15 July 2004 15:21:10:

The problem is that different program do not have the same speed change from different processors.
It is possible that some engine is optimized for the PIV and is only 1.5 times faster when it is using A3000 when another program is optimized for A3000 and is 2 times faster when it is using A3000.
Uri

Hello Uri,
this is also correct. But engine tournaments will never have the accuracy of a scientific experiment. The idea also is to have a lot of fun, to play games at a high level and to say "thank you all for your interesting engines giving us a nice hobby and many hours full of suspense". And I think on this level many will replay the games and some authors will get hints to improve their engines even more.
Best Regards
Heinz

[Heinz van Kempen]

Geschrieben von:/Posted by: Heinz van Kempen at 15 July 2004 15:38:28:
Als Antwort auf:/In reply to: Re: just a try geschrieben von:/posted by: Volker Annuss at 15 July 2004 15:29:11:

Hi Heinz,
that sounds interesting. My PIV 2.66GHz is idle some days (and nights) per week. Sometimes is is very busy for two or three days for my own tests.
I would like to test some weaker engines, that do not get so much public interest, but its your project, so you decide who will play.
I don't like engines that require Installation of many megabytes of aditional software that might affect my system stability, especially the .net runtime system. There is no problem with cygwin.dll.
I will definitely not test my own engine in a public tournament because that may be regarded as unfair.
Volker

Hi Volker,
it would also be interesting to have a League for weaker engines. So if enough testers are participating and support this second tournament more authors will receive interesting games and I am convinced that programmers of weaker engines work as hard and ambitiously to improve as all others.
Authors who are running tournaments should exclude their own engine, as you stated correctly. Their engine can still participate, only that those games will be run by another tester.
Best Regards
Heinz

[Igor Gorelikov]

Geschrieben von:/Posted by: Igor Gorelikov at 15 July 2004 15:44:01:
Als Antwort auf:/In reply to: Re: another try geschrieben von:/posted by: Heinz van Kempen at 15 July 2004 15:22:07:

to be very strict you are correct with all those conditions. Anyway I think that results will be comparable within the usual statistical error margins.
And wanting to be too accurate is too pedantic for such a project and will make it finally impossible.
No one of us will have for example the same hardware, so only time adaptations can be done.
Of course it will be done that way that results and exact conditions from any tester will be kept seperately with a crosstable. But I think an total score crosstable would also be interesting to see who are the best with "overall conditions". The most important thing here should be the almost the same time control.
There are a lot of testers using different hardware or making some adaptations and combine the results either in crosstables or from different tournaments with different hardware in rating lists.

No, a single error in engine settings can have great influence.
- Yes, it's so.
But I can repeat once more with small correction (settings instead of conditions):
"For the different computers, I see only one reasonable solution,
that is to run the same event
- with the very same engine settings,
- with the same participants.
And then compare individual results between them and against overall."
- The most important thing here should be to keep the same time control and engine settings.
What a mess! I don't like it.
Igor

[Heinz van Kempen]

Geschrieben von:/Posted by: Heinz van Kempen at 15 July 2004 15:59:08:
Als Antwort auf:/In reply to: Re: another try geschrieben von:/posted by: Igor Gorelikov at 15 July 2004 15:44:01:

But I can repeat once more with small correction (settings instead of conditions):
"For the different computers, I see only one reasonable solution,
that is to run the same event
- with the very same engine settings,
- with the same participants.

Hello Igor,
this conditions should be fulfilled. Same participants, same engine versions, no updates allowed during the tournament. Same settings for hash and ini.files.
Anyway everyone can judge for himself if the tournament is worth a try. It is not the WCCC. It is only a tournament with a lot of games with more time for quality. It shall be interesting to follow, but will not pretend to be the ultimate wisdom.
Best Regards
Heinz

[Eric Oldre]

Geschrieben von:/Posted by: Eric Oldre at 15. July 2004 16:03:
Als Antwort auf:/In reply to: Re: just a try geschrieben von:/posted by: Olivier Deville at 15 July 2004 12:50:20:

My Athlon 64 is playing the games of ChessWar tournament night and day, but I have free CPU on my 2 other computers : a PIV 3G and an Athlon 1700+.

I agree with Volker concerning the tournament manager , but we will give our opinion, of course !
Olivier

Olivier,
I've been working very hard on trying to get a new engine created in time to have it be ready for the chesswar tounament you are starting after this one finishes. It's already a bit stronger than murderhole was but there is still much work to be done.
I didn't know that you ran ChessWar on an Athlon 64. Does that mean that it would be possible to use a engine in ChessWar that is compilied for 64-bit use?



http://www.oldre.com/Murderhole

[Igor Gorelikov]

Geschrieben von:/Posted by: Igor Gorelikov at 15 July 2004 16:25:50:
Als Antwort auf:/In reply to: Re: another try geschrieben von:/posted by: Heinz van Kempen at 15 July 2004 15:59:08:

But I can repeat once more with small correction (settings instead of conditions):
"For the different computers, I see only one reasonable solution,
that is to run the same event
- with the very same engine settings,
- with the same participants.

Hello Igor,
this conditions should be fulfilled. Same participants, same engine versions, no updates allowed during the tournament. Same settings for hash and ini.files.
Anyway everyone can judge for himself if the tournament is worth a try. It is not the WCCC. It is only a tournament with a lot of games with more time for quality. It shall be interesting to follow, but will not pretend to be the ultimate wisdom.
Best Regards
Heinz

I would like to participate under above conditions.
My handshake and suggestions:
1. Make you the project coordinator.
2. Firstly, work out the list of chess engines - participants. I vote for 10 participants (top engines) for a start.
3. Secondly, specify settings for each engine.
Best regards,
Igor

[Slobodan R. Stojanovic]

Geschrieben von:/Posted by: Slobodan R. Stojanovic at 15 July 2004 16:37:13:
Als Antwort auf:/In reply to: just a try geschrieben von:/posted by: Heinz van Kempen at 15 July 2004 10:41:16:

The idea is good, but I think, before starting with its realization, we should:

a) define 3, 5, 7, ot 9 tournaments patterns, then
b) the way how to convert hardware differences to the same tournament patterns.
All these done, we could start our tournamnets, exchanging PGN files, and creating i big database which could be used by all, inclusivly for ratings calculations.
So, we need a lot of work for standardization before starting with your idea, IMHO.
SL - chess philosopher

Hi ,
although I know that there a already a lot of interesting tournaments I just wanted to ask if there are other testers willing to combine their efforts and CPU power to run a really big tournament over a few weeks or longer for the best amateur engines with more time (40 moves in 40 minutes minimum).
My proposal would be that at least 20 engines (better some more) should be included, every GUI should be allowed and maybe we can have a mix of own books, Nunn, Noomen or other positions. All results combined should give the overall crosstable. Identical should be the time given for each game, slower computers add a bit of time and faster ones give a bit less, maybe that way that all are simulating a 2000 Mhz CPU. Of course I am also open for alternative proposals. To decide the participants for instance every tester could give a list with preferred engines via email to the others and those will be included that have the most votes. Another fair method would be to run a qualify with more engines to choose those that will play the main event.
I can´t do this alone when wanting to have a statistically sufficient number of games, but I can add at least two of my computers when we start in two weeks.
There must not be a chief organizer, but when anyone wants to do that job, it will be welcome. All testers should interchange results and games, so that interim reports and download links can be given here or at the websites.
In my opinion that would be an opportunity to say thanks to all who are still giving us strong engines for free.
Comments here and emails to hvankempen@web.de are highly appreciated.
I have few hopes that there will be a lot of interest, but it was worth a try.
Best Regards
Heinz

[Olivier Deville]

Geschrieben von:/Posted by: Olivier Deville at 15 July 2004 16:39:29:
Als Antwort auf:/In reply to: Re: just a try geschrieben von:/posted by: Slobodan R. Stojanovic at 15 July 2004 16:37:13:

The idea is good, but I think, before starting with its realization, we should:

a) define 3, 5, 7, ot 9 tournaments patterns, then
b) the way how to convert hardware differences to the same tournament patterns.
All these done, we could start our tournamnets, exchanging PGN files, and creating i big database which could be used by all, inclusivly for ratings calculations.
So, we need a lot of work for standardization before starting with your idea, IMHO.
SL - chess philosopher

Hi ,
although I know that there a already a lot of interesting tournaments I just wanted to ask if there are other testers willing to combine their efforts and CPU power to run a really big tournament over a few weeks or longer for the best amateur engines with more time (40 moves in 40 minutes minimum).
My proposal would be that at least 20 engines (better some more) should be included, every GUI should be allowed and maybe we can have a mix of own books, Nunn, Noomen or other positions. All results combined should give the overall crosstable. Identical should be the time given for each game, slower computers add a bit of time and faster ones give a bit less, maybe that way that all are simulating a 2000 Mhz CPU. Of course I am also open for alternative proposals. To decide the participants for instance every tester could give a list with preferred engines via email to the others and those will be included that have the most votes. Another fair method would be to run a qualify with more engines to choose those that will play the main event.
I can´t do this alone when wanting to have a statistically sufficient number of games, but I can add at least two of my computers when we start in two weeks.
There must not be a chief organizer, but when anyone wants to do that job, it will be welcome. All testers should interchange results and games, so that interim reports and download links can be given here or at the websites.
In my opinion that would be an opportunity to say thanks to all who are still giving us strong engines for free.
Comments here and emails to hvankempen@web.de are highly appreciated.
I have few hopes that there will be a lot of interest, but it was worth a try.
Best Regards
Heinz

Axon benchmark could be used in order to compare hardware.
Olivier



ChessWar

[Heinz van Kempen]

Geschrieben von:/Posted by: Heinz van Kempen at 15 July 2004 16:43:01:
Als Antwort auf:/In reply to: Re: another try geschrieben von:/posted by: Igor Gorelikov at 15 July 2004 16:25:50:

I would like to participate under above conditions.
My handshake and suggestions:
1. Make you the project coordinator.
2. Firstly, work out the list of chess engines - participants. I vote for 10 participants (top engines) for a start.
3. Secondly, specify settings for each engine.
Best regards,
Igor

Hello Igor,
fine that you also want to participate. We need experienced testers. I can coordinate but do not want to make all the decisions, especially do not want to carry the burden of having to exclude a lot of fine engines that also deserve to participate. Maybe a qualify would be the best and fairest solution, although it is time consuming.
I would like to collect all email addresses from testers, that are not available here, so all interested up to now and all those who come later, please write one email to hvankempen@web.de. There will be a democratic decision how to proceed concerning qualify and/or choice of participants. So maybe you can all write down a list of favourite engines in the mail.
For settings I would make the following proposal:
40 moves in 40 minutes on a 2Ghz machine
64 MB Hash
All other proposals for settings that are considered to be optimal from the authors themselves.
Here also other proposals are welcome and we can make a voting from all testers if there are different ones concerning time control for instance.
Ten engines is not really a lot and what I would consider a big tournament. When getting more than 5 testers, more engines should be able to participate.
Best Regards
Heinz

[Heinz van Kempen]

Geschrieben von:/Posted by: Heinz van Kempen at 15 July 2004 16:50:57:
Als Antwort auf:/In reply to: Re: just a try geschrieben von:/posted by: Slobodan R. Stojanovic at 15 July 2004 16:37:13:

The idea is good, but I think, before starting with its realization, we should:

a) define 3, 5, 7, ot 9 tournaments patterns, then
b) the way how to convert hardware differences to the same tournament patterns.
All these done, we could start our tournamnets, exchanging PGN files, and creating i big database which could be used by all, inclusivly for ratings calculations.
So, we need a lot of work for standardization before starting with your idea, IMHO.
SL - chess philosopher

Hi Slobadan,
I think that it should be a round robin event with always the same participants and engine versions, where every engines will run the same amount of games against any other.
Creating a big database available for download and a rating list or different ones in the end is also a nice idea.
For hardware differences we could use Axon Benchmark as Olivier proposes.
If you want to participate please write also to hvankempen@web.de and we can discuss all your proposals.
Best Regards
Heinz

[Heinz van Kempen]

Geschrieben von:/Posted by: Heinz van Kempen at 15 July 2004 17:07:37:
Als Antwort auf:/In reply to: Re: just a try geschrieben von:/posted by: Heinz van Kempen at 15 July 2004 16:50:57: