Winboard Forum

by **Heinz van Kempen** » 25 Jul 2004, 03:03

Geschrieben von:/Posted by: Heinz van Kempen at 25 July 2004 04:03:38:

Hi all :-)

.
Brian Richardson (Tinker) kindly offers his help for our project.
We are all very happy that so many are interested. For every 4 or 5 new testers we can open a new tournament class. For the moment we are trying to build Rook class and also hope for a Knight Class with a few people more.
There are so many engines that deserve tournaments with more time.
Best Regards
AEGT group

by **Will Singleton** » 25 Jul 2004, 04:06

Geschrieben von:/Posted by: Will Singleton at 25 July 2004 05:06:07:
Als Antwort auf:/In reply to: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Heinz van Kempen at 25 July 2004 04:03:38:

Oohrah

by **Uri Blass** » 25 Jul 2004, 04:11

Geschrieben von:/Posted by: Uri Blass at 25 July 2004 05:11:28:
Als Antwort auf:/In reply to: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Heinz van Kempen at 25 July 2004 04:03:38:

I think that long time control games are not very important for the weak engines(rating

by **Heinz van Kempen** » 25 Jul 2004, 04:19

Geschrieben von:/Posted by: Heinz van Kempen at 25 July 2004 05:19:48:
Als Antwort auf:/In reply to: Re: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Uri Blass at 25 July 2004 05:11:28:

Hello Uri,
that might be a bit egoistic :-)

. I put this for discussion here and also want to know what authors of weaker engines think about this.
Best Regards
Heinz

by **Uri Blass** » 25 Jul 2004, 04:43

Geschrieben von:/Posted by: Uri Blass at 25 July 2004 05:43:46:
Als Antwort auf:/In reply to: Re: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Heinz van Kempen at 25 July 2004 05:19:48:

I think that even movei is still not at a level that is high enough to care much about long time control games and there are still improvement that I can make that will be productive at all time controls so even if you decide to do more testing only for engines with more than 2650 in order not to include movei then it is ok but I think that it may be interesting for readers to see how much quark earns from long time control games relative to blitz when for weaker engines than 2500 it is simply not interesting because I believe that there is no chance that engines with less than 2500 at blitz will be stronger than Crafty at 40/40 time control.
Uri

by **Will Singleton** » 25 Jul 2004, 04:44

Geschrieben von:/Posted by: Will Singleton at 25 July 2004 05:44:18:
Als Antwort auf:/In reply to: Re: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Uri Blass at 25 July 2004 05:11:28:

Weaker than Quark isn't interesting? You are eliminating a lot of engines here, arbitrarily. It's just possible that you ought to refrain from comments of this sort, unless you want to join the test group, and communicate your thoughts via email.
Will

by **Heinz van Kempen** » 25 Jul 2004, 04:48

Geschrieben von:/Posted by: Heinz van Kempen at 25 July 2004 05:48:38:
Als Antwort auf:/In reply to: Re: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Uri Blass at 25 July 2004 05:43:46:

Hello Uri,
I just think that programmers of weaker engines work as hard on their engines like the others. Most do it with heartblood and for me it is the same suspense to run weaker engines in tournament.
I really want that the work of as much authors as possible will be honoured a bit more and the strongest already have beta testers, much more tournaments and games.
Best Regards
Heinz

by **Uri Blass** » 25 Jul 2004, 05:05

Geschrieben von:/Posted by: Uri Blass at 25 July 2004 06:05:05:
Als Antwort auf:/In reply to: Re: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Heinz van Kempen at 25 July 2004 05:48:38:

The question is if they need long time control games.
I think that it may be more productive for them to test some versions at blitz
instead of one version at long time control.
I remember that Jos Carlos wanted testers to test which averno is best when he had many versions(I do not know if he found testers to help him at that job).
Uri

by **Roger Brown** » 25 Jul 2004, 05:08

Geschrieben von:/Posted by: Roger Brown at 25 July 2004 06:08:09:
Als Antwort auf:/In reply to: Re: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Uri Blass at 25 July 2004 05:11:28:

Well, well, is this the final say in these matters? I have to say that I am no expert but I have read repeatedly about the validity of blitz versus long timecontrol games. The quality tends to be higher with more time to think about the position - for human and silicon.
Test suite results have been heavily criticised by Dr. Robert Hyatt among others. It seems that it is possible to tune an engine for test suites and the improvement does not carry over to OTB play.
Incidenatlly Uri, your engine is so strong that I do not see how you can speak with such authority about weaker engines....
:-)

Thanks for sharing. Man, I know that in this business one has to have a hide of iron but could you at least wait until the thing starts before you pan it? You may be perfectly correct but don't you think the novelty of the idea alone is worth giving it the benefit of scientific doubt?
That may be on the cards but you know what? It has to be fun first and second and third and.....Data that you may find useful takes a backseat to that. We are doing this first and foremost because it fascinates us.
Scientific, useful data may emerge but I assure you, that will be only a side-effect. Well, I am being a little cute because the data should be good and interesting BUT the fun is the thing.
Incidentally, personally, I find the idea of testing learning on versus learning off a little, uninteresting.
Aha, you are more than welcome to join Uri. An extra machine would be welcome. Heinz has more ideas than are good for him. He may just explode!

Later.

by **Volker Boehm** » 25 Jul 2004, 09:03

Geschrieben von:/Posted by: Volker Boehm at 25 July 2004 10:03:04:
Als Antwort auf:/In reply to: Re: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Uri Blass at 25 July 2004 05:11:28:

Hi Uri,
partly you are right. For "weaker" engines like Spike and for engines that changes verry fast blitz games are usually good enough to test advancement.
You shouldn´t compare two Spike versions at long time control to find out witch is better - use blitz to get much more games improving the statistic correctness of the result.
On the other hand Spike never played a large amount of games with longer time control. Thus we are not sure if Spike has problems there. The newer version 0.6 we use for this tournament will last some weeks or even month as it gets harder to improve the playing strength.
I think the criteria for joining the tournament is not the strength of the engine, but the amount of games on longer time controls that has been played and the speed the engine changes.
Thus IMHO exclude engines that didn´t change for a year and engines that haven´t got stable versions while the tournament is running, but not weaker engines if the authors are interested in participating and you find testers that are interested too.
Greetings Volker

by **Mridul Muralidharan** » 25 Jul 2004, 09:45

Geschrieben von:/Posted by: Mridul Muralidharan at 25 July 2004 10:45:36:
Als Antwort auf:/In reply to: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Heinz van Kempen at 25 July 2004 04:03:38:

Hi,
Pardon my asking , but what exactly is the AEGT group ?
Where can I get more info about it ? What does it stand for ?
I dont keep myself updated as I used to ....
Thanks in advance
Mridul

by **Olivier Deville** » 25 Jul 2004, 10:51

Geschrieben von:/Posted by: Olivier Deville at 25 July 2004 11:51:32:
Als Antwort auf:/In reply to: Re: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Mridul Muralidharan at 25 July 2004 10:45:36:

Dear Mridul
AEGT stands for Amateur Engines Grand Test. It is a tournament where all testers put their CPUs in common. There are some rules at Igor Gorelikov's website at http://www.digichess.gr/infiniteloop/etc/AEGT_draft.txt.
There are several groups of 12 engines, playing a double round robin tournament at the time control of 40/40 (or a little more or less according to the hardware speed). The goal is to get many games at this time control for these engines.
If you have an idle computer and want to participate as a tester, you are very welcome . And if you have a new/private engine, you can ask the group to test it, if it is stable enough.

ChessWar

by **Thomas Mayer** » 25 Jul 2004, 12:42

Geschrieben von:/Posted by: Thomas Mayer at 25 July 2004 13:42:15:
Als Antwort auf:/In reply to: Re: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Uri Blass at 25 July 2004 06:05:05:

Hi Uri,

No, this is in fact not the question... The idea is simply to test as many engines as possible with a long time control and to get maybe statisticaly relevant results, something a single person can not do himself.
Also it's about having fun, getting contacts with other tester which also like long time controls.
But as a side effect all the programmers get very much stuff about their creation in long games. With Quark I have tested long time controls even at the beginning of it's life. There are some features in Quark that only work in long time controls, e.g. mobility calculation which costs around 40% of speed but definitely has it's pay-offs in long terms. In blitz those additional 40% would help more then the mobility.
thanks god the spirit of the group is not to be productive... At least not in an economic sense...
Greets, Thomas

by **Jose Carlos** » 26 Jul 2004, 01:29

Geschrieben von:/Posted by: Jose Carlos at 26 July 2004 02:29:41:
Als Antwort auf:/In reply to: Re: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Uri Blass at 25 July 2004 06:05:05:

Some people helped me, but I did most of the job myself, with about 15000 games. The results were truly surprising to me:
- I tested several search ideas with the same eval and all the rest the same. The 13 versions had a rating difference of ~50 points from the first to the last (I also included other programs in the test, for example one old version of Movei). I find incredible that one version with only null move and no prunung in QSearch performed almost identical to one with SEE, lazy cuts, futility in last two plies and in QSearch, Fail low reductions and many more things.
- The version that performed best was one with only null move in regular search, and all possible pruning in QSearch
- Later versions with some additions in other parts of the program didn't work the same, and versions with forward pruning in regular search performed better.
But it's been some weeks since I last used Averno at all, and probably I won't be able to continue until february or march, so the big misteries will remain unsolved to me...
José C.

by **Dan Honeycutt** » 26 Jul 2004, 07:12

Geschrieben von:/Posted by: Dan Honeycutt at 26 July 2004 08:12:10:
Als Antwort auf:/In reply to: Re: AEGT: Brian Richardson joins group of testers geschrieben von:/posted by: Roger Brown at 25 July 2004 06:08:09:

I'll offer my opinion.
Mine is a very green engine that needs a lot of tuning and testing. With limited time and resources I work on development using short time controls almost exclusively. I'm very interested to see how it fares at longer time control. I think Heinz has a great idea. My sincere thanks to him and all the AEGT team members.

Dan H.

Winboard Forum

Who is online