about testing 178 and 178d

[Uri Blass]

Geschrieben von:/Posted by: Uri Blass at 06 April 2004 20:56:29:

I suspect that 178d is better at longer time control but I admit that I know nothing and everything can be a statistical noise.
I asked Leo to use 178d and not 178 after my bad results with 178d.
I tested 178d against 174 and not against 178 but as far as I know there is no difference between 174 and 178 except the fact that 178 is bearly 2% faster.
results that I have in match between 178d and 174 on A1000(ponder off)
1)178d lost 30-20 in the nunn2 match at 4 minutes/40 moves.
2)178d lost 11.5-8.5 in the nunn1 match at 12 minutes/40 moves
3)178d is leading 11.5-8.5 in the nunn2 match at 40 minutes/40 moves.
Note that in the first 70 games that were played at faster time control I have only one case when 178 was the leader in a sequence of 20 consecutive games.
games 1-20 11.5-8.5 for 174
games 2-21 11-9 for 174
games 3-22 10.5-9.5 for 174
games 4-23 11.5-8.5 for 174
games 5-24 12-8 for 174
games 6-25 12.5-7.5 for 174
games 7-26 13.5-6.5 for 174
games 8-27 14.5-5.5 for 174
games 9-28 15-5 for 174
games 10-29 14.5-5.5 for 174
games 11-30 14-6 for 174
games 12-31 14-6 for 174
games 13-32 13.5-6.5 for 174
games 14-33 13.5-6.5 for 174
games 15-34 13-7 for 174
games 16-35 13.5-6.5 for 174
games 17-36 13.5-6.5 for 174
games 18-37 14.5-5.5 for 174
games 19-38 14.5-5.5 for 174
games 20-39 14.5-5.5 for 174
games 21-40 14-6 for 174
games 22-41 14-6 for 174
174 scored 3 points in games 42-48 so 174 was clearly the leader before game 49.
games 30-49 12-8 for 174
games 31-50 12-8 for 174
games 32-51 11.5-8.5 for 174
games 33-52 12-8 for 174
games 34-53 11-9 for 174
games 35-54 11-9 for 174
games 36-55 11-9 for 174
games 37-56 11.5-8.5 for 174
games 38-57 11-9 for 174
games 39-58 10.5-9.5 for 174
games 40-59 10.5-9.5 for 178!!
games 41-60 10-10
games 42-61 10-10
games 43-62 10.5-9.5 for 174
games 44-63 11-9 for 174
games 45-64 11.5-8.5 for 174
games 46-65 10.5-9.5 for 174
games 47-66 10-10
games 48-67 10.5-9.5 for 174
games 49-68 10.5-9.5 for 174
games 50-69 11.5-8.5 for 174
games 51-70 11.5-8.5 for 174

The result of these tests is that now I asked Leo to use 178d and not 178(if the rules still allow it) but the only thing that I am sure about is that I know nothing and I may change my mind when I get more information.
Uri

[Joachim Rang]

Geschrieben von:/Posted by: Joachim Rang at 06 April 2004 21:43:06:
Als Antwort auf:/In reply to: about testing 178 and 178d geschrieben von:/posted by: Uri Blass at 06 April 2004 20:56:29:

I suspect that 178d is better at longer time control but I admit that I know nothing and everything can be a statistical noise.
I asked Leo to use 178d and not 178 after my bad results with 178d.
I tested 178d against 174 and not against 178 but as far as I know there is no difference between 174 and 178 except the fact that 178 is bearly 2% faster.
results that I have in match between 178d and 174 on A1000(ponder off)
1)178d lost 30-20 in the nunn2 match at 4 minutes/40 moves.
2)178d lost 11.5-8.5 in the nunn1 match at 12 minutes/40 moves
3)178d is leading 11.5-8.5 in the nunn2 match at 40 minutes/40 moves.
Note that in the first 70 games that were played at faster time control I have only one case when 178 was the leader in a sequence of 20 consecutive games.
games 1-20 11.5-8.5 for 174
games 2-21 11-9 for 174
games 3-22 10.5-9.5 for 174
games 4-23 11.5-8.5 for 174
games 5-24 12-8 for 174
games 6-25 12.5-7.5 for 174
games 7-26 13.5-6.5 for 174
games 8-27 14.5-5.5 for 174
games 9-28 15-5 for 174
games 10-29 14.5-5.5 for 174
games 11-30 14-6 for 174
games 12-31 14-6 for 174
games 13-32 13.5-6.5 for 174
games 14-33 13.5-6.5 for 174
games 15-34 13-7 for 174
games 16-35 13.5-6.5 for 174
games 17-36 13.5-6.5 for 174
games 18-37 14.5-5.5 for 174
games 19-38 14.5-5.5 for 174
games 20-39 14.5-5.5 for 174
games 21-40 14-6 for 174
games 22-41 14-6 for 174
174 scored 3 points in games 42-48 so 174 was clearly the leader before game 49.
games 30-49 12-8 for 174
games 31-50 12-8 for 174
games 32-51 11.5-8.5 for 174
games 33-52 12-8 for 174
games 34-53 11-9 for 174
games 35-54 11-9 for 174
games 36-55 11-9 for 174
games 37-56 11.5-8.5 for 174
games 38-57 11-9 for 174
games 39-58 10.5-9.5 for 174
games 40-59 10.5-9.5 for 178!!
games 41-60 10-10
games 42-61 10-10
games 43-62 10.5-9.5 for 174
games 44-63 11-9 for 174
games 45-64 11.5-8.5 for 174
games 46-65 10.5-9.5 for 174
games 47-66 10-10
games 48-67 10.5-9.5 for 174
games 49-68 10.5-9.5 for 174
games 50-69 11.5-8.5 for 174
games 51-70 11.5-8.5 for 174

The result of these tests is that now I asked Leo to use 178d and not 178(if the rules still allow it) but the only thing that I am sure about is that I know nothing and I may change my mind when I get more information.
Uri

you are asking Leo to use 178d although it scored worse in 70 games, because it's narrowly leading in 20 games?
It's interesting to see your struggle to get a decision, which version will do better but I don't understand your problems.
Dy you have good reason (I don't mean games, but programming differences) to believe, that 178d performs relatively better on longer time controls than 174?
Did you change your extensions/reductions significantly? If not I'd rather take the risk, that they perform unequal at different time controls but would base my decision on a number of games with statistical significance: run two gauntlets against 10 opponnents with the nunn2-positons one with 178d and one with 178 at 2/1. This would take you 4 to 5 days but then you would have statistical conclusive data which version will be stronger.
regards Joachim

[Uri Blass]

Geschrieben von:/Posted by: Uri Blass at 06 April 2004 22:55:33:
Als Antwort auf:/In reply to: Re: about testing 178 and 178d geschrieben von:/posted by: Joachim Rang at 06 April 2004 21:43:06:

I suspect that 178d is better at longer time control but I admit that I know nothing and everything can be a statistical noise.
I asked Leo to use 178d and not 178 after my bad results with 178d.
I tested 178d against 174 and not against 178 but as far as I know there is no difference between 174 and 178 except the fact that 178 is bearly 2% faster.
results that I have in match between 178d and 174 on A1000(ponder off)
1)178d lost 30-20 in the nunn2 match at 4 minutes/40 moves.
2)178d lost 11.5-8.5 in the nunn1 match at 12 minutes/40 moves
3)178d is leading 11.5-8.5 in the nunn2 match at 40 minutes/40 moves.
Note that in the first 70 games that were played at faster time control I have only one case when 178 was the leader in a sequence of 20 consecutive games.
games 1-20 11.5-8.5 for 174
games 2-21 11-9 for 174
games 3-22 10.5-9.5 for 174
games 4-23 11.5-8.5 for 174
games 5-24 12-8 for 174
games 6-25 12.5-7.5 for 174
games 7-26 13.5-6.5 for 174
games 8-27 14.5-5.5 for 174
games 9-28 15-5 for 174
games 10-29 14.5-5.5 for 174
games 11-30 14-6 for 174
games 12-31 14-6 for 174
games 13-32 13.5-6.5 for 174
games 14-33 13.5-6.5 for 174
games 15-34 13-7 for 174
games 16-35 13.5-6.5 for 174
games 17-36 13.5-6.5 for 174
games 18-37 14.5-5.5 for 174
games 19-38 14.5-5.5 for 174
games 20-39 14.5-5.5 for 174
games 21-40 14-6 for 174
games 22-41 14-6 for 174
174 scored 3 points in games 42-48 so 174 was clearly the leader before game 49.
games 30-49 12-8 for 174
games 31-50 12-8 for 174
games 32-51 11.5-8.5 for 174
games 33-52 12-8 for 174
games 34-53 11-9 for 174
games 35-54 11-9 for 174
games 36-55 11-9 for 174
games 37-56 11.5-8.5 for 174
games 38-57 11-9 for 174
games 39-58 10.5-9.5 for 174
games 40-59 10.5-9.5 for 178!!
games 41-60 10-10
games 42-61 10-10
games 43-62 10.5-9.5 for 174
games 44-63 11-9 for 174
games 45-64 11.5-8.5 for 174
games 46-65 10.5-9.5 for 174
games 47-66 10-10
games 48-67 10.5-9.5 for 174
games 49-68 10.5-9.5 for 174
games 50-69 11.5-8.5 for 174
games 51-70 11.5-8.5 for 174

The result of these tests is that now I asked Leo to use 178d and not 178(if the rules still allow it) but the only thing that I am sure about is that I know nothing and I may change my mind when I get more information.
Uri

you are asking Leo to use 178d although it scored worse in 70 games, because it's narrowly leading in 20 games?
It's interesting to see your struggle to get a decision, which version will do better but I don't understand your problems.
Dy you have good reason (I don't mean games, but programming differences) to believe, that 178d performs relatively better on longer time controls than 174?
Did you change your extensions/reductions significantly?
regards Joachim

The 20 games were at slower time control and I also got from a tester
results that supported 178d after his first results supported 178(if I do not count game against previous versions but only games against different programs than the total result is that 178d scored 1/2 point more than 178 against the same opponents after 30 games).
I also think that beating Ufim 3.5-0.5 is better than losing 3-1 against the baron inspite of knowing that Ufim is not equal strength to the baron.

The only difference between 178d and 178 is the evaluation.
I suspect that it is possible that reducing the value of pawns is better for long time control.
Some of my pruning are based on evaluation so they were slightly changed and reducing the value of pawns mean slightly less pruning.
I have 178c that beated 174 in the nunn2 match in my tests.
The only difference between 178c and 178d is slightly reducing the value of pawns(not by much and it is at most 0.03 pawns difference).
I see no reason to assume that it is not possible that a change in the evaluation can lead to negative results at slow time control and positive result at long time control.
The only thing that I can be sure about is that
I do not know.

If not I'd rather take the risk, that they perform unequal at different time controls but would base my decision on a number of games with statistical significance: run two gauntlets against 10 opponnents with the nunn2-positons one with 178d and one with 178 at 2/1. This would take you 4 to 5 days but then you would have statistical conclusive data which version will be stronger.
I will only know which version is stronger at 2/1 and not which version is stronger at 40/40
Uri