New engine rating system : the Lars Scale of Lousyness

Everything what does not fit in the other forums. Chess related or not, trivial or sophisticated, but keep it civilized and respect others please.

Moderator: Andres Valverde

New engine rating system : the Lars Scale of Lousyness

Postby JuLieN » 05 Feb 2011, 14:09

During a massive brainstorming on OpenWar's chat, it appeared to us that a more accurate and realistic scale could be defined to rate chess engines, that would be a lot more handy than the now aging Elo scale, that has proven to not be able to grasp emotional and psychological states of mind of the testers and users.

We call it the Lars Scale of Lousyness (or LSL). The idea being that the more sucky the engine, the more Lars points it receives. The method is still in development and opened for discussion, but here are some elements to take into consideration :

[*] The strength, in Elo. This should give up to 1000 Lars points (for an engine being under 1000 Elo) and at least 1 Lars (Engine being >3000 Elo)
[*] The Engine crashes : +200 Lars
[*] The Engine doesn't start : + 500 Lars
[*] The engine has no opening book : + 100 Lars
[*] The engine is a proven clone : +100 Lars
[*] The engine is suspected of being a clone : + 50 Lars (1)
[*] The engine isn't swedish : + 10 Lars
[*] The engine lacks important functions : + 20 Lars by function
[*] Time loss in 1min game : + 5 Lars (2)
[*] The engine doesn't run on a Pentium I : + 50 Lars (3)
[*] Displays wrong version number : + 30 Lars (4)
[*] Not WB 3 compliant : + 15 Lars (2)
[*] Doesn't play variants : + 25 Lars (2)
[*] Play illegal moves : + 55 Lars
[*] Can only play as white : + 35 Lars (2)
[*] Can only play as black : + 35 Lars (2)
[*] Can't give Mate in One : + 25 Lars (4)
[*] Don't exit memory : +15 Lars (2)
[*] Steals opponent's time : +20 Lars (2)
[*] The author is unknown : +15 Lars
[*] Obfuscates informations (nps, depth...) : +30 Lars
[*] Engine not publicly released : +25 Lars (5)
[*] Engine is too memory hungry : +35 Lars
[*] Newest version is weakier than the older one : +17 Lars (6)
[*] The engine is coded in Java : +45 Lars (5)
[*] Lars told you your engine is lousy : +51 Lars (2)
[*] Who the heck is Lars ? : +15 Lars

Features to have no to get +20 Lars/each :
[*] SMP
[*] Multi-PV
[*] bitbases
[*] Both UCI & Wb compatible
[*] Ponder mode

Also, the Lars scale should be easily convertible to the Olivier Scale of Wonders (or OSW).

This project will give birth to an open-sourced Lars Calculator under GPL.

The project is intended to replace the now deliquescent Elo scale and is still in very early stage of development, so we won't insist enough on how important it is for you to contribute now by any useful (or not) suggestion.

(1) all engines start by receiving this Lars bonus, until proven not guilty
(2) Andrew Fan Contribution
(3) Fonzy Bluemers contribution
(4) Olivier Deville contribution
(5) Lars Hallerström contribution
(6) Tony Mokonen Contribution
Last edited by JuLieN on 06 Feb 2011, 22:14, edited 11 times in total.
Why do always those who are wrong disagree with me?
User avatar
JuLieN
 
Posts: 61
Joined: 07 Jun 2008, 09:16
Location: Nantes (France)

Re: New engine rating system : the Lars Scale of Lousyness

Postby F. Bluemers » 05 Feb 2011, 16:00

Remember that engines should also be able to run on a Pentium I, if not it's good for another 50 Lars points :D
F. Bluemers
 
Posts: 175
Joined: 04 Sep 2008, 16:56
Location: Netherlands

Re: New engine rating system : the Lars Scale of Lousyness

Postby JuLieN » 05 Feb 2011, 17:49

A first, rudimentary, version of the Lars Points Calculator has been published (look at this thread).

Some things will be fixed with next versions :

[*] Points for playing illegal moves
[*] Points for checking both "proven clone" and "unproven clone" boxes
[*] Exiting memory :) (the calculator received a +300 Lars Points for that! )
Why do always those who are wrong disagree with me?
User avatar
JuLieN
 
Posts: 61
Joined: 07 Jun 2008, 09:16
Location: Nantes (France)

Re: New engine rating system : the Lars Scale of Lousyness

Postby JuLieN » 05 Feb 2011, 20:53

A second release, was published : see this thread.

Changes list :
- fixed the "remains in memory" lousyness feature
- added 11 new terms to the evaluation and two secret bonuses

Another idea for a next version :
- Engine not publicly released : +25 Lars
Why do always those who are wrong disagree with me?
User avatar
JuLieN
 
Posts: 61
Joined: 07 Jun 2008, 09:16
Location: Nantes (France)

Re: New engine rating system : the Lars Scale of Lousyness

Postby JuLieN » 06 Feb 2011, 13:04

Third version released : see this thread.

Changes list :
- added more terms to the evaluation
- fixed evaluation bugs for supporting features (1)
- cosmetic improvement (centering texts)
- compressed the fat exe and divided its size by two

(1) sorry for this bug : if you larsed your engine with an older version of the calculator, you got a lousy lousy rank.
Why do always those who are wrong disagree with me?
User avatar
JuLieN
 
Posts: 61
Joined: 07 Jun 2008, 09:16
Location: Nantes (France)

Re: New engine rating system : the Lars Scale of Lousyness

Postby JuLieN » 06 Feb 2011, 14:45

Fourth version of the amazing Lars Scale Points Calculator released.

Changes list :
[*] added more terms to the evaluation
[*] reversed the Lars points for "engines crashes" to those of "engine doesn't start" and vice-versa
[*] fixed some typoes

Go get it in this thread.
Why do always those who are wrong disagree with me?
User avatar
JuLieN
 
Posts: 61
Joined: 07 Jun 2008, 09:16
Location: Nantes (France)


Return to Anything else

Who is online

Users browsing this forum: No registered users and 1 guest

cron