Elo Coaster Ranking Project

[Burniel]

Hello everyone!

I've been spending a lot of time trying to devise the best possible method of compiling a modern-day Mitch Hawker style coaster poll. I read this thread with great interest, and was disappointed to see that it seemingly didn't come to anything in the end. I've therefore designed a system which takes heavy inspiration from both Mitch Hawker and the ideas expressed in that thread, which I believe (and testing has proved) is quite possibly the closest we're going to get to a fair and relatively easy to use system.

The system is based on Elo and, like Mitch Hawker, is pair-based, such that a certain number of Elo points are transferred every time a coaster is ranked above another in someone's list. The advantages to the Elo system is that it doesn't matter how many times a coaster is submitted, nor how many coasters each person submits, as every coaster and viewpoint has a chance.

The plan is for people to submit their top x coasters (with x being whatever the user desires, provided it's at least 15). Standard procedures to keep the list tidy will be in place, such as kiddie coasters are out, clones are only listed once, etc. The list is a combination of wood and steel. Coasters won't be included in the formal list until they reach five riders. In an attempt to make this as user-friendly as possible, if you don't want to have to filter out clones etc, just send me a copy of list/spreadsheet/whatever and I'm happy to sort it.

Obviously, those who have a very large number of credits may not wish to rank them all from start to end. Therefore, after a list of your top x coasters, please simply assign the rest of your coasters a category from the following:

• Good, but didn't make my list.
• Okay, didn't do loads for me.
• Bad, a very unenjoyable experience, I hated it.

Submissions are accepted from now, and a closing date will be decided as and when necessary. Hopefully we can make something out of this!

-Dan

 

[HeartlineCoaster]

Could you elaborate more on the system for me, perhaps with a couple of examples?

Also a bit worried about the 'top x coasters' part, the issues of which I outlined in the thread you mentioned.
Probably gonna end in the same old backlash about not wanting to rank every single coaster in order, but there's ways around that.

 

[Burniel]

Could you elaborate more on the system for me, perhaps with a couple of examples?

Also a bit worried about the 'top x coasters' part, the issues of which I outlined in the thread you mentioned.
Probably gonna end in the same old backlash about not wanting to rank every single coaster in order, but there's ways around that.

Err ... ok.

It's done by a program I've designed which takes in a list in order along with their current elo scores. It then makes every possible pairing between the coasters in the list (without repeating itself), and essentially does a 1v1 for all of them. The Elo scoring system uses estimated results whereby the likelihood of each coaster winning is calculated based on the difference between their current scores, and compares the actual result to the estimate to see how "impressive" the win was, determining how many points should be awarded.

As an example: coaster 1 has 1000 Elo points (what they all start with), and so does coaster 2. Both coasters are estimated to have a 0.5 chance of winning. Coaster 1 is ranked higher and thus "wins". The amount of points that coaster 1 gains and coaster 2 loses is equal to (a constant)*(1-the estimation). The constant I'm using is 32, thus it's 32*0.5, or 16 points. Coaster 1 rises to 1016, coaster 2 falls to 984.

As another: Coaster 3 has 950 Elo points, while coaster 4 has 1050. Coaster 3 is estimated at 0.69 and coaster 4 at 0.31. Coaster 3 wins, and thus receives 32*(1-0.31) points, or roughly 22 points. Coaster 3 now has a score of 972, Coaster 4 has a score of 1028. The advantage of the system is that it, in theory, shouldn't matter too much how many times a coaster is submitted.

I believe the main worry about "top x coasters" is that a coaster may feature right near the top of some lists, and may not feature at all on others, but I'm not sure this is a problem. Let's say for example that coaster 5 is ranked near the top of the list of a load of people who have ridden <50 coasters, but doesn't even make the top 50 of someone who has ridden 1000, and so isn't entered by them. All it takes is for the person with a higher count (and arguably more knowledge) to have submitted a few coasters which feature alongside coaster 5 in most people's submissions, and the system should (to an extent) sort itself out. EG - If The Smiler is currently ranked below Nemesis, and you feature Nemesis in your top x but not The Smiler, then coasters you rank above Nemesis will, assuming other entries agree with you, eventually climb above both Nemesis and The Smiler. It's not perfect, but it certainly doesn't seem catastrophic. In any case, it would obviously be ideal if people submitted as many as possible.

Hopefully this clears some of it up.

 

[Pink Cadillac]

What happens regarding coasters that aren't open anymore?
Are we rating stuff like Winjas as one coaster?

 

[Burniel]

What happens regarding coasters that aren't open anymore?
Are we rating stuff like Winjas as one coaster?

Defunct coasters can be submitted, though it's entirely possible that some won't reach the minimum ridership.
Duelling coasters in general count as one, unless the two layouts have significant differences. I'd say Winjas would qualify as two though, seeing as they're different layouts and experiences.

 

[HeartlineCoaster]

I believe the main worry about "top x coasters" is that a coaster may feature right near the top of some lists, and may not feature at all on others, but I'm not sure this is a problem.

I do think this is a big problem for... what's that phrase being thrown around recently... divisive coasters.
You're missing out on so many comparisons if you assume that every 'well rated coaster' will be included in every list. Some people who have ridden just really don't care and don't mention it, when it should lose to maybe 50, 100 other coasters in a single list.
Would that many losses not have a huge impact on that ranking in your method of scoring?

All I suggest for this is that you ask for, in a user-friendly way, the 'top x coasters' and then everything else below that lumped into a few of categories of personal preference such as 'I didn't care for this' and 'I hated it'. It's easy enough for people to rate things to that extent and you aren't missing out on any data.

 

[Burniel]

All I suggest for this is that you ask for, in a user-friendly way, the 'top x coasters' and then everything else below that lumped into a few of categories of personal preference such as 'I didn't care for this' and 'I hated it'. It's easy enough for people to rate things to that extent and you aren't missing out on any data.

I like this idea, I've just never enjoyed the prospect of managing it. It's entirely possible though.

If we had a list (minimum 15) followed by "Good, but doesn't make my list", "Doesn't do anything for me", and "I hated it", that could work. In pairings, everything in the main list would get a victory over those in the categories, everything in the higher categories would gain victories over the lower categories and everything in the categories would get losses to those above them.

It's a little more effort for all parties, but I would agree that it'll be fairer. I shall edit the original post to reflect this new rule. Hopefully this encourages thorough entries.

 

[HeartlineCoaster]

Good man.

 

[Hyde]

This is all awesome, and I have for years wanted to create a better roller coaster poll over Golden Ticket, Mitch Hawker, etc. - hence my excitement in that original discussion!

I really like this system design. It's simple, easy to use, and would be easily replicable for anyone else wanting to emulate. The ongoing challenge of dealing with behemoth's that are mediocre yet very well ridden as @HeartlineCoaster mentioned is always present, but indeed there could be an easy way to account for mentionable roller coasters.

Count me in for helping design this - love creating system models like these.