At first glance some of the character tiers seem bonkers - especially for the online versions of games. And indeed, they seem crazy to me too - but that's what the numbers say.
But first, how, specifically, are they ranked? You may notice that the ranks do not correlate exactly with win rate. Instead, I construct a score interval - specifically, the Wilson score interval for binomial distributions. You can google more about the specifics, but you can think of it as a adjusted type of probability interval.
So let's say a character has a 60% winrate with a 1% pick rate. Is that character better or worse than one with a 55% winrate and a 5% pick rate? Isn't it somewhat unfair that the 55% one is worse, even though it has such a higher sample size?
That's what using the score interval controls for. Specifically, the lower bound of the score interval. That way outliers have a harder time skyrocketing to the top. After all, if MkLeo plays some random character twice and won both times, does that mean that they're the best character? Probably not.
Then, after the characters are ranked, they are tiered simply based on percentile cutoffs. That may be a bit overly simplistic - after all, it means that it becomes zero sum when maybe it shouldn't be. The S-tier is always the same size in this model.
Like with any metric, the ranking algorithm is not perfect. But really, it can't be perfect, and I think the current one, which is often used by sites in other games, is interesting on its own. After all, why is Lucina S tier in LAN but has a 40% winrate and D tier in Online?
For LAN games, it is mostly what you'd expect. In Online, it's a very strange world - but also one that's more rigorous, in some ways, since online data is rich in character and stage details. This means that LAN results are heavily biased by elite players who main them, while that is not true of online results. They are more honest, average results across the playerbase.
If you have any thoughts or suggestions, feel free to contact me at email@example.com.