In my last blog post, I covered how I ended up calculating the comparative skill ratings of professional teams using an Elo-style rating system. In this post, I'll look at how it fared when it came to predicting the results of IPL 5.
Pre-IPL 5 Ratings
Here are the pre-IPL ratings of each participating team, along with a little commentary on their recent results heading into the competition.
2256 - Azubu Blaze - The favorites alongside WE. Had been winning most of their matches in Korea against strong teams. Recent NA LAN experience in Dallas crushing the competition (albeit with a substitute as their top lane Shy) and tore through the Korean IPL 5 qualifiers. However, this would be Flame's (their new top laner) first international LAN, and they were only 6-4 since their IPL 5 qualifiers win.
2239 - Team WE - Similar to Blaze, had been absolutely dominating their competition since the world finals, including wins over strong Korean and Chinese teams.
2042 - Taipei Assassins - The world champions didn't have much strong competition to prove themselves against since the world championships, so their rating stagnated while others rose.
2040 - Fnatic - Fnatic entered IPL 5 full of confidence, having won their last two tournaments, including the finals of DreamHack against CLG EU.
2038 - CLG Prime - Had been dominating the North American scene, and thus the inflated rating. But...it was the North American scene. Hadn't proved though that they could beat non-American powerhouses. This rating was the one that stuck out to me as highlighting one of the weaknesses that I discussed in my previous blog post: inflated ratings for teams that may have only played (and won against) teams from the same region.
2007 - CLG EU - Had looked a bit shaky since the World Championships, losing to Fnatic, Eclypsia and Meet Your Makers. By their own admission, they hadn't had much opportunity to practice.
1996 - Moscow Five - Like CLG EU, M5 had been underperforming since the world championships. However, recent show match wins against World Elite and Invictus Gaming and their secret preparation sessions in China were promising.
1920 - Curse NA - Won the last two TSM Series weekly tournaments, but had not placed particularly well at LANs.
1882 - Curse EU - The loss of their support compounded on their tournament troubles since their win at Tales of the Lane 3 weeks prior.
1835 - Team FeaR - Can always be counted on to give their fellow NA teams a run for their money, but LAN results had been lacking.
1794 - TSM - Had been in a major slump for a month, having lost a majority of their matches to fellow NA teams.
1749 - Meat Playground - Like TSM, boasted a losing record in the prior month. Recent wins against TSM and TD probably gave them a bit of confidence going into IPL 5.
1703 - Team Dynamic - Lost the form they displayed earlier in the year, and their rating took a nose dive.
1699 - IceLanD - A very rough guestimate for this rating based 1 win/2 losses against TPA and 2 losses against KT Rolster B.
1603 - Singapore Sentinels - The undisputed king of moving YouTube videos. Unfortunately, their competition in the GPL other than TPA were mediocre, and they managed to lose quite a few matches in the league before the season wrapped up.
Unranked (1500) - BlackBean - No matches against teams on the known ranked list to assign a meaningful rating.
The ratings did a good job recognizing the hot streak of WE and Fnatic (even if for the latter, it didn't quite know just how strong they would be). As I mention above, I thought CLG Prime was rated a bit high due to their incredible winning streak against NA teams leading up to the event.
Here's how the ratings predicted the group stage alongside the actual results in parens. Incorrect placements are italicized. I'm feeding the algorithm the correct participants for each match (rather than what the ratings originally predicted) and the updated ratings based on the previous matches.
- Azubu Blaze (3)
- Team WE (1)
- Fnatic (2)
- Team Dynamic (4)
3 out 5 matches predicted correctly. Incorrect predictions were Azubu Blaze over WE and Azubu Blaze over Fnatic.
- CLG NA (2)
- CLG EU (1)
- Team FeaR (3)
- IceLanD (4)
4 out 5 matches predicted correctly. Incorrect prediction was CLG NA over CLG EU.
- Curse EU (1)
- Team SoloMid (2)
- Meat Playground (4)
- Singapore Sentinels (3)
4 out 5 matches predicted correctly. Incorrect prediction was Meat Playground over the Singapore Sentinels.
- Taipei Assassins (1)
- Moscow 5 (2)
- Curse NA (3)
- Black Bean (4)
5 out 5 matches predicted correctly
Overall, the ratings correctly predicted 16 out of 20 matches (80%).
Here's how the ratings fared in the bracket stage for each set, again adjusted after previous sets and with the correct participants. Incorrect predictions (overall, 3 out of 22) in italics.
Losers Round 1 (3 out of 4)
- Team Fear over Meat Playground
- Azubu Blaze over BlackBean
- Curse NA over Team Dynamic
- Singapore Sentinels over IceLanD
Winners Quarterfinals (3 out of 4)
- Team WE over Moscow 5
- CLG EU over TSM
- CLG Prime over Curse EU
- Taipei Assassins over Fnatic (despite Fnatic's win over Azubu Blaze, they were still rated slightly lower than TPA due to their loss against WE, whereas TPA had the luxury of going 2-0 against M5 and BlackBean.)
Losers Round 2 (4 out of 4)
- Moscow Five over Meat Playground
- Azubu Blaze over TSM
- Curse NA over Curse EU
- Taipei Assassins over Singapore Sentinels
Losers Round 3 (1 out of 2)
- Azubu Blaze over Moscow Five
- Taipei Assassins over Curse NA
Winners Semifinals (2 out of 2)
- Team WE over CLG EU
- Fnatic over CLG Prime
Losers Round 4 (2 out of 2)
- Moscow 5 over CLG Prime
- TPA over CLG EU
Losers Round 5 (1 out of 1)
- TPA over Moscow 5
Winners Finals (1 out of 1)
- Team WE over Fnatic
Losers Finals (1 out of 1)
- Fnatic over TPA (By this point, Fnatic was now the slight favorites in ratings)
Grand Finals (1 out of 1)
- Team WE over Fnatic
Overall, I'm pretty pleased with how well the ratings held up in predicting matches, accurately predicting 35 out of the 42 sets played at IPL 5 (Azubu Blaze flaming out accounted for 3 of those wrong predictions, of which I'm not sure any amount of tweaking to the algorithm would have resulted in accurate predictions). Furthermore, when the two teams were rated close to each other, the sets often resulted in 2-1 decisions, validating the ratings. Next steps: play with the variables to see how they affect accuracy across all tournaments, and see if there's a way to weigh offline vs. online tournaments in a logical manner.