elo system
Most FGC tournaments follow the same format. Players compete in a double elimination format that contains sets of either best of three or best of five depending on the game, tournament round, and time allocated for the event. Tournaments generally run a few hours, and the night ends with one winner and a Top 8 graphic.
For CFGC, coming up with a format was much more complicated.
“When it comes to the league structure, I utilized scheduling techniques typically used in Operations Research for optimal inventory flow. Operation Research deals with batch processing with objectives of minimizing time, so as a theoretical basis, I took those theories and applied them to a fighting game tournament by thinking of players as inventory within the league system.”
Dr. Kurtis Homan is an Assistant Professor at St. Cloud State University in Minnesota, as well as the head of Research and Development at CFGC. He’s long focused on bracketology and tournament organizing, writing his PhD thesis on “Tournament predictive indicators and tournament subgame theory for Tekken 7”. He’s now implementing the theories he’s spent years studying to create a CFGC format that creates the best experience for everyone.
“In order to create appropriate batch sizes, or how many players should be in a group, I reflected on the time constraints that exists for the number of playable weeks in a semester. After some analysis, 10-weeks was selected that would work well for a college students’ semester and allow time for players to register and allow me to create split schedule in the 4-week seeding period, in which the range of players would follow more Monrad pairing compared to Burstein seeding that is commonly used in traditional tournaments. The Monrad approach, also known as aggressive pairing, utilizes what we can think of as skill-based match making. By recording past performance of players, this allows us to pair up these players together to have a ‘closer’ match.”
In essence, Dr. Homan designed a league structure that split the season in two: pre and post-seeding. Especially since many players competing don’t have much traditional tournament experience, providing potential seeding data, sectioning off several weeks for seeding data helps create a better competing experience for everyone.
“After the 4-week seeding period, players are readjusted for the ‘regular season’ that takes place for the remaining 6-weeks. These player pools are more restricted to minimize any player gaps to have more competitive games as the weeks transpire up until week 10 where some of our players have already named it ‘hell week’ having to face their closest match ups, all be design of the schedule. At the end of the 10-week season, players are anticipated to play against at least 20 unique players and have approximately 100 games total recorded.”
On top of preparing the tournament format, Dr. Homan has also given much consideration to the issue of players not showing up for their matches. Given that seeding is based on prior match history, having incomplete data can cause various issues going forward.
“A sad reality of many tournaments and leagues is the fact that players sign up, but do not attend their pool assignments. If a pool has two or three people assigned, it is possible that no games will be played. This semester I have been utilizing other analytics to ‘overbook’ pool ranges by making educated guesses of who will be the players who do not show up, with the intent at making an overall better play experience for the players that are scheduled and show up on time. Many of these techniques are supported by operation research as well. Continued analysis is something that is planned during the summer as we have collected player behavioral data throughout the league to determine what will be the best method to maximize player experience in the future.”
Another one of Dr. Homan’s considerations when planning CFGC’s format was the problems with traditional seeding and how that negatively affects lower-skilled players.
“In the FGC, double elimination (DE) tournaments use Burstein seeding, also known as traditional seeding, meaning the best player will be paired versus the lowest players. That system by observation can be OK for small groups, however the system begins to break down the experience for the early rounds of a DE tournament due to wider skill gaps. This makes it a bad experience for the higher rated player and lower rated player. The higher rated player is an ‘easy’ win so it’s a cost of time, and the lower rated player effectively is only playing against 1 possible opponent, if they go 0-2 which will happen to ¼ of all entries. This also means it creates large barriers of improvement if this DE tournament is a series of tournaments. The lower rating player will constantly face an overpowered opponent and will have a chance to have a closer opponent during losers side, compared to the chance of being evenly paired based on player skill.”
CFGC contains a large skill gap of players, which is part of what makes the collegiate experience exciting. Players that don’t have experience competing in traditional FGC weeklies are willing to compete in a collegiate-catered experience, so making a positive experience for those less-versed players was top of mind.
“To battle this problem, a rework of the traditional method of win/loss was used. Other games have either power ranking or qualitative approaches to evaluate players in the leagues. This qualitative approach doesn’t have its problems and could have bias either intended or unintended by those for seeding and match making purposes. This is where the thought to have a quantifiable measure to have an unbiased measurement of player performance. Looking at ranking systems, and having done prior research of player performance system, the simplest to test if it works in a league-like structure in the Elo system, as it has largely been researched and used in a variety of sports, games, and competitive video games.”
“The Elo system allows an improved skill match making experience for all levels of players when compared to previous approaches of ranking. The skill match matching is further used to create subgroups to minimize the range of skill gaps between players, meaning players will be paired with a wide range of unique players rather than seeing the same opponent’s week to week. An added benefit that has already been observed, for engagement for players across the board has had steady increase. In a DE tournament, maybe the top 8 will discuss their progress from week to week, but now with the Elo system, we have seen players across all skill levels make comments about their progress they have had with more detail than saying, ‘but it was close!’”
Since creating the initial format, Dr. Homan has been hard at work analyzing what’s been working and what could use further improvement.
“The system appears to be working,” he told me. “The time savings and having players of all the skill levels having a good experience. This is supported by our growing numbers from semester to semester. As we generate more people joining in the league, this will allow for future research projects to take place to make data driven decisions for future benefit of tournaments.”
“The main task to improve on is to automate process features. The key features that are in progress for automation are related to weekly fight cards (scheduling), update ratings as match results are entered, and to reschedule matches. Each of these conditions help make the league run smoother and should make players and school team managers happier about the process.”
All of this work on CFGC’s elo and scheduling system ties back to Dr. Homan’ work during his PhD program. Collecting more data points and experimenting with new systems is key to learning more about what works and what doesn’t in creating the best possible experience for players, organizers, and spectators alike.
“A large benefit of the CFGC is the amount of data that we are able to gather related to player behavior and game results data. Back in 2020 I worked on my dissertation for my PhD and found a limiting factor was the historical data for each player. For top players, finding match results during major tournaments, but for the rest of the player base of tournament players, there were not enough statistically significant matches that were recorded.”
“That prior research was a big influence on designing a system that could capture data of all levels of players for improvements related to seeding practices and player experience. After now collecting 3 semester’s worth of data using the Elo system, it is my intent to use the dissertation rating/prediction indicators that will be retro fitted behind the scene to measure accuracy/precision and the amount of needed required games to have an accurate measurement of a player’s rating that can be used for league play and a quantifiable metric that outside organizations could use for seeding/marketing purposes.”
Written by: Blinka
