Having participated in the 2021 and 2022 Gain Jams, I have a number of issues I hope to address with how the Jams have been run. I hope to outline problems I have noticed with the jam this year and propose some solutions to make the Jam more inviting as a friendly competition for developers to flex their skills and focus on making the best games they can under the criteria given.
In my response, I will reference Grot’s three major goals for the Jam:
- Challenge devs to improve their skills;
- Encourage development of rapid prototyping skills & try new concepts;
- Try to encourage new devs to at least try their hands at game dev.
ISSUES
WILD CARDS
Wild card, while an interesting addition, failed to either meet the three main goal or address the problems of last year. While the idea of a simple theme and optional, more complex wildcards are good in theory, any serious competitor finds themselves required to attempt them all due to how the games are scored. If not already inconvenient, judges would on occasion miss wildcard additions entirely, something that could be fixed by simply attaching a form to the corresponding post.While there is certainly fair challenge in implementing all three wild cards, all three cards leaning in a mechanical direction pulled too hard towards certain ideas. With the cards being mechanical in nature meaning that using it in a narrative sense would be potentially invalid. The basic pass/fail was presented more clearly, but it was unclear what would result in full marks or not.
For the goal of getting participants to try new concepts, it certainly paints clear goals of things to try, but announcing the wildcards weeks in advance seems to be actively opposed to the goal of rapid prototyping. To me, the excitement of a game jam is making an idea that did not exist before the timer started and having as complete and realized an product as possible by the time it’s over. I think it’s telling that Minions was such a prevalent idea in many games this year, with every subsequent wildcard falling off in usage. Rapid prototyping is directly undercut by having weeks of advance time to develop concepts before the competition.
As for encouraging new developers, it isn’t hard to imagine that having these additional rulings, some of which are only clarified by scouring through forum posts or participating in the discord, would present a noticeable barrier.
ADMIN RESPONSE
When asking questions on the forum and discord regarding the scoring of wildcards, as well as the distinction of fetish elements and their anomalous splitting apart and different scores in the second round, there were a series of problematic responses.Often, questions regarding the second round were answered by judges who had only been in charge of the first round. Though certain scoring areas were inevitably measured differently by every judge, a lack of cohesion contributed towards inequalities that ultimately cost points.
The role of Alex as community manager is perplexing if, when answering questions, everything is overridden by Grot regardless. This indicates a lack of communication within the admin team.
SCORING
Since last year, the problematic scoring system was functionally unchanged. Every score was dropped by a decimal point and wild cards were introduced, meaning that the actual weight of each category was unaltered. While the wildcards in theory would offset lower scores, anyone who wanted to maximize their scores is required to go for them.Judge’s Preference has been a mark of contention. The wiki currently defines this category as “Points that can be awarded based on a judge’s discretion.” which means any judge is free to use this however they want, giving them more freedom than the more defined definitions of other categories. Having fresh guest judges helps prevent the scoring from being a checklist and it’s unreasonable to think there’s a truly objective way to evaluate whether one game is better than the other.
With a guest judge’s atypical scoring approach for the majority of the games, this particular issue has become a major problem for participants. There was a non-zero amount number of game rankings that could have been altered by this category being filled in rather than ignored. While the Preference category is allowed to mean whatever it wants to the individual judge, that does not mean it doesn’t matter to the participant. A Preference score that is significantly above or below the average of their other scores, should signify that there are elements of the game that cannot be cleanly defined on the score sheet. When a judge hands out scores of zero, it says that they find nothing of value in the game. Numerous people were sullen by the thought that a judge disliked their game enough to give a zero. Giving a game a ten simply to boost it’s score makes the other points feel meaningless in comparison when they’re also worth the same amount. A game that already scored well should have no need for this. While many of the games in the top five did receive zeros in Preference, it’s about the feedback this gives, not the placement that results.
PROPOSED SOLUTIONS
WILD CARDS
I would recommend looking at how other jam integrate them into their competitions. Make all the cards mysteries up until the day the jam starts and require a form to be filled out to declare what wild cards are used and how they are used. Removing the ambiguity of inconsistently placed in-game cards and clarifying the intentions will help judges understand how the wild card is being used. If the community card poll remains, the results should be hidden.Furthermore, wildcards should be expanded to more areas than just mechanics. Settings, themes and other more creative directives would help prevent the overlap many cards have and the strain it places on a participant.
One final suggestion would be the removal of the admin card. If the administrative team wants to have a more challenging theme for the jam, then they should make it so rather than have an awkward double dip into the dictation. Having to include an expansion fetish element to the game is already a fourth theme in it’s own right. Main theme, fetish, and two wildcards feels more than enough to provide challenge to participants. If not, reducing the maximum bonus score to 10 instead of fifteen means a card can be safely ignored, or effort can be divided among them with less stress about how perfectly they are done compared to the main game.
ADMIN RESPONSE
The continued development of the Weight Gaming wiki will help alleviate these he said/she said scenarios that occur between admins and judges when scoring is questioned. In addition, I suggest appointing a member of the administrative team to be a jam director, someone who has a clear say on the rules and organization that both the other admins and participants can build their understanding off of. What's most important is that this person is NOT a judge, and serves purely to facilitate the jam.This would also remove the need for two round judging. While many object to the time it takes, the bigger complaint is that it is not fair for a pair of people to have a far more important say over the efforts of others. The opinions of a judge who played every single game and evaluated them are worth magnitudes more than a judge who plays a mere fraction of them.
SCORING
The mechanics section refers to the base aspects of the game function, not how engaging the actual gameplay is. A game that focuses on being a gameplay experience first and foremost can be at a deficit. In addition to mechanics being a measure of how well a game functions, game play should be added as a category for how well the game feels. While this may sound confusing, allow me to provide a pair of examples to help clarify;A world simulation game that has an elaborate set of environmental interactions that the player can trigger might be high on mechanics, but if it’s dull to watch these interactions play out, then it would be low on gameplay.
Likewise, a single button rhythm game based around following a call and response pattern would be rather low on the mechanics, but if the rhythms are interesting, if well timed pressed result in interesting scenarios, if failure to keep time results in negative feed back, then that would be high on gameplay.
If gameplay is introduced as a category, my two recommendations would be reducing mechanics to 5 points while gameplay is 10, and include a “Best in Gameplay” award.
Renaming the writing category to narrative would allow for less strict evaluations of what is literally written word vs what is story. Narrative is the emotions conveyed by the medium, which in videos games can be done with literal story told through words, voices, and environments, but also gameplay. This can allow more arcade style games to shine in scoring. A puzzle game that gets increasingly chaotic with it’s mechanics and visuals can still convey a narrative more engaging than, say, an rpg with dull story beats and uninteresting characters.
Finally, Judge’s Preference needs to be altered. This category is valuable for allowing a judge to give feedback on elements that aren’t merely tied to scores or represent a synergy between different factors. A clearer definition of “Judge’s personal response to the game” would help, and this score should be reduced by 50%. There is no reason a game should be docked a full category’s worth of points simply due to personal preference with no accountability.
PARTING THOUGHTS
The Gain Jam is a meaningful event for me; it’s where I got my start in game development last year. Last year, I voiced confusion about the winner of Best Writing not being publicly announced. My concerns were brushed aside and later I was given a half-hearted apology. This year, I asked why the second round of judging had multiple fetish categories each worth double the normal amount. My concerns were brushed aside and later I was given a half-hearted apology.
My greatest desire in commenting on these and various issues is to have problems addressed before they result in errors. Seeing the confusing changes made to the jam this year makes me feel like the wrong lessons have been learned from the 2021 Jam, changes made that seem deaf to the desires of participants entering the jams. The developer response survey is a fantastic step in the right direction but with it’s limited format and leading answers, I fear will not allow the full extent of participant frustration to be expressed and addressed.
Forty-four entries times by two weeks equals nearly two years worth of development time across dozens of people and their teams and that effort need to be respected. I know I and many others took time off of work and other commitments to try and make the best games we could, but it’s with a heavy heart that I have to say that without meaningful change it is simply not worth my time to participate next year.